Business Systems Modernization: IRS Needs to Better Balance
Management Capacity with System Acquisition Workload (28-FEB-02,
GAO-02-356).
GAO reviewed the Internal Revenue Service's (IRS) fifth
expenditure plan requesting $391 million from its Business
Systems Modernization (BSM) fund. Although IRS's November 2001
expenditure plan satisfied the conditions specified in the
appropriations act, IRS must still fully implement the controls
and capabilities described in the plan. Since GAO's June 2001
report, IRS has made important progress in implementing
modernization management controls and capabilities and addressing
GAO's past recommendations. However, IRS's modernization
management capacity is still not where it needs to be. Examples
of modernization management controls and capabilities that are
not yet fully implemented include software acquisition
management, configuration management, quality assurance, risk
management, enterprise architecture implementation, human capital
management, integrated program scheduling, and cost and schedule
estimating. The increased risk of IRS's proceeding without these
controls and capabilities has contributed to project cost,
schedule, and performance shortfalls. IRS acknowledges that it
needs to strengthen its modernization management controls. IRS
recognizes that these controls become more critical as the size
and complexity of the BSM program continue to increase. Although
IRS has actions underway to fully implement these controls, IRS
plans to compensate for their immaturity by applying experienced
human capital. Reliance on a combination of existing immature
processes and individual expertise and heroic efforts is a
short-term solution to a long-term need. Because the immaturity
of these controls has already led to project cost, schedule, and
performance shortfalls, IRS needs a better strategy to mitigate
the risks associated with its fifth expenditure plan.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-02-356
ACCNO: A02807
TITLE: Business Systems Modernization: IRS Needs to Better
Balance Management Capacity with System Acquisition Workload
DATE: 02/28/2002
SUBJECT: Appropriated funds
Information systems
Internal controls
Performance measures
Strategic planning
Systems development life cycle
IRS Business Systems Modernization
Program
IRS Customer Account Data Engine Project
SEI Software Acquisition Capability
Maturity Model
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-02-356
United States General Accounting Office
GAO Report to Congressional Committees
February 2002
BUSINESS SYSTEMS MODERNIZATION
IRS Needs to Better Balance Management Capacity with Systems Acquisition
Workload
a
GAO-02-356
United States General Accounting Office February 2002
Accountability Integrity Reliability Management
Capacity with Systems Acquisition
Highlights Workload
Highlights of GAO-02-356, a report to the Subcommittee on Treasury and
General Government, Senate Committee on Appropriations and the Subcommittee
on Treasury, Postal Service and General Government, House Committee on
Appropriations.
Why GAO Did This Study What GAO Found
The Internal Revenue Service IRS's November 2001 expenditure plan defines
the next incremental set (IRS) has underway a multiyear, of management and
systems acquisition activities that are intended to multibillion-dollar
business move IRS closer to its systems modernization. Related to the
activities systems modernization program. proposed in this plan, IRS has to
date made important progress in Since 1995, GAO has designated establishing
the systems infrastructure needed to allow the introduction this program
high-risk, in part of future business application systems. Similar progress
has also been because of its size and made in establishing the key
modernization management controls complexity and its immense needed to
achieve the kind of systems acquisition activities provided for importance
to improving IRS in this plan. Nevertheless, as IRS moves forward to
implement the plan, it mission performance and faces increasing risk that it
will be unable to deliver promised system accountability. The Congress has
capabilities on time and within budget. Reasons for this include: mandated
that GAO review IRS's
plans for spending funds on this � IRS lacks certain modernization
management controls and program. This study responds to capabilities. For
example, it lacks a human capital management that mandate by providing the
strategy for the program and a reliable project cost-estimating results of
GAO's review of IRS's process. It has also yet to implement mature systems
and software November 2001 expenditure plan. acquisition process controls;
its plans for correcting this allow core
modernization projects to continue for at least 2 more years without such
controls.
� Interdependencies among ongoing systems acquisition projects and the
complexity of associated activities to be performed during the period
covered by the plan are an order of magnitude greater than
What GAO Recommends those of the previous plan. This workload increase
raises concern,
To address the escalating risks because IRS was not able to meet many of the
project cost and
facing IRS in this critical program, schedule commitments made in the less
demanding prior plan.
GAO recommends that the IRS � IRS's plan provides for increasing its
workload by starting additional
commissioner reconsider the projects that will further tax existing
capability.
planned scope and pace of the
program as defined in the In attempting to balance the need for introducing
modern systems and
November 2001 expenditure plan. their associated benefits as soon as
possible, with the need for greater
This is necessary to better management capacity to ensure that these systems
are introduced
balance the number of systems successfully, IRS's November 2001 plan
emphasizes the former. This
acquisition projects underway and emphasis adds significant risk that
escalates as the program advances.
planned with IRS's capacity to
manage this workload. GAO is
providing specific
recommendations to the
commissioner on how to strike M anagem ent
such a balance. In response, the Capability
commissioner agreed with GAO's
recommendations, and described
planned and ongoing efforts to
address each.
Project W orkload
This is a test for developing highlights for a GAO report. The full report,
including GAO's objectives, scope, methodology, and analysis, is available
at www.gao.gov/cgi-bin/getrpt?GAO-02-356. For additional information about
the report, contact Randolph C. Hite (202-512-3439). To provide comments on
this test highlights, contact Keith Fultz (202-512-3200) or E-mail
[email protected].
Contents
Letter 1
Recommendations for Executive Action 3
Agency Comments 4
Appendixes
Appendix I: Briefing Slides from December 10 and 11, 2001, Briefings to
the Senate and House Appropriations Subcommittee Staffs 6
Appendix II: Comments from the Internal Revenue Service 81
Appendix III: GAO Contacts and Staff Acknowledgments 85
GAO Contact 85
Acknowledgments 85
Abbreviations
BSM Business Systems Modernization
IRS Internal Revenue Service
OMB Office of Management and Budget
SA-CMM Software Acquisition Capability Maturity Model
A
United States General Accounting Office Washington, D.C. 20548
February 28, 2002
The Honorable Byron L. Dorgan
Chairman
The Honorable Ben Nighthorse Campbell
Ranking Minority Member
Subcommittee on Treasury and General Government
Committee on Appropriations
United States Senate
The Honorable Ernest J. Istook, Jr.
Chairman
The Honorable Steny H. Hoyer
Ranking Minority Member
Subcommittee on Treasury, Postal Service
and General Government
Committee on Appropriations
House of Representatives
Pursuant to the Department of the Treasury's fiscal year 2002
appropriations act, the Internal Revenue Service (IRS), in November 2001,
submitted to the congressional appropriations committees its fifth
expenditure plan, requesting $391 million from its Business Systems
Modernization (BSM) fund.1 As required by the act, we reviewed the plan.
Our objectives were to (1) determine whether the plan satisfied the
conditions specified in the act,2 (2) determine IRS's progress in
implementing modernization management controls and capabilities, and
(3) provide any other observations about the plan and IRS's BSM program.
On December 10 and 11, 2001, we briefed your respective offices on the
results of our review. This report transmits the materials used at those
briefings, and reiterates the recommendations to the commissioner of
internal revenue that we specified in the briefings. The full briefing
1 The Treasury and General Government Appropriations Act, 2002 (P. L.
107-67).
2 The act specifies that BSM funds are unavailable until IRS submits to
congressional appropriations committees for approval a modernization
expenditure plan that (1) meets the Office of Management and Budget's (OMB)
capital planning and information technology investment control review
requirements; (2) complies with IRS's enterprise architecture; (3) meets
IRS's life-cycle management requirements; (4) is approved by IRS, Treasury,
and OMB; (5) is reviewed by GAO; and (6) complies with federal acquisition
requirements and management practices.
materials, including our scope and methodology, are reprinted in appendix
I. In summary, we made the following five major points:
* IRS's November 2001 expenditure plan satisfied the conditions specified in
the appropriations act. However, while the plan provided for satisfying
these conditions, IRS must still fully implement the controls and
capabilities described in the plan.
* Since our June 2001 report,3 IRS has made important progress in
implementing modernization management controls and capabilities and
addressing our past recommendations. Nevertheless, IRS's modernization
management capacity is still not where it needs to be, given (1) the number
of systems acquisition projects that the November 2001 plan identifies as
being underway, (2) the fact that several of these ongoing projects have
already entered the critical building stage of their life cycles (milestone
3) and are to begin deployment (milestone 4) during this year, and (3) IRS's
plan to begin additional projects. Examples of modernization management
controls and capabilities that are not yet fully implemented include
software acquisition management,4 configuration management,5 quality
assurance, risk management, enterprise architecture6 implementation, human
capital management, integrated program scheduling, and cost and schedule
estimating.
* The increased risk of IRS's proceeding without these controls and
capabilities has contributed to actual project cost, schedule, and
performance shortfalls. For example, in the fifth plan, IRS reports that the
Customer Account Data Engine's (release 1) milestone 4 date has been delayed
by 6 months, and its cost has increased by $5 million (13
3 U.S. General Accounting Office, Business Systems Modernization: Results of
Review of IRS's March 2001 Expenditure Plan, GAO-01-716 (Washington, D.C.:
June 29, 2001).
4 Carnegie Mellon University's Software Engineering Institute has developed
criteria, known as the Software Acquisition Capability Maturity Model?
(SA-CMM?), for determining organizations' software acquisition management
effectiveness or maturity. Capability Maturity Model and CMM are registered
in the U.S. Patent and Trademark Office.
5 Configuration management is the means for ensuring the integrity and
consistency of systems modernization programs and project products
throughout their life cycles. Through effective configuration management,
for example, integration among related projects and alignment between
projects and the enterprise architecture can be achieved.
6 An enterprise architecture is an institutional blueprint defining how an
enterprise operates today, in both business and technological terms, and how
it wants to operate in the future. It also includes a roadmap for
transitioning between these environments.
percent above the fourth plan's funding level). Also, since submitting the
fifth plan for approval, IRS announced that the Security and Technology
Infrastructure Release's milestone 4 date has been delayed by 3 months, and
its cost has increased by $6 million (24 percent above the fourth plan's
funding level).
* IRS acknowledges the need to strengthen its modernization management
controls, and recognizes that these controls become more critical as the
size and complexity of the BSM program continues to increase. It also has
actions underway to fully implement these controls and, until then, plans to
compensate for their immaturity by applying experienced human capital.
* In our view, reliance on a combination of existing immature processes and
individual expertise and heroic efforts is a short-term solution to a
long-term need. Given that the immaturity of these controls has already
contributed to project cost, schedule, and performance shortfalls, and will
likely continue to do so, IRS needs a better strategy for mitigating the
risks it faces in implementing its fifth expenditure plan. To assist IRS in
striking a proper balance between the need to quickly introduce modernized
systems yet prudently manage the risks inherent in such an undertaking, we
made the following recommendations to the commissioner of internal revenue.
Recommendations for Executive Action
To address the escalating risks facing IRS on its BSM program, we recommend
that the commissioner of internal revenue reconsider the planned scope and
pace of the BSM program as defined in the fifth expenditure plan, with the
goal of better balancing the number of systems acquisition projects underway
and planned with IRS's capacity to manage this workload. At a minimum, the
commissioner's reconsideration should include
* slowing ongoing projects and delaying new project starts to reduce
Business Systems Modernization Office resource demands,
* making correcting modernization management weaknesses a top priority and a
matter of top management attention, and
* reapplying resources-financial and human capital-available from slowed and
delayed projects toward correction of control weaknesses.
We further recommend that the commissioner take the following actions with
respect to each of the modernization management weaknesses that we
identified.
First, for software acquisition management,
* immediately assess critical BSM projects (i.e., Customer Account Data
Engine, Security and Technology Infrastructure Release, and e-Services)
against the Software Engineering Institute's Software Acquisition Capability
Maturity Model? (SA-CMM?) level 2 requirements;
* based on this assessment, develop a plan for correcting identified
weaknesses for these projects, including having an independent SA�CMM
evaluation performed on them before submission of the next BSM expenditure
plan;
* submit, with the next expenditure plan, the results of this independent
evaluation, along with a plan for ensuring that all BSM projects that have
passed milestone 3 will meet SA-CMM level 2 requirements; and
* require all projects that did not pass milestone 3 as of December 31,
2001, to be assessed as SA-CMM level 2, and have a plan for correcting any
project weaknesses found as a condition of milestone 3 approval.
Second, for configuration management, risk management, enterprise
architecture implementation, human capital strategic management, integrated
program scheduling, and cost and schedule estimating, ensure that
commitments discussed herein for addressing residual weaknesses are
implemented as planned, and report any deviations from these planned
commitments to IRS's appropriations subcommittees.
Third, until contractor quality assurance weaknesses are corrected, increase
the level of IRS oversight, scrutiny, and quality assurance of contractor
activities.
Finally, to allow for effective congressional oversight of the program, we
reiterate our prior recommendation that the commissioner report to IRS's
appropriations subcommittees any changes to expenditure plan commitments
concerning systems requirements/capabilities to be delivered and the
associated benefits to be realized, and continue to report such performance
measures in future expenditure plans.
Agency Comments In commenting on a draft of this report, the commissioner of
internal revenue agreed with our recommendations, adding that IRS leadership
understands the importance of correcting the issues that we identified and
is moving aggressively to resolve them. In this regard, the commissioner
described IRS's ongoing and planned efforts relating to addressing each of
our recommendations. The commissioner's written comments are reprinted in
appendix II.
We are sending copies of this report to the chairmen and ranking minority
members of other Senate and House committees and subcommittees that have
appropriations, authorization, and oversight responsibilities for the
Internal Revenue Service. We are also sending copies to the commissioner of
internal revenue, the secretary of the treasury, the chairman of the IRS
oversight board, and the director of the Office of Management and Budget.
Copies are also available at our Web site at www.gao.gov.
Should you or your offices have questions on matters discussed in this
report, please contact me at (202) 512-3439. I can also be reached by e-mail
at [email protected]. Key contributors to this report are listed in appendix
III.
Randolph C. Hite Director, Information Technology Architecture
and Systems Issues
Appendix I
Briefing Slides from December 10 and 11, 2001, Briefings to the Senate and
House Appropriations Subcommittee Staffs
Results of Review of IRS' November 2001 Business Systems Modernization
Expenditure Plan
Briefing to the Staffs of
the Senate Committee on Appropriations,
Subcommittee on Treasury and General Government
(on December 11, 2001)
and
the House Committee on Appropriations,
Subcommittee on Treasury, Postal Service,
and General Government
(on December 10, 2001)
1
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
* Introduction
* Objectives
* Scope and Methodology
* Background
* Results in Brief
* Results
* Conclusions
* Recommendations
2
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
* As mandated by IRS' FY 2002 appropriations act, Business Systems
Modernization (BSM) funds are unavailable until IRS submits to the
congressional appropriations committees for approval, a modernization
expenditure plan that:
* Meets the Office of Management and Budget's (OMB) capital planning and
information technology (IT) investment control review requirements;
* Complies with IRS' enterprise architecture;1
* Meets IRS life cycle management requirements;2
* Is approved by IRS, Treasury, and OMB;
* Is reviewed by GAO; and
* Complies with federal acquisition requirements and management practices.
* Since mid-1999, IRS has submitted a series of expenditure or "spending"
plans requesting release of BSM appropriated funds.
1An Enterprise Architecture (EA) is an institutional blueprint defining how
an enterprise operates today, in both business and technology terms, 3and
how it wants to operate at some point in the future. An EA also includes a
roadmap for transitioning between these environments. 2IRS refers to its
life cycle management program as the Enterprise Life Cycle (ELC), which is
graphically depicted in the Background Section.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
* To date, about $968 million has been appropriated for BSM, including about
$391 million for FY 2002. Of the $968, $577 million has been released; about
$391 million remains.
* On November 13, 2001, IRS submitted its fifth plan, seeking release of the
remaining $391 million.
* If the plan is approved, the BSM fund will have a zero balance. To
replenish the fund, IRS plans to request $450 million via its FY 2003 budget
request.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
* As agreed with IRS' appropriations subcommittees, our objectives were to
* determine whether the fifth expenditure plan satisfies the legislative
conditions,
* determine what progress IRS has made in implementing modernization
management controls and capabilities, and
* provide any other observations about the fifth plan and IRS' BSM program.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Scope and Methodology
* To accomplish our objectives, we
* Reviewed the fifth expenditure plan and met with IRS program officials to
understand the scope and content of the plan;
* Analyzed the plan against the legislative conditions to identify any
variances;
* Reviewed program and project management reports and briefings to assess
progress in implementing modernization management controls and capabilities;
* Observed modernization executive steering committee and subcommittee
meetings to, among other things, document how the plan was developed and
reviewed;
* Interviewed program and project management officials to corroborate our
understanding of the plan and other BSM activities.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Scope and Methodology
* Analyzed available evidence on recent efforts to implement modernization
management controls and capabilities. Specifically, we analyzed progress and
plans for
* enterprise architecture (EA) definition and implementation,
* ELC definition and implementation, including configuration management,
quality assurance and risk management,
* software acquisition maturity, as defined by the Software Engineering
Institute's (SEI) Software Acquisition Capability Maturity
Model[tm](SA-CMM),
* human capital management,
* cost and schedule estimating practices,
* integrated program schedule development, and
* key projects, such as the Security and Technology Infrastructure Release
(STIR), the Customer Account Data Engine (CADE), e-Services, and the
Internet Refund and Fact of Filing project.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Scope and Methodology
* Collaborated with the Treasury Inspector General for Tax Administration
(TIGTA) to avoid duplication of effort in reviewing BSM initiatives and
incorporated TIGTA results in this briefing where appropriate.
* Projects addressed by TIGTA included Customer Communications, e-Services,
the Telecommunications Enterprise Strategic Program, and Customer
Relationship Management-Exam.
* Program-level processes addressed in a recent capping report on key system
development processes (e.g. requirements management, configuration
management, risk management).
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Scope and Methodology
* As agreed with your offices, we did not independently validate planned
initiatives' cost estimates or confirm, through system and project
management documentation, the validity of IRS-provided information on the
initiatives' content and progress.
* We provided a draft of this briefing on December 7, 2001, to IRS BSM
program executives (Deputy Associate Commissioners for Program Management
and Systems Integration and the Executive Program Advisor for Risk
Management), and have incorporated their comments where appropriate.
* We performed our work from October through December 2001 in accordance
with generally accepted government auditing standards.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Summary of IRS' Fifth Expenditure Plan ($000)3
Program Management and Architecture Activities
Program Management Office $ 7,918 Business Transformation Planning $ 10,461
Architecture & Systems Integration $ 32,539 Quality Management and Assurance
$ 10,082 Federally Funded Research and Development Center -MITRE $ 18,070
Subtotal $ 79,070
Project Level and Infrastructure Activities4
Core Infrastructure Support Projects (e.g. STIR) $107,959 Business Systems
Support Projects ( Customer Account Data Engine, $112,224 Custodial
Accounting Project, Core Financial Systems) Business Systems Projects $
89,686
Subtotal $309,869
Addition to Management Reserve $ 2,061
Total $391,000
Source: IRS
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
* IRS' plan is to (1) continue ongoing program-level initiatives through
mid-November 2002 and 16 ongoing projects to their next milestones and (2)
start 6 new projects. Examples of new projects are
* Reporting Compliance, which is to, among other things, select returns for
examination
* Filing and Payment Compliance, which is to enable access to taxpayer data
in CADE and the Custodial Accounting Project system to determine compliance
and allow taxpayers to resolve account issues electronically
* Compared to the total BSM funds requested in the third and fourth plans
combined, the fifth plan represents an 18 percent decrease in program
management spending and a 27 percent increase in project acquisition
spending.
* The third and fourth plans generally provided funding for FY 2001, and the
fifth plan does so for FY 2002.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
* Like its previous plans, IRS' fifth expenditure plan covers contractor
costs, such as the Prime Systems Integration Support (PRIME) contractor and
the systems engineering and technical assistance contractor (MITRE), and not
IRS internal costs, such as IRS BSM program office (BSMO) staff costs. As we
previously reported,5
* IRS' actual use of prior BSM funding has been limited to the modernization
program
* IRS' actual use of prior IS funding has included modernization activities.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Summary of Prior GAO Expenditure Plan Reviews
* To date, GAO has reviewed and reported on six requests for BSM funding
releases.6
* Since mid-1999, we have reported7 on the risks associated with IRS'
approach of concurrently building systems while developing and implementing
program management capabilities such as having a fully operational program
management office and implementing the ELC.
6For details on our past review results, see appendix II.
7For example, see Business Systems Modernization: Results of Review of IRS'
March 2001 Expenditure Plan (GAO-01-716, June 29, 2001). 15
and Internal Revenue Service: Progress Continues But Serious Management
Challenges Remain (GAO-01-562T, April 2, 2001).
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
* In summary, we reported that attempting to acquire modernized systems
before having the requisite management capability increases the risk that
systems will experience cost, schedule, and performance shortfalls.
* EA, configuration management, quality assurance, and risk management, are
but four of many management controls required under IRS' ELC, which is a
structured method for managing system modernization program and project
investments throughout their life cycles (see below simplified diagram of
ELC).
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
* We have also reported8 that the risks associated with building systems
without the requisite management controls are not as severe early in
projects' life cycles when they are being planned (project definition and
preliminary system design), but escalate as projects are built (detailed
design and development) and implemented (enterprise deployment).
* In the case of IRS and its ELC, this point of risk escalation is ELC
Milestone 3, as is shown in the following graphic. From this point through
deployment (Milestone 4) to operations and support (Milestone 5), risk can
increase significantly.
* In June 2001 report,9 we identified key IRS projects that were approaching
or had passed Milestone 3 that were beginning to experience such cost,
schedule, and performance shortfalls, and concluded that program risks were
increasing (see graphic on next page).
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results in Brief
* Nevertheless, IRS' modernization management capability is still not where
it should be given (1) the number of acquisition projects that are underway,
(2) the fact that several of these projects have already entered the
critical building stage of their life cycles (Milestone 3) and are to begin
deployment (Milestone 4) during calendar year 2002, and (3) IRS' plan to
begin additional projects.
* Modernization management controls and capabilities that are not yet fully
implemented are
* Software acquisition management
* Configuration management
* Quality assurance
* Risk management
* EA implementation
* Human capital management,
* Integrated program scheduling, and
* Cost and schedule estimating
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results in Brief
* The increased risk of proceeding without these controls and capabilities
is now contributing to actual project cost, schedule, and performance
shortfalls. For example,
* In the fifth plan, IRS reports that CADE's (Release 1) Milestone 4 date
has slipped by 6 months, and its cost has increased by $5 million (13% above
the fourth plan's funding level), and
* Since submitting the fifth plan for approval, IRS announced that STIR's
Milestone 4 date has slipped by 3 months, and its cost has increased by $6
million (24% above fourth plan funding level).
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results in Brief
* Since IRS submitted its fifth plan, we shared our findings and conclusions
with IRS. In response, IRS
* acknowledges the need to strengthen its modernization management controls,
* recognizes these controls are more critical as the size and complexity of
the BSM program continues to increase, and
* has efforts underway intended to fully implement these controls, and until
then, plans to compensate for their immaturity by applying experienced human
capital.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results in Brief
* In our view, reliance on a combination of existing immature processes and
individual expertise and heroic efforts is a short-term solution to a
long-term need. Given that these controls' immaturity have already
contributed to project cost, schedule, and performance shortfalls, and will
likely lead to future shortfalls, IRS needs a better strategy for mitigating
the risks it faces in implementing its fifth expenditure plan.
* To assist IRS in striking a proper balance between the need to quickly
introduce modernized systems and prudently manage the risks inherent in
doing so, we are making recommendations to the Commissioner of Internal
Revenue.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
Legislative
Conditions Expenditure Plan Provisions
IRS' fifth expenditure plan
provides for managing
investments as part of a
portfolio through its
Investment Decision
Management process. This
includes conducting periodic
portfolio reviews to assess
1.� Meets the changes in business
OMB capital planning priorities and project
and IT investment schedules. The fifth plan
control review provides for such an
requirements. assessment in January 2002.
The fifth plan also provides
for IRS to revise and
implement its business case
guidance to achieve better
system investment decisions
based on compelling
return-on-investment
justifications.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
Legislative
Conditions Expenditure Plan Provisions
4.�
Approved by
IRS,
Treasury, and *� IRS - September 17, 2001 *� Treasury - October 19, 2001 *� OMB - October 29, 2001 *� Submitted to IRS' appropriations subcommittees - November
OMB. 13, 2001
*� GAO - December 10, 2001 briefing to IRS'
5.� House appropriation subcommittee - December 11,
Reviewed by 2001 briefing to IRS'
GAO. ���������������������Senate
appropriation subcommittee
6.�
Complies with
the
acquisition
rules,
requirements,
guidelines,
and systems
acquisition
management
practices of As part of the ELC, IRS has defined processes, roles, responsibilities, etc. for implementing Software Engineering Institute (SEI) Software Acquisition Capability Maturity
the federal ModelTM practices within the level 2 key process areas. 12 These practices are consistent with federal acquisition requirements and management practices, and the plan calls
Government. for implementation of the ELC on all projects.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* Since we reported on IRS' last plan,13 IRS has made important progress in
implementing modernization controls and capabilities and addressing our
recommendations.
* However, key controls and capabilities are still not fully implemented.
�����������������������������������������������
�����������������������������������������������
����������������������������������������
��
���������������������������������������������������� ����������
��������������� ``
�� ������������������������������������������ 65533;��� ����������
����������������������������������������
�� ����������������������������������������������
�������������������������������������������� onica,
��������������������������������������������������
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
�����������������������������������������������
�����������������������������������������������
������������������������������������������������
�����������������������������������������������
�������������������������
�� ������������������������������������������������
�������������������������������������������������
�� �����������������������������������������������
�������������������������������������������������
��
�������������������������������������������������������
�����������������������������������
�� ���������������������������������������������������
�������������������������������������������������
������������������������������������������������
�� ������������������������������������������������� #65533;
������������������
�� ����������������������������������������������
�������������������������� 65533;����������
����������������
�� �������������������������������������������������
�� ����������������������������������������������������
��������������
�� ������������������������������������������
���������������������������� �� ���������������
�� ����������������������������������������������
�������������������������������������������������
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
IRS Has Not Implemented Effective Software Acquisition
* The Clinger-Cohen Act requires the establishment of effective IT
management processes. SEI's Software Acquisition Capability Maturity
Model[tm] defines such processes for managing software acquisitions. Since
1995, we have recommended that IRS establish, at a minimum, the "repeatable"
level of SEI's software acquisition management processes (Level 2).14
* IRS has not completed implementation of SEI's Level 2 software processes,
although it committed in its fourth plan to do so by September 2001.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* IRS officials attribute the delay to their underestimation of how long it
takes to implement the practices.
* To effectively implement mature acquisition management processes, an
organization needs to
* first assess its strengths and weaknesses and
* then develop a plan for leveraging existing strengths and correcting
weaknesses, paying special attention to projects that are most critical and
vulnerable to weaknesses.
* To address its acquisition management control weaknesses, IRS has recently
* engaged SEI to help better estimate how long this will take, and
* developed a plan for implementing these practices which includes
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* focusing first on two new BSM projects,
* having these two projects internally assessed by June 2002,
* having these two projects independently evaluated by December 2002, and
* implementing the practices on remaining BSM projects during FY 2004.
* In effect, this plan means that IRS will not implement mature acquisition
processes on key projects until after initial system increments have passed
critical, later life cycle phases. As a result, IRS will continue to run the
risk that these projects' promised capabilities will not be delivered on
time and within budget.
* The significance of this risk is magnified in light of the multiple
dependencies that exist among projects, where for example, delays in one can
cause cascading delays in others.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
IRS Has Made Important Progress, But Does Not Yet Have Effective
* Effective configuration management is an essential control for ensuring
the integrity and consistency of system modernization program and project
products throughout their life cycles.
* In June 2001, we reported15 BSM configuration management was ineffective.
Accordingly, we made recommendations to address this weakness.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* IRS has made important progress in addressing our recommendations. For
example, it has
* identified configuration items and components to be controlled for BSM
projects,
* had the PRIME revise its program configuration management plan and
procedures, which was completed in July 2001,
* trained contractor and selected BSM program office personnel in
configuration management,
* established baselines for approved program and project products,
* initiated periodic internal configuration management process audits on
selected projects to ensure plans and procedures are being followed, and
* established configuration control boards, including defining thresholds
governing which board should address which request.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* However, effective configuration management, as we previously recommended,
requires an enterprise-wide configuration management process, including
plans and procedures governing PRIME, BSM program office, and IRS IT
services organization products and systems. At a minimum, this also involves
* establishing an integrated program-wide configuration library (i.e., a
repository),
* defining and implementing a program-wide configuration status account
reporting mechanism, and
* identifying configuration items and components for current (legacy)
systems affected by near-term (2002 and 2003) BSM project releases.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* IRS officials acknowledge these steps have not been completed but have
plans to do so by March 2002, except for the 2003 project releases, for
which a date has not yet been established.
* IRS officials also stated that these steps have not yet been fully
implemented because all the steps needed to implement an effective
configuration management process were not originally understood.
* While IRS has made progress in implementing effective configuration
management, until it fully does so, IRS cannot adequately assure that
systems are being designed and developed in accordance with enterprise-wide
needs and requirements, and thus the likelihood that projects will
eventually require expensive re-work is increased.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
IRS Quality Assurance Efforts Have Identified PRIME Problems
* Quality assurance (QA) provides independent assessments of whether
management process requirements are being followed and whether product
standards and requirements are being satisfied.
* In 2000, IRS established a QA organization within its BSM program office
to, among other things, determine how well
* the BSM program office was following ELC acquisition management processes
and product standards and
* the PRIME's QA function was performing its role of assessing the PRIME's
adherence to quality processes and standards in producing BSM deliverables.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* IRS reviewed the PRIME's QA function, and in March 2001, reported 5 major
findings and concluded that the PRIME's QA was not always effective.
Examples of findings were that the PRIME's QA
* had not planned its work adequately, which contributed to inadequate
process audits and product reviews,
* had not defined program and project metrics for assessing process and
product quality, and
* was not organizationally independent.
* To address these weaknesses, IRS defined 24 corrective actions for the
PRIME. As of November 2001, IRS reported that the PRIME had implemented 7 of
the corrective actions, but 17 remained open. Examples of open actions are
* Ensure that QA reviews and approves deliverables before they are
transmitted to IRS
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* Require PRIME program and project quality plans to adhere to ELC
standards.
* Until the PRIME corrects these weaknesses, the probability of PRIME
deliverables not meeting expectations is unnecessarily increased, and the
level of IRS contract oversight and control needed to compensate for these
weaknesses increases.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Despite Progress, Risk Management Not Yet Fully Implemented
* A risk represents a potential problem. The purpose of risk management is
to identify program and project risks before they result in a problem,
appropriately respond to a risk based on its significance (probability of
occurrence times its potential impact), and to actively manage it.
* Effectively managing these risks is one way to minimize the chances of
program and project cost, schedule, and performance problems occurring.
Consistent with our efforts to constructively engage with IRS on BSM, in
1999 we orally recommended to IRS' former CIO that IRS implement this
capability across the BSM program and projects.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* IRS has defined risk management policies and procedures for its ELC. These
included processes to
* identify risks,
* evaluate probability of occurrence and potential impact on each project
and across the BSM program,
* develop risk mitigation plans and track their disposition,
* establish a repository to maintain an inventory of identified risks, and
* report on the status of risks and mitigation plans.
* Despite these steps, IRS has yet to fully implement its risk management
program policies and procedures. For example,
* IRS' inventory of BSM risks does not identify all known risks (e.g. lack
of mature software processes, lack of effective configuration management).
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* Risk plans and their disposition are not in all cases being tracked and
updated.
* To strengthen risk management, IRS hired a senior executive in June 2001
to implement effective risk management. This official
* attributed IRS' delay in implementing risk management to a lack of
incentives to report risks, and
* created an action team to review IRS' risk management procedures and
develop a plan for "revitalizing" the program, including incorporating
incentives for escalating risks.
* IRS plans to have risk management fully implemented by February 2002.
* Until this is done, IRS cannot assure it is adequately managing program
and project risks. Thus, the likelihood of BSM projects experiencing further
cost, schedule, and performance shortfalls is increased.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
IRS Has Established Processes To Certify Alignment of BSM Projects
* EA is an institutional blueprint defining how an enterprise operates
today, in both business and technology terms, and how it wants to operate at
some point in the future. An EA also includes a roadmap for transitioning
between these environments.
* Since 199516, we have recommended that IRS
* define and implement an EA to guide and constrain the acquisition of its
modernized systems and
* map BSM projects to the EA to ensure they are built in accordance with the
EA, which reduces the risk of expensive rework, especially after projects
have begun detailed design and development.
* We reported in our June 2001 report17 that while IRS has developed EA
versions 1.0 and 1.1, it had not yet performed the requisite mappings.
16For example, Tax Systems Modernization: Management and Technical
Weaknesses Must Be Corrected If Modernization Is to Succeed 43
(GAO/AIMD-95-156, July 26, 1995).
17Business Systems Modernization: Results of Review of IRS' March 2001
Expenditure Plan (GAO-01-716, June 29, 2001).
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* Accordingly, we also recommended that IRS not approve projects exiting
Milestone 3 until the required assessments were performed to certify that
projects are aligned with the EA.
* IRS has taken important steps to implement this recommendation. For
example
* In September 2001, IRS issued processes for certifying project compliance
with the EA at Milestone 3, before projects proceed with detailed design and
development.
* Nevertheless, IRS has not yet certified all BSM projects that are at or
past Milestone 3 via this process and the current EA version, including
* Security and Technology Infrastructure Release,
* Internet Refund and Fact of Filing (IRFoF), and
* Custodial Accounting Project.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* IRS officials stated these projects were delayed in being certified
because issuing a draft of EA version 2.0 in October 2001 for comment was
their highest priority. Also, STIR and CAP were certified compliant with an
earlier EA version. IRS plans to complete certification of these projects by
late-December 2001.
* However, until this is done, IRS cannot provide assurance that the project
requirements and design are properly aligned with the EA. Without this
alignment being established early and continuously throughout a project's
life cycle, IRS increases the chances that expensive project rework will be
required to meet IRS business and systems needs.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Weaknesses in Controls and Capabilities Increase Risks
* Weaknesses in any one of the aforementioned modernization management
controls introduces an unnecessary element of risk to the BSM program, but
the combination of these weaknesses introduces a level of risk that
increases exponentially over time. Given that the fifth plan provides
funding for later phases of key projects, continued development of other
projects, and starting of new projects, it is likely that BSM projects will
encounter additional cost, schedule, and performance shortfalls. This
combination of circumstances and events is represented in the following
updated graphic.
* As discussed later in this briefing, IRS reported in the fifth plan that
BSM projects have already encountered cost, schedule, and/or performance
shortfalls in meeting commitments made in the fourth plan. In those cases
where the fifth plan cited a reason for the shortfall, our analysis of the
reasons showed that weak management controls were a contributor--either via
(1) proactive prudent IRS decision making not to start or continue projects
because of immature controls (for example, see Reporting Compliance on p. 51
of this briefing) or (2) immature controls allowed project shortfalls to
occur.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* IRS acknowledges these risks. According to the CIO, until the weaknesses
are fully addressed, IRS is mitigating them by
* relying on existing immature processes,
* leveraging the knowledge, skills, and abilities of experienced senior
executives to make sure project issues are proactively managed, and
* hiring additional experienced executives.
* In our view, reliance on a combination of immature processes and
individual capabilities and heroic efforts is not a recipe for success. Past
government and industry experience shows the probability of repeating
successes on projects using this approach is low.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
Observation 1: Plan Discloses and Explains Project Cost Changes, But Continues to Omit Changes to Expected Benefits
* In our June 1999 report on IRS' first plan,19 we recommended that IRS, in
future expenditure plans, report progress against incremental project
commitments. Since then, we have reported with each plan that IRS has
improved its reporting but has not included progress on all incremental
project commitments, such as promised system capabilities and expected
system benefits.
* In the fifth plan, IRS disclosed that 18 projects have experienced cost
and/or schedule shortfalls against commitments made in its fourth and other
prior plans.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* Of the 18,
* 7 were program management initiatives that had cost increases ranging from
1% to 84% ($150,000 to $1.9 million), and
* 11 were system acquisition projects, of which
* 10 experienced schedule delays ranging from 1.5 months to 14 months; 6 of
the 10 experienced delays of 4 months or more; and 1 project had its
schedule shortened by 4 months.
* 8 experienced cost increases ranging from 4% to 66% ($500,000 to $6.1
million); 3 had cost decreases ranging from 4% to 23% ($365,000 to $4.1
million).
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* In addition, since the plan was completed, other slippages have occurred.
For example, on October 23, 2001, IRS moved STIR's Milestone 4 from October
2001 to January 2002, increasing its reported schedule slippage from 1
months to 4 months.
* According to IRS officials, the omission of such scope and benefit
information was an oversight, and they plan to correct it in the next
expenditure plan.
* By not fully providing scope and benefit change information, congressional
oversight of IRS modernization management performance and accountability is
constrained.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Observation 2: IRS Beginning to Develop BSM Human
* As we have previously reported21, strategic human capital centers on
viewing people as assets whose value to an organization can be enhanced
through investment. As the value of people increases, so does the
performance capacity of the organization. To maintain and enhance the
capabilities of IT staff, organizations should, among other things,
* assess knowledge and skills needed to effectively perform IT operations to
support agency mission and goals,
* inventory the knowledge and skills of current IT staff,
* identify gaps between requirements and current staffing, and
* develop and implement plans to fill the gaps.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* IRS has begun to address this issue. For example, it has hired a human
capital specialist to develop a plan by January 2002, for defining and
implementing an IT human capital strategy.
* Until IRS develops and implements this strategy, it will not know whether
it has the right IT knowledge and skills to effectively manage the BSM
program. Without this, IRS increases the risk of BSM program and project
cost, schedule, and performance shortfalls.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Observation 3: IRS' Integrated Master Schedule Has Yetto be Finalized
* The Integrated Master Schedule is an important tool for managingIRS'
acquisition of modernized systems. According to IRS, it is to
* specify about 20,000 major tasks and associated schedules, involving a
dozen organizations, for acquiring and implementingthe portfolio of BSM
projects, and
* identify the numerous and complex dependencies across these projects.
* IRS officials recognize the importance of having such an
integratedschedule and committed to completing one before they
beginimplementing FY 2002 system releases (scheduled to begin inJanuary
2002). However, despite important progress, IRS has not yet completed the
schedule because
* implementation plans (through milestone 5) have not yet beendeveloped for
all BSM projects to be implemented in FY 2002,
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* all change requests that impact BSM projects baselines have not been
analyzed, approved, and incorporated into the schedule, and
* 33 of 61 changes that need to be made to existing (legacy) systems have
not yet been analyzed, approved, and incorporated into the schedule.
* IRS officials stated that they plan to have the Integrated Master Schedule
completed by late December 2001.
* They attributed the delay in finalizing the schedule to this being the
first time IRS has attempted to develop a management tool of this size and
complexity, and it has taken longer than anticipated.
* Until these steps are completed, IRS does not have the means to adequately
manage the interdependencies within and among projects, thus increasing the
risk of project delays, overruns, and rework.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Observation 4: IRS Has Not Yet Implemented Effective Schedule Estimating Practices
* Producing reliable estimates of expected costs for program-level
activities and projects is essential to determining a project's
cost-effectiveness. Without this information, the likelihood of poor
investment decisions is increased.
* As part of our ongoing constructive engagement with IRS, we have orally
recommended to the former and current IRS CIOs that IRS adopt effective cost
and schedule estimating practices. SEI defines such practices in its model
for evaluating organizational cost and schedule estimating capabilities.22
The practices include having
* a historical database,
* structured processes for estimating product size and reuse,
* extrapolation mechanisms,
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* audit trails,
* integrity in dealing with dictated costs and schedules, and
* data collection and feedback processes.
* IRS officials recognize the importance of each of these practices.
However, IRS is not currently performing them.
* Consequently and as has been the case in the prior plans, the cost and
schedule estimates in IRS' fifth plan are contractor-provided, "rough order
of magnitude" estimates, that have not been subjected to meaningful,
reliable validation by IRS.
* IRS officials stated that heretofore they have not been able to dedicate
time to implementing effective estimating practices because of other
competing program and project priorities.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Results
* However, IRS has tasked the PRIME to develop a plan to implement an
estimating capability, that is to include
* Selection of an estimating method,
* Development of a plan for implementing the method by mid-January 2002, and
* Implementation of the plan beginning by the end of February 2002.
* IRS then plans to develop and implement an approach to oversee PRIME
estimating efforts by February 2002.
* IRS officials acknowledge that IRS' lack of an effective estimating
process has contributed to project delays and cost overruns. Without
improved estimation practices, these project shortfalls are likely to
continue.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
* IRS' fifth plan satisfies the legislative conditions. However, what has
and continues to challenge IRS is implementing the planned management
controls and capabilities. Since our last report, IRS has made important
progress in implementing pockets of modernization management capability.
However, this capability is still not where it should be because it has not
received the same level of priority and attention as BSM system projects. In
our view, these modernization control weaknesses will continue to put IRS at
risk of building systems that may not perform as intended, and/or cost more
and take longer than necessary to complete.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
* Moreover, the risks facing IRS have and will continue to become more
severe as projects begin to be built and implemented (ELC Milestones 3 and
4). Consequently, we are particularly concerned about those projects that
have or are going to proceed beyond these milestones. This concern is
heightened because (1) several of these projects are to provide the
foundational infrastructure upon which later projects depend, and (2) these
projects are already beginning to experience cost and schedule delays.
Exacerbating this situation are IRS' plans to simultaneously start more
projects while confronting these other challenges.
* IRS' risk mitigation strategy of relying on experienced executives until
control weaknesses are corrected is at best a stop gap measure and is not
sufficient given that IRS' timeline for addressing software acquisition
weaknesses extends 2 years. Until IRS addresses its management control
weaknesses, it will expose BSM to unnecessary risk.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
* IRS' fifth plan also does not fully provide whether projects' scope and
expected benefit commitments have changed. Such information is critical to
congressional oversight of IRS modernization management performance and
accountability.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
To address the escalating risks facing IRS on its BSM program, we recommend
that the Commissioner of Internal Revenue
* reconsider the planned scope and pace of the BSM program as defined in the
fifth expenditure plan with the goal of better balancing the number of
system acquisition projects underway and planned with IRS' capacity to
manage this workload. At a minimum, the Commissioner's reconsideration
should include
* slowing ongoing projects and/or delaying new project starts to reduce BSMO
resource demands,
* making correcting modernization management weaknesses a top priority and a
matter of top management attention, and
* reapplying resources (financial and human capital) available from slowed
and delayed projects toward correction of control weaknesses.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Executive Action
* To that end, we further recommend that the Commissioner do the following
with respect to each of modernization management weaknesses that we
identified.
* First, for software acquisition management,
* immediately assess CADE, STIR, and e-Services against SEI SA-CMM level 2
requirements,
* based on this assessment, develop a plan for correcting identified
weaknesses for these projects, including having an independent CMM
evaluation performed on these projects before submission of the next BSM
expenditure plan,
* submit with the next expenditure plan, the results of this independent
evaluation, along with a plan for ensuring that all BSM projects that have
passed milestone 3 will meet CMM level 2 requirements, and
* require all projects that have not passed milestone 3 as of December 31,
2001, to be assessed as CMM level 2 and have a plan for correcting any
project weaknesses found as a condition of milestone 3 approval.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Executive Action
* Second, for configuration management, risk management, EA implementation,
human capital strategic management, integrated program scheduling, and cost
and schedule estimating, ensure that commitments discussed in this briefing
for addressing residual weaknesses are implemented as planned, and report
any deviations, from these planned commitments to IRS' appropriations
subcommittees.
* Third, until PRIME quality assurance weaknesses are corrected, increase
the level of IRS oversight, scrutiny, and quality assurance of PRIME
activities.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Executive Action
* In addition, to allow for effective congressional oversight of the
program, we reiterate our prior recommendation that the Commissioner report
to IRS' appropriations subcommittees on any changes to expenditure plan
commitments concerning system requirements/capabilities to be delivered and
the associated benefits to be realized, and continue to report such
performance measures in future expenditure plans.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
* In commenting on a draft of this briefing, BSM executives stated that they
generally agreed with our findings, conclusions, and recommendations.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Appendix I: IRS' Expenditure Plan
BSM Spending Plan, Fiscal Year 2002 ($000)
Proposed Modernization Milestone
Initiatives Milestone Amount
Date Requested
Program Level
Activities*
Nov.
Program Management FY 02$7,918
(formerly Prime Program
Management Office)
Nov.
Business Integration FY 02$10,461
Architecture & Nov. 02$3
Integration FY ,539
Nov.
Management Processes FY 02$10,082
Nov.
FFRDC (MITRE) FY 02$18,070
$79,070
Core Infrastructure
Support Projects*
Security and Technology May
Infrastructure Releases FY 02$43,973
Infrastructure Shared Nov.
Services FY 02$39,747
Enterprise Systems July
Management FY 02$11,32 3
Development Integration
& Testing Environment FY May 02$12 ,916
$107,959
*Program Level and Core Infrastructure Support Projects are funded on a
fiscal year (FY) basis rather than by milestone.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Appendix I:
Data Projects
Customer Account Data Engine -IMF R1
Customer Account Data Engine -IMF R3
Enterprise Data Warehouse R3
Integrated Financial Services/ Core Financial Systems
Business Projects
IR/FoF R1
e-Services 2002
Customer Account Management
Filing & Payment Compliance
Reporting Compliance -Individual
HR Connect
������������������������������ MS2,3 Feb. 04
Total Business Systems Modernization Program ���������� MS2,3 Feb. 04
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Spending Plan Results of GAO Review
* The plan satisfied the legislative
conditions for the use of ITIA funds and
was consistent with our open
recommendations. * The plan was an
appropriate first step, but the key to
1st Spending Plan (May 1999) success would be effective implementation
($35 million request) of the plan. * Future plans should specify
progress against prior plan commitments,
and the next plan should clarify
IRS/contractor roles and responsibilities.
(See Tax Systems Modernization: Results of
Review of IRS' Initial Expenditure Plan,
GAO/AIMD/GGD-99-206, June 15, 1999)
* The plan raised concerns about projects
that were scheduled to begin detailed
design and software development before,
among other things, the enteprise
architecture was completed and the ELC was
1st Interim Spending Plan defined and implemented. * IRS should
(Dec 1999) ($33 million expedite completion of the architecture and
request) implementation of the ELC. * Future plans
should explain how IRS plans to manage the
risk of performing detailed design or
development work if the architecture is not
sufficiently completed or the ELC is not
sufficiently implemented.
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Appendix II:
Spending Plan Results of GAO Review
*� IRS met relatively few
commitments in its $35 million first ITIA
spending plan, even though the Service
later received an additional $33 million
and nearly 5 months of extra time to
accomplish the goals set forth in the
first plan. *� The plan satisfied
the legislative conditions for the use of
ITIA funds, and was generally consistent
2nd Spending Plan (Mar 2000) with recommendations contained in our
($176 million request) earlier reports. *� The key to
success would be whether IRS effectively
implements the plan. *� Until IRS
completes its initiated actions to
redirect and restructure its
modernization effort, it would continue
to lack key modernization and technical
controls. (See Tax Systems Modernization:
Results of Review of IRS' March 7, 2000,
Expenditure Plan, GAO/AIMD-00-175, May
24, 2000)
*� IRS had not adhered to the
approved and funded March 7, 2000,
spending plan. *� On selected
initiatives, IRS had not met cost and
schedule commitments made in its March 7,
2000 spending plan. *� Most
modernization initiatives had
nevertheless made important progress
since March 2000. IRS fully addressed two
of its modernization management
capability weaknesses, and it was making
progress in addressing others. *�
One project, Custodial Accounting Project
(CAP), had been approved for product
development without sufficient definition
2nd Interim Spending Plan and without a compelling business case.
(Aug 2000) ($33 million Further investment in CAP should be
request) limited until IRS demonstrates sufficient
business value and reports to the House
and Senate committees on risk mitigation.
*� Another project, Security and
Technology Infrastructure Release (STIR),
was being preliminarily designed without
sufficient requirements definition and
economic justification. The STIR project
should be directed to complete a security
risk assessment as soon as possible, and
ensure that STIR requirements and the
proposed design solution are economically
justified through a business case. (See
Tax Systems Modernization: Results of
Review of IRS' August 2000 Interim
Spending Plan, GAO-01-91, November 8,
2000)
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Appendix II:
Spending Plan Results of GAO Review
* IRS' plan satisfied the legislative
conditions for the use of ITIA funds, and
was making important progress towards
satisfying the congressional direction on
two projects - CAP and STIR. * IRS was
making important progress in establishing
effective modernization management
capability, but important and challenging
work remained. Until IRS completed its
initiated actions to fully implement its
system life cycle methodology and business
systems modernization office, and resolve
issues concerning the completeness and
accuracy of enterprise architecture, it
continued to lack key modernization and
3rd Spending Plan (Oct technical controls. * Five modernization
2000) ($200 million initiatives experienced schedule delays
request) and/or cost increases. However, the third
plan did not address whether projects' prior
commitments for delivery of promised systems
capabilities (requirements) and
benefit/business value were being met. * IRS
used contractor-provided "rough
order-of-magnitude" estimates in preparing
the third expenditure plan. IRS planned to
validate the third plan's estimates as part
of its process to negotiate and definitize
contract task orders. Previously, this
process resulted in finalized contract costs
below the estimates, totalling $9 million.
(See Tax Systems Modernization: Results of
Review of IRS' Third Expenditure Plan,
GAO-01-227, January 22, 2001)
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Appendix II:
Spending Plan Results of GAO Review
* IRS' plan satisfied the conditions
specified in the appropriations acts. *
IRS continued to make important progress
in implementing modernization management
controls and capabilities. Nevertheless,
IRS' modernization management capacity is
still not where it should be, given (1)
the number of systems acquisition projects
that the March 2001 plan identifies as
underway and planned and (2) the fact that
several of the ongoing projects are
entering critical stages in their life
cycles. For example, IRS did not have a
sufficiently defined version of the
enterprise architecture to guide and
constrain projects, and employing rigorous
configuration management practices. * Due
to missing management capacity, key IRS
projects were beginning to experience
cost, schedule, and performance shortfalls
against the commitments the agency made in
its third expenditure plan. For example,
deployment of the Customer Communications
2001 project was three months behind
4th Spending Plan (March schedule, and promised system capabilities
2001) ($128 million request) and associated benefits had been deferred.
Also, a critical infrastructure project,
STIR, was reported to be 1.5 months late
in trying to complete its preliminary
design phase (Milestone 3); and the agency
was still working to finalize 6 of 19 work
products needed to complete the phase.
Thus, the project was actually almost five
months late. * IRS officials recognized
the need to address its modernization
management capacity before key ongoing
projects moved into critical life-cycle
phases, and before additional projects
were started. Accordingly, IRS planned or
had initiated steps to address these
weaknesses. In particular the Commissioner
had decided to slow ongoing and new
projects, giving priority to putting in
place missing management capacity. We
believed this decision was prudent and
appropriate and made recommendations to
ensure IRS followed through on this
decision. (See Business Systems
Modernization: Results of Review of IRS'
March 2001 Expenditure Plan, GAO-01-716,
June 29, 2001)
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs
Appendix II
Comments from the Internal Revenue Service
Appendix II Comments from the Internal Revenue Service
Appendix II Comments from the Internal Revenue Service
Appendix II Comments from the Internal Revenue Service
Appendix III
GAO Contacts and Staff Acknowledgments
GAO Contact Gary N. Mountjoy, (202) 512-6367
Acknowledgments In addition to the individual named above, other key
contributors were Bernard R. Anderson, William G. Barrick, Timothy D.
Hopkins, Ona M. Noble, Pietro L. Salatti, Aaron W. Thorne, and William F.
Wadsworth.
GAO's Mission The General Accounting Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to good
government is reflected in its core values of accountability, integrity, and
reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents is through the
Internet. GAO's Web site (www.gao.gov) contains abstracts and full-text
files of current reports and testimony and an expanding archive of older
products. The Web site features a search engine to help you locate documents
using key words and phrases. You can print these documents in their
entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its Web
site daily. The list contains links to the full-text document files. To have
GAO E-mail this list to you every afternoon, go to www.gao.gov and select
"Subscribe to daily e-mail alert for newly released products" under the GAO
Reports heading.
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out to
the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. General Accounting Office
P.O. Box 37050
Washington, D.C. 20013
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
Visit GAO's Document GAO Building
Room 1100, 700 4th Street, NW (corner of 4th and G Streets, NW)Distribution
Center Washington, D.C. 20013
To Report Fraud, Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm,Waste, and Abuse in E-mail:
[email protected], orFederal Programs 1-800-424-5454 or (202) 512-7470
(automated answering system).
Public Affairs Jeff Nelligan, Managing Director, [email protected] (202)
512-4800 U.S. General Accounting Office, 441 G. Street NW, Room 7149,
Washington, D.C. 20548
Presorted Standard
Postage & Fees Paid
GAO
Permit No. GI00
United States
General Accounting Office
Washington, D.C. 20548-0001
Official Business
Penalty for Private Use $300
Address Correction Requested
*** End of document. ***