Business Systems Modernization: IRS Needs to Better Balance	 
Management Capacity with System Acquisition Workload (28-FEB-02, 
GAO-02-356).							 
                                                                 
GAO reviewed the Internal Revenue Service's (IRS) fifth 	 
expenditure plan requesting $391 million from its Business	 
Systems Modernization (BSM) fund. Although IRS's November 2001	 
expenditure plan satisfied the conditions specified in the	 
appropriations act, IRS must still fully implement the controls  
and capabilities described in the plan. Since GAO's June 2001	 
report, IRS has made important progress in implementing 	 
modernization management controls and capabilities and addressing
GAO's past recommendations. However, IRS's modernization	 
management capacity is still not where it needs to be. Examples  
of modernization management controls and capabilities that are	 
not yet fully implemented include software acquisition		 
management, configuration management, quality assurance, risk	 
management, enterprise architecture implementation, human capital
management, integrated program scheduling, and cost and schedule 
estimating. The increased risk of IRS's proceeding without these 
controls and capabilities has contributed to project cost,	 
schedule, and performance shortfalls. IRS acknowledges that it	 
needs to strengthen its modernization management controls. IRS	 
recognizes that these controls become more critical as the size  
and complexity of the BSM program continue to increase. Although 
IRS has actions underway to fully implement these controls, IRS  
plans to compensate for their immaturity by applying experienced 
human capital. Reliance on a combination of existing immature	 
processes and individual expertise and heroic efforts is a	 
short-term solution to a long-term need. Because the immaturity  
of these controls has already led to project cost, schedule, and 
performance shortfalls, IRS needs a better strategy to mitigate  
the risks associated with its fifth expenditure plan.		 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-02-356 					        
    ACCNO:   A02807						        
  TITLE:     Business Systems Modernization: IRS Needs to Better      
Balance Management Capacity with System Acquisition Workload	 
     DATE:   02/28/2002 
  SUBJECT:   Appropriated funds 				 
	     Information systems				 
	     Internal controls					 
	     Performance measures				 
	     Strategic planning 				 
	     Systems development life cycle			 
	     IRS Business Systems Modernization 		 
	     Program						 
                                                                 
	     IRS Customer Account Data Engine Project		 
	     SEI Software Acquisition Capability		 
	     Maturity Model					 
                                                                 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-02-356
     
United States General Accounting Office

GAO Report to Congressional Committees

February 2002

BUSINESS SYSTEMS MODERNIZATION

IRS Needs to Better Balance Management Capacity with Systems Acquisition
Workload

                                      a

GAO-02-356

           United States General Accounting Office February 2002

                
   Accountability Integrity Reliability                                  Management

Capacity with Systems Acquisition

Highlights Workload

Highlights of GAO-02-356, a report to the Subcommittee on Treasury and
General Government, Senate Committee on Appropriations and the Subcommittee
on Treasury, Postal Service and General Government, House Committee on
Appropriations.

Why GAO Did This Study What GAO Found

The Internal Revenue Service IRS's November 2001 expenditure plan defines
the next incremental set (IRS) has underway a multiyear, of management and
systems acquisition activities that are intended to multibillion-dollar
business move IRS closer to its systems modernization. Related to the
activities systems modernization program. proposed in this plan, IRS has to
date made important progress in Since 1995, GAO has designated establishing
the systems infrastructure needed to allow the introduction this program
high-risk, in part of future business application systems. Similar progress
has also been because of its size and made in establishing the key
modernization management controls complexity and its immense needed to
achieve the kind of systems acquisition activities provided for importance
to improving IRS in this plan. Nevertheless, as IRS moves forward to
implement the plan, it mission performance and faces increasing risk that it
will be unable to deliver promised system accountability. The Congress has
capabilities on time and within budget. Reasons for this include: mandated
that GAO review IRS's

plans for spending funds on this ï¿½ IRS lacks certain modernization
management controls and program. This study responds to capabilities. For
example, it lacks a human capital management that mandate by providing the
strategy for the program and a reliable project cost-estimating results of
GAO's review of IRS's process. It has also yet to implement mature systems
and software November 2001 expenditure plan. acquisition process controls;
its plans for correcting this allow core

modernization projects to continue for at least 2 more years without such
controls.

ï¿½ Interdependencies among ongoing systems acquisition projects and the
complexity of associated activities to be performed during the period
covered by the plan are an order of magnitude greater than

What GAO Recommends those of the previous plan. This workload increase
raises concern,
To address the escalating risks because IRS was not able to meet many of the
project cost and
facing IRS in this critical program, schedule commitments made in the less
demanding prior plan.

GAO recommends that the IRS ï¿½ IRS's plan provides for increasing its
workload by starting additional
commissioner reconsider the projects that will further tax existing
capability.
planned scope and pace of the

program as defined in the In attempting to balance the need for introducing
modern systems and
November 2001 expenditure plan. their associated benefits as soon as
possible, with the need for greater
This is necessary to better management capacity to ensure that these systems
are introduced
balance the number of systems successfully, IRS's November 2001 plan
emphasizes the former. This
acquisition projects underway and emphasis adds significant risk that
escalates as the program advances.
planned with IRS's capacity to

manage this workload. GAO is
providing specific
recommendations to the
commissioner on how to strike M anagem ent
such a balance. In response, the Capability
commissioner agreed with GAO's
recommendations, and described
planned and ongoing efforts to
address each.

                             Project W orkload

This is a test for developing highlights for a GAO report. The full report,
including GAO's objectives, scope, methodology, and analysis, is available
at www.gao.gov/cgi-bin/getrpt?GAO-02-356. For additional information about
the report, contact Randolph C. Hite (202-512-3439). To provide comments on
this test highlights, contact Keith Fultz (202-512-3200) or E-mail
[email protected].

Contents

                                  Letter 1

Recommendations for Executive Action 3

Agency Comments 4

Appendixes

Appendix I: Briefing Slides from December 10 and 11, 2001, Briefings to
the Senate and House Appropriations Subcommittee Staffs 6

Appendix II: Comments from the Internal Revenue Service 81

Appendix III: GAO Contacts and Staff Acknowledgments 85
GAO Contact 85
Acknowledgments 85

Abbreviations

BSM Business Systems Modernization
IRS Internal Revenue Service
OMB Office of Management and Budget
SA-CMM Software Acquisition Capability Maturity Model

A

United States General Accounting Office Washington, D.C. 20548

February 28, 2002

The Honorable Byron L. Dorgan
Chairman
The Honorable Ben Nighthorse Campbell
Ranking Minority Member
Subcommittee on Treasury and General Government
Committee on Appropriations
United States Senate

The Honorable Ernest J. Istook, Jr.
Chairman
The Honorable Steny H. Hoyer
Ranking Minority Member
Subcommittee on Treasury, Postal Service

and General Government
Committee on Appropriations
House of Representatives

Pursuant to the Department of the Treasury's fiscal year 2002
appropriations act, the Internal Revenue Service (IRS), in November 2001,
submitted to the congressional appropriations committees its fifth
expenditure plan, requesting $391 million from its Business Systems
Modernization (BSM) fund.1 As required by the act, we reviewed the plan.
Our objectives were to (1) determine whether the plan satisfied the
conditions specified in the act,2 (2) determine IRS's progress in
implementing modernization management controls and capabilities, and
(3) provide any other observations about the plan and IRS's BSM program.

On December 10 and 11, 2001, we briefed your respective offices on the
results of our review. This report transmits the materials used at those
briefings, and reiterates the recommendations to the commissioner of
internal revenue that we specified in the briefings. The full briefing

1 The Treasury and General Government Appropriations Act, 2002 (P. L.
107-67).

2 The act specifies that BSM funds are unavailable until IRS submits to
congressional appropriations committees for approval a modernization
expenditure plan that (1) meets the Office of Management and Budget's (OMB)
capital planning and information technology investment control review
requirements; (2) complies with IRS's enterprise architecture; (3) meets
IRS's life-cycle management requirements; (4) is approved by IRS, Treasury,
and OMB; (5) is reviewed by GAO; and (6) complies with federal acquisition
requirements and management practices.

materials, including our scope and methodology, are reprinted in appendix

I. In summary, we made the following five major points:

* IRS's November 2001 expenditure plan satisfied the conditions specified in
the appropriations act. However, while the plan provided for satisfying
these conditions, IRS must still fully implement the controls and
capabilities described in the plan.

* Since our June 2001 report,3 IRS has made important progress in
implementing modernization management controls and capabilities and
addressing our past recommendations. Nevertheless, IRS's modernization
management capacity is still not where it needs to be, given (1) the number
of systems acquisition projects that the November 2001 plan identifies as
being underway, (2) the fact that several of these ongoing projects have
already entered the critical building stage of their life cycles (milestone
3) and are to begin deployment (milestone 4) during this year, and (3) IRS's
plan to begin additional projects. Examples of modernization management
controls and capabilities that are not yet fully implemented include
software acquisition management,4 configuration management,5 quality
assurance, risk management, enterprise architecture6 implementation, human
capital management, integrated program scheduling, and cost and schedule
estimating.

* The increased risk of IRS's proceeding without these controls and
capabilities has contributed to actual project cost, schedule, and
performance shortfalls. For example, in the fifth plan, IRS reports that the
Customer Account Data Engine's (release 1) milestone 4 date has been delayed
by 6 months, and its cost has increased by $5 million (13

3 U.S. General Accounting Office, Business Systems Modernization: Results of
Review of IRS's March 2001 Expenditure Plan, GAO-01-716 (Washington, D.C.:
June 29, 2001).

4 Carnegie Mellon University's Software Engineering Institute has developed
criteria, known as the Software Acquisition Capability Maturity Model?
(SA-CMM?), for determining organizations' software acquisition management
effectiveness or maturity. Capability Maturity Model and CMM are registered
in the U.S. Patent and Trademark Office.

5 Configuration management is the means for ensuring the integrity and
consistency of systems modernization programs and project products
throughout their life cycles. Through effective configuration management,
for example, integration among related projects and alignment between
projects and the enterprise architecture can be achieved.

6 An enterprise architecture is an institutional blueprint defining how an
enterprise operates today, in both business and technological terms, and how
it wants to operate in the future. It also includes a roadmap for
transitioning between these environments.

percent above the fourth plan's funding level). Also, since submitting the
fifth plan for approval, IRS announced that the Security and Technology
Infrastructure Release's milestone 4 date has been delayed by 3 months, and
its cost has increased by $6 million (24 percent above the fourth plan's
funding level).

* IRS acknowledges the need to strengthen its modernization management
controls, and recognizes that these controls become more critical as the
size and complexity of the BSM program continues to increase. It also has
actions underway to fully implement these controls and, until then, plans to
compensate for their immaturity by applying experienced human capital.

* In our view, reliance on a combination of existing immature processes and
individual expertise and heroic efforts is a short-term solution to a
long-term need. Given that the immaturity of these controls has already
contributed to project cost, schedule, and performance shortfalls, and will
likely continue to do so, IRS needs a better strategy for mitigating the
risks it faces in implementing its fifth expenditure plan. To assist IRS in
striking a proper balance between the need to quickly introduce modernized
systems yet prudently manage the risks inherent in such an undertaking, we
made the following recommendations to the commissioner of internal revenue.

Recommendations for Executive Action

To address the escalating risks facing IRS on its BSM program, we recommend
that the commissioner of internal revenue reconsider the planned scope and
pace of the BSM program as defined in the fifth expenditure plan, with the
goal of better balancing the number of systems acquisition projects underway
and planned with IRS's capacity to manage this workload. At a minimum, the
commissioner's reconsideration should include

* slowing ongoing projects and delaying new project starts to reduce
Business Systems Modernization Office resource demands,

* making correcting modernization management weaknesses a top priority and a
matter of top management attention, and

* reapplying resources-financial and human capital-available from slowed and
delayed projects toward correction of control weaknesses.

We further recommend that the commissioner take the following actions with
respect to each of the modernization management weaknesses that we
identified.

First, for software acquisition management,

* immediately assess critical BSM projects (i.e., Customer Account Data
Engine, Security and Technology Infrastructure Release, and e-Services)
against the Software Engineering Institute's Software Acquisition Capability
Maturity Model? (SA-CMM?) level 2 requirements;

* based on this assessment, develop a plan for correcting identified
weaknesses for these projects, including having an independent SAï¿½CMM
evaluation performed on them before submission of the next BSM expenditure
plan;

* submit, with the next expenditure plan, the results of this independent
evaluation, along with a plan for ensuring that all BSM projects that have
passed milestone 3 will meet SA-CMM level 2 requirements; and

* require all projects that did not pass milestone 3 as of December 31,
2001, to be assessed as SA-CMM level 2, and have a plan for correcting any
project weaknesses found as a condition of milestone 3 approval.

Second, for configuration management, risk management, enterprise
architecture implementation, human capital strategic management, integrated
program scheduling, and cost and schedule estimating, ensure that
commitments discussed herein for addressing residual weaknesses are
implemented as planned, and report any deviations from these planned
commitments to IRS's appropriations subcommittees.

Third, until contractor quality assurance weaknesses are corrected, increase
the level of IRS oversight, scrutiny, and quality assurance of contractor
activities.

Finally, to allow for effective congressional oversight of the program, we
reiterate our prior recommendation that the commissioner report to IRS's
appropriations subcommittees any changes to expenditure plan commitments
concerning systems requirements/capabilities to be delivered and the
associated benefits to be realized, and continue to report such performance
measures in future expenditure plans.

Agency Comments In commenting on a draft of this report, the commissioner of
internal revenue agreed with our recommendations, adding that IRS leadership
understands the importance of correcting the issues that we identified and
is moving aggressively to resolve them. In this regard, the commissioner
described IRS's ongoing and planned efforts relating to addressing each of

our recommendations.  The commissioner's  written comments are  reprinted in
appendix II.

We are sending copies of this report to the chairmen and ranking minority
members of other Senate and House committees and subcommittees that have
appropriations, authorization, and oversight responsibilities for the
Internal Revenue Service. We are also sending copies to the commissioner of
internal revenue, the secretary of the treasury, the chairman of the IRS
oversight board, and the director of the Office of Management and Budget.
Copies are also available at our Web site at www.gao.gov. 
Should you or your offices have questions on matters discussed in this
report, please contact me at (202) 512-3439. I can also be reached by e-mail
at [email protected]. Key contributors to this report are listed in appendix
III.

Randolph C. Hite Director, Information Technology Architecture

and Systems Issues

Appendix I

Briefing Slides from December 10 and 11, 2001, Briefings to the Senate and
House Appropriations Subcommittee Staffs

Results  of Review  of  IRS' November  2001 Business   Systems Modernization
Expenditure Plan

                          Briefing to the Staffs of
                   the Senate Committee on Appropriations,
               Subcommittee on Treasury and General Government
                           (on December 11, 2001)
                                     and
                   the House Committee on Appropriations,
                  Subcommittee on Treasury, Postal Service,
                           and General Government
                           (on December 10, 2001)

                                      1

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

* Introduction

* Objectives

* Scope and Methodology

* Background

* Results in Brief

* Results

* Conclusions

* Recommendations

                                     2

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

* As mandated by IRS' FY 2002 appropriations act, Business Systems
Modernization (BSM) funds are unavailable until IRS submits to the
congressional appropriations committees for approval, a modernization
expenditure plan that:

* Meets the Office of Management and Budget's (OMB) capital planning and
information technology (IT) investment control review requirements;

* Complies with IRS' enterprise architecture;1

* Meets IRS life cycle management requirements;2

* Is approved by IRS, Treasury, and OMB;

* Is reviewed by GAO; and

* Complies with federal acquisition requirements and management practices.

* Since mid-1999, IRS has submitted a series of expenditure or "spending"
plans requesting release of BSM appropriated funds.

1An Enterprise Architecture (EA) is an institutional blueprint defining how
an enterprise operates today, in both business and technology terms, 3and
how it wants to operate at some point in the future. An EA also includes a
roadmap for transitioning between these environments. 2IRS refers to its
life cycle management program as the Enterprise Life Cycle (ELC), which is
graphically depicted in the Background Section.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

* To date, about $968 million has been appropriated for BSM, including about
$391 million for FY 2002. Of the $968, $577 million has been released; about
$391 million remains.

* On November 13, 2001, IRS submitted its fifth plan, seeking release of the
remaining $391 million.

* If the plan is approved, the BSM fund will have a zero balance. To
replenish the fund, IRS plans to request $450 million via its FY 2003 budget
request.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

* As agreed with IRS' appropriations subcommittees, our objectives were to

* determine whether the fifth expenditure plan satisfies the legislative
conditions,

* determine what progress IRS has made in implementing modernization
management controls and capabilities, and

* provide any other observations about the fifth plan and IRS' BSM program.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                            Scope and Methodology

* To accomplish our objectives, we

* Reviewed the fifth expenditure plan and met with IRS program officials to
understand the scope and content of the plan;

* Analyzed the plan against the legislative conditions to identify any
variances;

* Reviewed program and project management reports and briefings to assess
progress in implementing modernization management controls and capabilities;

* Observed modernization executive steering committee and subcommittee
meetings to, among other things, document how the plan was developed and
reviewed;

* Interviewed program and project management officials to corroborate our
understanding of the plan and other BSM activities.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                            Scope and Methodology

* Analyzed available evidence on recent efforts to implement modernization
management controls and capabilities. Specifically, we analyzed progress and
plans for

* enterprise architecture (EA) definition and implementation,

* ELC definition and implementation, including configuration management,
quality assurance and risk management,

* software acquisition maturity, as defined by the Software Engineering
Institute's (SEI) Software Acquisition Capability Maturity
Model[tm](SA-CMM),

* human capital management,

* cost and schedule estimating practices,

* integrated program schedule development, and

* key projects, such as the Security and Technology Infrastructure Release
(STIR), the Customer Account Data Engine (CADE), e-Services, and the
Internet Refund and Fact of Filing project.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                            Scope and Methodology

* Collaborated with the Treasury Inspector General for Tax Administration
(TIGTA) to avoid duplication of effort in reviewing BSM initiatives and
incorporated TIGTA results in this briefing where appropriate.

* Projects addressed by TIGTA included Customer Communications, e-Services,
the Telecommunications Enterprise Strategic Program, and Customer
Relationship Management-Exam.

* Program-level processes addressed in a recent capping report on key system
development processes (e.g. requirements management, configuration
management, risk management).

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                            Scope and Methodology

* As agreed with your offices, we did not independently validate planned
initiatives' cost estimates or confirm, through system and project
management documentation, the validity of IRS-provided information on the
initiatives' content and progress.

* We provided a draft of this briefing on December 7, 2001, to IRS BSM
program executives (Deputy Associate Commissioners for Program Management
and Systems Integration and the Executive Program Advisor for Risk
Management), and have incorporated their comments where appropriate.

* We performed our work from October through December 2001 in accordance
with generally accepted government auditing standards.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

Summary of IRS' Fifth Expenditure Plan ($000)3

Program Management and Architecture Activities

Program Management Office $  7,918 Business Transformation Planning $ 10,461
Architecture & Systems Integration $ 32,539 Quality Management and Assurance
$ 10,082 Federally Funded Research and Development Center -MITRE $ 18,070

                             Subtotal $ 79,070

Project Level and Infrastructure Activities4

Core Infrastructure Support Projects (e.g. STIR) $107,959 Business Systems
Support Projects ( Customer Account Data Engine, $112,224 Custodial
Accounting Project, Core Financial Systems) Business Systems Projects $
89,686 
                             Subtotal $309,869

Addition to Management Reserve $ 2,061

Total $391,000 
Source: IRS

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

* IRS' plan is to (1) continue ongoing program-level initiatives through
mid-November 2002 and 16 ongoing projects to their next milestones and (2)
start 6 new projects. Examples of new projects are

* Reporting Compliance, which is to, among other things, select returns for
examination

* Filing and Payment Compliance, which is to enable access to taxpayer data
in CADE and the Custodial Accounting Project system to determine compliance
and allow taxpayers to resolve account issues electronically

* Compared to the total BSM funds requested in the third and fourth plans
combined, the fifth plan represents an 18 percent decrease in program
management spending and a 27 percent increase in project acquisition
spending.

* The third and fourth plans generally provided funding for FY 2001, and the
fifth plan does so for FY 2002.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

* Like its previous plans, IRS' fifth expenditure plan covers contractor
costs, such as the Prime Systems Integration Support (PRIME) contractor and
the systems engineering and technical assistance contractor (MITRE), and not
IRS internal costs, such as IRS BSM program office (BSMO) staff costs. As we
previously reported,5

* IRS' actual use of prior BSM funding has been limited to the modernization
program

* IRS' actual use of prior IS funding has included modernization activities.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

Summary of Prior GAO Expenditure Plan Reviews 
* To date, GAO has reviewed and reported on six requests for BSM funding
releases.6

* Since mid-1999, we have reported7 on the risks associated with IRS'
approach of concurrently building systems while developing and implementing
program management capabilities such as having a fully operational program
management office and implementing the ELC.

6For details on our past review results, see appendix II.
7For example, see Business Systems Modernization: Results of Review of IRS'
March 2001 Expenditure Plan (GAO-01-716, June 29, 2001). 15
and Internal Revenue Service: Progress Continues But Serious Management
Challenges Remain (GAO-01-562T, April 2, 2001).

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

* In summary, we reported that attempting to acquire modernized systems
before having the requisite management capability increases the risk that
systems will experience cost, schedule, and performance shortfalls.

* EA, configuration management, quality assurance, and risk management, are
but four of many management controls required under IRS' ELC, which is a
structured method for managing system modernization program and project
investments throughout their life cycles (see below simplified diagram of
ELC).

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

* We have also reported8 that the risks associated with building systems
without the requisite management controls are not as severe early in
projects' life cycles when they are being planned (project definition and
preliminary system design), but escalate as projects are built (detailed
design and development) and implemented (enterprise deployment).

* In the case of IRS and its ELC, this point of risk escalation is ELC
Milestone 3, as is shown in the following graphic. From this point through
deployment (Milestone 4) to operations and support (Milestone 5), risk can
increase significantly.

* In June 2001 report,9 we identified key IRS projects that were approaching
or had passed Milestone 3 that were beginning to experience such cost,
schedule, and performance shortfalls, and concluded that program risks were
increasing (see graphic on next page).

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                              Results in Brief

* Nevertheless, IRS' modernization management capability is still not where
it should be given (1) the number of acquisition projects that are underway,
(2) the fact that several of these projects have already entered the
critical building stage of their life cycles (Milestone 3) and are to begin
deployment (Milestone 4) during calendar year 2002, and (3) IRS' plan to
begin additional projects.

* Modernization management controls and capabilities that are not yet fully
implemented are

* Software acquisition management

* Configuration management

* Quality assurance

* Risk management

* EA implementation

* Human capital management,

* Integrated program scheduling, and

* Cost and schedule estimating

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                              Results in Brief

* The increased risk of proceeding without these controls and capabilities
is now contributing to actual project cost, schedule, and performance
shortfalls. For example,

* In the fifth plan, IRS reports that CADE's (Release 1) Milestone 4 date
has slipped by 6 months, and its cost has increased by $5 million (13% above
the fourth plan's funding level), and

* Since submitting the fifth plan for approval, IRS announced that STIR's
Milestone 4 date has slipped by 3 months, and its cost has increased by $6
million (24% above fourth plan funding level).

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                              Results in Brief

* Since IRS submitted its fifth plan, we shared our findings and conclusions
with IRS. In response, IRS

* acknowledges the need to strengthen its modernization management controls,

* recognizes these controls are more critical as the size and complexity of
the BSM program continues to increase, and

* has efforts underway intended to fully implement these controls, and until
then, plans to compensate for their immaturity by applying experienced human
capital.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                              Results in Brief

* In our view, reliance on a combination of existing immature processes and
individual expertise and heroic efforts is a short-term solution to a
long-term need. Given that these controls' immaturity have already
contributed to project cost, schedule, and performance shortfalls, and will
likely lead to future shortfalls, IRS needs a better strategy for mitigating
the risks it faces in implementing its fifth expenditure plan.

* To assist IRS in striking a proper balance between the need to quickly
introduce modernized systems and prudently manage the risks inherent in
doing so, we are making recommendations to the Commissioner of Internal
Revenue.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

                                                     Legislative
                  Conditions       Expenditure Plan Provisions
                                    IRS' fifth expenditure plan
                                       provides for managing
                                     investments as part of a
                                       portfolio through its
                                        Investment Decision
                                     Management process. This
                                   includes conducting periodic
                                    portfolio reviews to assess
             1.� Meets the       changes in business
             OMB capital planning     priorities and project
             and IT investment       schedules. The fifth plan
             control review            provides for such an
             requirements.          assessment in January 2002.
                                   The fifth plan also provides
                                       for IRS to revise and
                                    implement its business case
                                    guidance to achieve better
                                    system investment decisions
                                        based on compelling
                                       return-on-investment
                                         justifications.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

 Legislative
 Conditions                                                                            Expenditure Plan Provisions
  4.�
 Approved by
     IRS,
Treasury, and *� IRS - September 17, 2001 *� Treasury - October 19, 2001 *� OMB - October 29, 2001 *� Submitted to IRS' appropriations subcommittees - November
    OMB.                                                                                        13, 2001
                                                                            *� GAO - December 10, 2001 briefing to IRS'
  5.�                                                                 House appropriation subcommittee - December 11,
 Reviewed by                                                                              2001 briefing to IRS'
    GAO.      ���������������������Senate
                                                                                       appropriation subcommittee
6.�
Complies with
the
acquisition
rules,
requirements,
guidelines,
and systems
acquisition
management
practices of   As part of the ELC, IRS has defined processes, roles, responsibilities, etc. for implementing Software Engineering Institute (SEI) Software Acquisition Capability Maturity
the federal    ModelTM practices within the level 2 key process areas. 12 These practices are consistent with federal acquisition requirements and management practices, and the plan calls
Government.                                                                  for implementation of the ELC on all projects.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* Since we reported  on IRS' last plan,13 IRS has made important progress in
implementing  modernization  controls and  capabilities  and addressing  our
recommendations.

* However, key controls and capabilities are still not fully implemented.

                    �����������������������������������������������                    
                    �����������������������������������������������                    
����������������������������������������
��
���������������������������������������������������� ���������� 
���������������                                        `` 	
                                �� ������������������������������������������ 65533;��� ����������
                                                ����������������������������������������  
                �� ����������������������������������������������
                                �������������������������������������������� onica,
        ��������������������������������������������������  

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

                                ����������������������������������������������� 
                                �����������������������������������������������  
                            ������������������������������������������������
                                �����������������������������������������������
                                                                                                                        �������������������������
                    �� ������������������������������������������������
                        �������������������������������������������������  
                        �� ����������������������������������������������� 
                        �������������������������������������������������  
��
������������������������������������������������������� 
�����������������������������������
        �� ��������������������������������������������������� 
                        ������������������������������������������������� 
                            ������������������������������������������������    
                �� ������������������������������������������������� #65533; 
                                                                                                                                                                
                                                                                                                                                    ������������������
                            �� ����������������������������������������������
                                                                                                                    ��������������������������  65533;����������    
                                                                                                                                                            ����������������
                �� �������������������������������������������������
                                                                                                                                                                
�� ����������������������������������������������������
��������������                                                
                                            �� ������������������������������������������                   
                                                                                                                                                                
                                       ���������������������������� �� ���������������               
                            �� ����������������������������������������������   
                        �������������������������������������������������  

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

IRS Has Not Implemented Effective Software Acquisition 
* The Clinger-Cohen Act requires the establishment of effective IT
management processes. SEI's Software Acquisition Capability Maturity
Model[tm] defines such processes for managing software acquisitions. Since
1995, we have recommended that IRS establish, at a minimum, the "repeatable"
level of SEI's software acquisition management processes (Level 2).14

* IRS has not completed implementation of SEI's Level 2 software processes,
although it committed in its fourth plan to do so by September 2001.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* IRS officials attribute the delay to their underestimation of how long it
takes to implement the practices.

* To effectively implement mature acquisition management processes, an
organization needs to

* first assess its strengths and weaknesses and

* then develop a plan for leveraging existing strengths and correcting
weaknesses, paying special attention to projects that are most critical and
vulnerable to weaknesses.

* To address its acquisition management control weaknesses, IRS has recently

* engaged SEI to help better estimate how long this will take, and

* developed a plan for implementing these practices which includes

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* focusing first on two new BSM projects,

* having these two projects internally assessed by June 2002,

* having these two projects independently evaluated by December 2002, and

* implementing the practices on remaining BSM projects during FY 2004.

* In effect, this plan means that IRS will not implement mature acquisition
processes on key projects until after initial system increments have passed
critical, later life cycle phases. As a result, IRS will continue to run the
risk that these projects' promised capabilities will not be delivered on
time and within budget.

* The significance of this risk is magnified in light of the multiple
dependencies that exist among projects, where for example, delays in one can
cause cascading delays in others.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

IRS Has Made Important Progress, But Does Not Yet Have Effective

* Effective configuration management is an essential control for ensuring
the integrity and consistency of system modernization program and project
products throughout their life cycles.

* In June 2001, we reported15 BSM configuration management was ineffective.
Accordingly, we made recommendations to address this weakness.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* IRS has made important progress in addressing our recommendations. For
example, it has

* identified configuration items and components to be controlled for BSM
projects,

* had the PRIME revise its program configuration management plan and
procedures, which was completed in July 2001,

* trained contractor and selected BSM program office personnel in
configuration management,

* established baselines for approved program and project products,

* initiated periodic internal configuration management process audits on
selected projects to ensure plans and procedures are being followed, and

* established configuration control boards, including defining thresholds
governing which board should address which request.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* However, effective configuration management, as we previously recommended,
requires an enterprise-wide configuration management process, including
plans and procedures governing PRIME, BSM program office, and IRS IT
services organization products and systems. At a minimum, this also involves

* establishing an integrated program-wide configuration library (i.e., a
repository),

* defining and implementing a program-wide configuration status account
reporting mechanism, and

* identifying configuration items and components for current (legacy)
systems affected by near-term (2002 and 2003) BSM project releases.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* IRS officials acknowledge these steps have not been completed but have
plans to do so by March 2002, except for the 2003 project releases, for
which a date has not yet been established.

* IRS officials also stated that these steps have not yet been fully
implemented because all the steps needed to implement an effective
configuration management process were not originally understood.

* While IRS has made progress in implementing effective configuration
management, until it fully does so, IRS cannot adequately assure that
systems are being designed and developed in accordance with enterprise-wide
needs and requirements, and thus the likelihood that projects will
eventually require expensive re-work is increased.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

IRS Quality Assurance Efforts Have Identified PRIME Problems 
* Quality assurance (QA) provides independent assessments of whether
management process requirements are being followed and whether product
standards and requirements are being satisfied.

* In 2000, IRS established a QA organization within its BSM program office
to, among other things, determine how well

* the BSM program office was following ELC acquisition management processes
and product standards and

* the PRIME's QA function was performing its role of assessing the PRIME's
adherence to quality processes and standards in producing BSM deliverables.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* IRS reviewed the PRIME's QA function, and in March 2001, reported 5 major
findings and concluded that the PRIME's QA was not always effective.
Examples of findings were that the PRIME's QA

* had not planned its work adequately, which contributed to inadequate
process audits and product reviews,

* had not defined program and project metrics for assessing process and
product quality, and

* was not organizationally independent.

* To address these weaknesses, IRS defined 24 corrective actions for the
PRIME. As of November 2001, IRS reported that the PRIME had implemented 7 of
the corrective actions, but 17 remained open. Examples of open actions are

* Ensure that QA reviews and approves deliverables before they are
transmitted to IRS

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* Require PRIME program and project quality plans to adhere to ELC
standards.

* Until the PRIME corrects these weaknesses, the probability of PRIME
deliverables not meeting expectations is unnecessarily increased, and the
level of IRS contract oversight and control needed to compensate for these
weaknesses increases.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

Despite Progress, Risk Management Not Yet Fully Implemented 
* A risk represents a potential problem. The purpose of risk management is
to identify program and project risks before they result in a problem,
appropriately respond to a risk based on its significance (probability of
occurrence times its potential impact), and to actively manage it.

* Effectively managing these risks is one way to minimize the chances of
program and project cost, schedule, and performance problems occurring.
Consistent with our efforts to constructively engage with IRS on BSM, in
1999 we orally recommended to IRS' former CIO that IRS implement this
capability across the BSM program and projects.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* IRS has defined risk management policies and procedures for its ELC. These
included processes to

* identify risks,

* evaluate  probability of  occurrence and potential impact  on each project
and across the BSM program,

* develop risk mitigation plans and track their disposition,

* establish a repository to maintain an inventory of identified risks, and

* report on the status of risks and mitigation plans.

* Despite  these steps, IRS has  yet to fully implement  its risk management
program policies and procedures. For example,

* IRS'  inventory of BSM risks does not identify  all known risks (e.g. lack
of mature software processes, lack of effective configuration management).

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* Risk plans and their disposition are not in all cases being tracked and
updated.

* To strengthen risk management, IRS hired a senior executive in June 2001
to implement effective risk management. This official

* attributed IRS' delay in implementing risk management to a lack of
incentives to report risks, and

* created an action team to review IRS' risk management procedures and
develop a plan for "revitalizing" the program, including incorporating
incentives for escalating risks.

* IRS plans to have risk management fully implemented by February 2002.

* Until this is done, IRS cannot assure it is adequately managing program
and project risks. Thus, the likelihood of BSM projects experiencing further
cost, schedule, and performance shortfalls is increased.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

IRS Has Established Processes To Certify Alignment of BSM Projects

* EA is an institutional blueprint defining how an enterprise operates
today, in both business and technology terms, and how it wants to operate at
some point in the future. An EA also includes a roadmap for transitioning
between these environments.

* Since 199516, we have recommended that IRS

* define and implement an EA to guide and constrain the acquisition of its
modernized systems and

* map BSM projects to the EA to ensure they are built in accordance with the
EA, which reduces the risk of expensive rework, especially after projects
have begun detailed design and development.

* We reported in our June 2001 report17 that while IRS has developed EA
versions 1.0 and 1.1, it had not yet performed the requisite mappings.

16For example, Tax Systems Modernization: Management and Technical
Weaknesses Must Be Corrected If Modernization Is to Succeed 43
(GAO/AIMD-95-156, July 26, 1995).
17Business Systems Modernization: Results of Review of IRS' March 2001
Expenditure Plan (GAO-01-716, June 29, 2001).

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* Accordingly, we also recommended that IRS not approve projects exiting
Milestone 3 until the required assessments were performed to certify that
projects are aligned with the EA.

* IRS has taken important steps to implement this recommendation. For
example

* In September 2001, IRS issued processes for certifying project compliance
with the EA at Milestone 3, before projects proceed with detailed design and
development.

* Nevertheless, IRS has not yet certified all BSM projects that are at or
past Milestone 3 via this process and the current EA version, including

* Security and Technology Infrastructure Release,

* Internet Refund and Fact of Filing (IRFoF), and

* Custodial Accounting Project.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* IRS officials stated these projects were delayed in being certified
because issuing a draft of EA version 2.0 in October 2001 for comment was
their highest priority. Also, STIR and CAP were certified compliant with an
earlier EA version. IRS plans to complete certification of these projects by
late-December 2001.

* However, until this is done, IRS cannot provide assurance that the project
requirements and design are properly aligned with the EA. Without this
alignment being established early and continuously throughout a project's
life cycle, IRS increases the chances that expensive project rework will be
required to meet IRS business and systems needs.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

Weaknesses in Controls and Capabilities Increase Risks 
* Weaknesses in any one of the aforementioned modernization management
controls introduces an unnecessary element of risk to the BSM program, but
the combination of these weaknesses introduces a level of risk that
increases exponentially over time. Given that the fifth plan provides
funding for later phases of key projects, continued development of other
projects, and starting of new projects, it is likely that BSM projects will
encounter additional cost, schedule, and performance shortfalls. This
combination of circumstances and events is represented in the following
updated graphic.

* As discussed later in this briefing, IRS reported in the fifth plan that
BSM projects have already encountered cost, schedule, and/or performance
shortfalls in meeting commitments made in the fourth plan. In those cases
where the fifth plan cited a reason for the shortfall, our analysis of the
reasons showed that weak management controls were a contributor--either via
(1) proactive prudent IRS decision making not to start or continue projects
because of immature controls (for example, see Reporting Compliance on p. 51
of this briefing) or (2) immature controls allowed project shortfalls to
occur.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* IRS acknowledges these risks. According to the CIO, until the weaknesses
are fully addressed, IRS is mitigating them by

* relying on existing immature processes,

* leveraging the knowledge, skills, and abilities of experienced senior
executives to make sure project issues are proactively managed, and

* hiring additional experienced executives.

* In our view, reliance on a combination of immature processes and
individual capabilities and heroic efforts is not a recipe for success. Past
government and industry experience shows the probability of repeating
successes on projects using this approach is low.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

Observation 1: Plan Discloses and Explains Project Cost Changes, But Continues to Omit Changes to Expected Benefits 
* In our June 1999 report on IRS' first plan,19 we recommended that IRS, in
future expenditure plans, report progress against incremental project
commitments. Since then, we have reported with each plan that IRS has
improved its reporting but has not included progress on all incremental
project commitments, such as promised system capabilities and expected
system benefits.

* In the fifth plan, IRS disclosed that 18 projects have experienced cost
and/or schedule shortfalls against commitments made in its fourth and other
prior plans.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* Of the 18,

* 7 were program management initiatives that had cost increases ranging from
1% to 84% ($150,000 to $1.9 million), and

* 11 were system acquisition projects, of which

* 10 experienced schedule delays ranging from 1.5 months to 14 months; 6 of
the 10 experienced delays of 4 months or more; and 1 project had its
schedule shortened by 4 months.

* 8 experienced cost increases ranging from 4% to 66% ($500,000 to $6.1
million); 3 had cost decreases ranging from 4% to 23% ($365,000 to $4.1
million).

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* In addition, since the plan was completed, other slippages have occurred.
For example, on October 23, 2001, IRS moved STIR's Milestone 4 from October
2001 to January 2002, increasing its reported schedule slippage from 1
months to 4 months.

* According to IRS officials, the omission of such scope and benefit
information was an oversight, and they plan to correct it in the next
expenditure plan.

* By not fully providing scope and benefit change information, congressional
oversight of IRS modernization management performance and accountability is
constrained.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

Observation 2: IRS Beginning to Develop BSM Human 
* As we have previously reported21, strategic human capital centers on
viewing people as assets whose value to an organization can be enhanced
through investment. As the value of people increases, so does the
performance capacity of the organization. To maintain and enhance the
capabilities of IT staff, organizations should, among other things,

* assess knowledge and skills needed to effectively perform IT operations to
support agency mission and goals,

* inventory the knowledge and skills of current IT staff,

* identify gaps between requirements and current staffing, and

* develop and implement plans to fill the gaps.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* IRS has begun to address this issue. For example, it has hired a human
capital specialist to develop a plan by January 2002, for defining and
implementing an IT human capital strategy.

* Until IRS develops and implements this strategy, it will not know whether
it has the right IT knowledge and skills to effectively manage the BSM
program. Without this, IRS increases the risk of BSM program and project
cost, schedule, and performance shortfalls.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

Observation 3: IRS' Integrated Master Schedule Has Yetto be Finalized

* The Integrated Master Schedule is an important tool for managingIRS'
acquisition of modernized systems. According to IRS, it is to

* specify about 20,000 major tasks and associated schedules, involving a
dozen organizations, for acquiring and implementingthe portfolio of BSM
projects, and

* identify the numerous and complex dependencies across these projects.

* IRS officials recognize the importance of having such an
integratedschedule and committed to completing one before they
beginimplementing FY 2002 system releases (scheduled to begin inJanuary
2002). However, despite important progress, IRS has not yet completed the
schedule because

* implementation plans (through milestone 5) have not yet beendeveloped for
all BSM projects to be implemented in FY 2002,

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* all change requests that impact BSM projects baselines have not been
analyzed, approved, and incorporated into the schedule, and

* 33 of 61 changes that need to be made to existing (legacy) systems have
not yet been analyzed, approved, and incorporated into the schedule.

* IRS officials stated that they plan to have the Integrated Master Schedule
completed by late December 2001.

* They attributed the delay in finalizing the schedule to this being the
first time IRS has attempted to develop a management tool of this size and
complexity, and it has taken longer than anticipated.

* Until these steps are completed, IRS does not have the means to adequately
manage the interdependencies within and among projects, thus increasing the
risk of project delays, overruns, and rework.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

Observation 4: IRS Has Not Yet Implemented Effective Schedule Estimating Practices 
* Producing reliable estimates of expected costs for program-level
activities and projects is essential to determining a project's
cost-effectiveness. Without this information, the likelihood of poor
investment decisions is increased.

* As part of our ongoing constructive engagement with IRS, we have orally
recommended to the former and current IRS CIOs that IRS adopt effective cost
and schedule estimating practices. SEI defines such practices in its model
for evaluating organizational cost and schedule estimating capabilities.22
The practices include having

* a historical database,

* structured processes for estimating product size and reuse,

* extrapolation mechanisms,

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* audit trails,

* integrity in dealing with dictated costs and schedules, and

* data collection and feedback processes.

* IRS officials recognize the importance of each of these practices.
However, IRS is not currently performing them.

* Consequently and as has been the case in the prior plans, the cost and
schedule estimates in IRS' fifth plan are contractor-provided, "rough order
of magnitude" estimates, that have not been subjected to meaningful,
reliable validation by IRS.

* IRS officials stated that heretofore they have not been able to dedicate
time to implementing effective estimating practices because of other
competing program and project priorities.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                   Results

* However, IRS has tasked the PRIME to develop a plan to implement an
estimating capability, that is to include

* Selection of an estimating method,

* Development of a plan for implementing the method by mid-January 2002, and

* Implementation of the plan beginning by the end of February 2002.

* IRS then plans to develop and implement an approach to oversee PRIME
estimating efforts by February 2002.

* IRS officials acknowledge that IRS' lack of an effective estimating
process has contributed to project delays and cost overruns. Without
improved estimation practices, these project shortfalls are likely to
continue.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

* IRS' fifth plan satisfies the legislative conditions. However, what has
and continues to challenge IRS is implementing the planned management
controls and capabilities. Since our last report, IRS has made important
progress in implementing pockets of modernization management capability.
However, this capability is still not where it should be because it has not
received the same level of priority and attention as BSM system projects. In
our view, these modernization control weaknesses will continue to put IRS at
risk of building systems that may not perform as intended, and/or cost more
and take longer than necessary to complete.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

* Moreover, the risks facing IRS have and will continue to become more
severe as projects begin to be built and implemented (ELC Milestones 3 and
4). Consequently, we are particularly concerned about those projects that
have or are going to proceed beyond these milestones. This concern is
heightened because (1) several of these projects are to provide the
foundational infrastructure upon which later projects depend, and (2) these
projects are already beginning to experience cost and schedule delays.
Exacerbating this situation are IRS' plans to simultaneously start more
projects while confronting these other challenges.

* IRS' risk mitigation strategy of relying on experienced executives until
control weaknesses are corrected is at best a stop gap measure and is not
sufficient given that IRS' timeline for addressing software acquisition
weaknesses extends 2 years. Until IRS addresses its management control
weaknesses, it will expose BSM to unnecessary risk.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

* IRS' fifth plan also does not fully provide whether projects' scope and
expected benefit commitments have changed. Such information is critical to
congressional oversight of IRS modernization management performance and
accountability.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

To address the escalating risks facing IRS on its BSM program, we recommend
that the Commissioner of Internal Revenue

* reconsider the planned scope and pace of the BSM program as defined in the
fifth expenditure plan with the goal of better balancing the number of
system acquisition projects underway and planned with IRS' capacity to
manage this workload. At a minimum, the Commissioner's reconsideration
should include

* slowing ongoing projects and/or delaying new project starts to reduce BSMO
resource demands,

* making correcting modernization management weaknesses a top priority and a
matter of top management attention, and

* reapplying resources (financial and human capital) available from slowed
and delayed projects toward correction of control weaknesses.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                              Executive Action

* To that end, we further recommend that the Commissioner do the following
with respect to each of modernization management weaknesses that we
identified.

* First, for software acquisition management,

* immediately assess CADE, STIR, and e-Services against SEI SA-CMM level 2
requirements,

* based on this assessment, develop a plan for correcting identified
weaknesses for these projects, including having an independent CMM
evaluation performed on these projects before submission of the next BSM
expenditure plan,

* submit with the next expenditure plan, the results of this independent
evaluation, along with a plan for ensuring that all BSM projects that have
passed milestone 3 will meet CMM level 2 requirements, and

* require all projects that have not passed milestone 3 as of December 31,
2001, to be assessed as CMM level 2 and have a plan for correcting any
project weaknesses found as a condition of milestone 3 approval.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                              Executive Action

* Second, for configuration management, risk management, EA implementation,
human capital strategic management, integrated program scheduling, and cost
and schedule estimating, ensure that commitments discussed in this briefing
for addressing residual weaknesses are implemented as planned, and report
any deviations, from these planned commitments to IRS' appropriations
subcommittees.

* Third, until PRIME quality assurance weaknesses are corrected, increase
the level of IRS oversight, scrutiny, and quality assurance of PRIME
activities.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                              Executive Action

* In addition, to allow for effective congressional oversight of the
program, we reiterate our prior recommendation that the Commissioner report
to IRS' appropriations subcommittees on any changes to expenditure plan
commitments concerning system requirements/capabilities to be delivered and
the associated benefits to be realized, and continue to report such
performance measures in future expenditure plans.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

* In commenting on a draft of this briefing, BSM executives stated that they
generally agreed with our findings, conclusions, and recommendations.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

Appendix I: IRS' Expenditure Plan

BSM Spending Plan, Fiscal Year 2002 ($000)

        Proposed Modernization      Milestone
             Initiatives           Milestone                Amount
                                                 Date     Requested
             Program Level
             Activities*
                                                      Nov.
          Program Management         FY          02$7,918
        (formerly Prime Program
          Management Office)
                                                      Nov.
         Business Integration        FY         02$10,461
            Architecture &                       Nov. 02$3
             Integration             FY              ,539
                                                      Nov.
         Management Processes        FY         02$10,082
                                                      Nov.
            FFRDC (MITRE)            FY         02$18,070
                                                            $79,070
          Core Infrastructure
          Support Projects*
        Security and Technology                        May
        Infrastructure Releases      FY         02$43,973
         Infrastructure Shared                        Nov.
               Services              FY         02$39,747
          Enterprise Systems                          July
              Management             FY           02$11,32        3
        Development Integration
        & Testing Environment        FY          May 02$12     ,916
                                                           $107,959

*Program Level and Core Infrastructure Support Projects are funded on a
fiscal year (FY) basis rather than by milestone.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                 Appendix I:

                                                                                                                 Data Projects
                                                                                                                                                                
                                                                                                      Customer Account Data Engine -IMF R1                      
                                                                                                                                                                
                                                                                                      Customer Account Data Engine -IMF R3                      
                                                                                                          Enterprise Data Warehouse R3                          
                                                                                                                                                                
                                                                                                                                                                
Integrated Financial Services/ Core Financial Systems                                                                                                           
                                                                                                                                                                
                                                                                                               Business Projects
                                                                                                                                                                
                                                                                                                   IR/FoF R1                                    
                                                                                                                e-Services 2002                                 
                                                                                                          Customer Account Management                           
                                                                                                          Filing & Payment Compliance                           
                                                                                                        Reporting Compliance -Individual                        

                                                                                                                   HR Connect                                   
                                                                                                                                                                
                                                                                                                                                                
������������������������������ MS2,3 Feb. 04  
                                                                                                  Total Business Systems Modernization Program                  ���������� MS2,3 Feb. 04  

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                            Spending Plan                   Results of GAO Review
                               * The plan satisfied the legislative
                               conditions for the use of ITIA funds and
                               was consistent with our open
                               recommendations. * The plan was an
                               appropriate first step, but the key to
  1st Spending Plan (May 1999) success would be effective implementation
     ($35 million request)     of the plan. * Future plans should specify
                               progress against prior plan commitments,
                               and the next plan should clarify
                               IRS/contractor roles and responsibilities.
                               (See Tax Systems Modernization: Results of
                               Review of IRS' Initial Expenditure Plan,
                               GAO/AIMD/GGD-99-206, June 15, 1999)
                               * The plan raised concerns about projects
                               that were scheduled to begin detailed
                               design and software development before,
                               among other things, the enteprise
                               architecture was completed and the ELC was
    1st Interim Spending Plan  defined and implemented. * IRS should
     (Dec 1999) ($33 million   expedite completion of the architecture and
            request)           implementation of the ELC. * Future plans
                               should explain how IRS plans to manage the
                               risk of performing detailed design or
                               development work if the architecture is not
                               sufficiently completed or the ELC is not
                               sufficiently implemented.

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                Appendix II:

          Spending Plan                   Results of GAO Review
                                *� IRS met relatively few
                                commitments in its $35 million first ITIA
                                spending plan, even though the Service
                                later received an additional $33 million
                                and nearly 5 months of extra time to
                                accomplish the goals set forth in the
                                first plan. *� The plan satisfied
                                the legislative conditions for the use of
                                ITIA funds, and was generally consistent
  2nd Spending Plan (Mar 2000)  with recommendations contained in our
  ($176 million request)        earlier reports. *� The key to
                                success would be whether IRS effectively
                                implements the plan. *� Until IRS
                                completes its initiated actions to
                                redirect and restructure its
                                modernization effort, it would continue
                                to lack key modernization and technical
                                controls. (See Tax Systems Modernization:
                                Results of Review of IRS' March 7, 2000,
                                Expenditure Plan, GAO/AIMD-00-175, May
                                24, 2000)
                                *� IRS had not adhered to the
                                approved and funded March 7, 2000,
                                spending plan. *� On selected
                                initiatives, IRS had not met cost and
                                schedule commitments made in its March 7,
                                2000 spending plan. *� Most
                                modernization initiatives had
                                nevertheless made important progress
                                since March 2000. IRS fully addressed two
                                of its modernization management
                                capability weaknesses, and it was making
                                progress in addressing others. *�
                                One project, Custodial Accounting Project
                                (CAP), had been approved for product
                                development without sufficient definition
    2nd Interim Spending Plan   and without a compelling business case.
     (Aug 2000) ($33 million    Further investment in CAP should be
            request)            limited until IRS demonstrates sufficient
                                business value and reports to the House
                                and Senate committees on risk mitigation.
                                *� Another project, Security and
                                Technology Infrastructure Release (STIR),
                                was being preliminarily designed without
                                sufficient requirements definition and
                                economic justification. The STIR project
                                should be directed to complete a security
                                risk assessment as soon as possible, and
                                ensure that STIR requirements and the
                                proposed design solution are economically
                                justified through a business case. (See
                                Tax Systems Modernization: Results of
                                Review of IRS' August 2000 Interim
                                Spending Plan, GAO-01-91, November 8,
                                2000)

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                Appendix II:

         Spending Plan                   Results of GAO Review
                             * IRS' plan satisfied the legislative
                             conditions for the use of ITIA funds, and
                             was making important progress towards
                             satisfying the congressional direction on
                             two projects - CAP and STIR. * IRS was
                             making important progress in establishing
                             effective modernization management
                             capability, but important and challenging
                             work remained. Until IRS completed its
                             initiated actions to fully implement its
                             system life cycle methodology and business
                             systems modernization office, and resolve
                             issues concerning the completeness and
                             accuracy of enterprise architecture, it
                             continued to lack key modernization and
  3rd Spending Plan (Oct     technical controls. * Five modernization
  2000) ($200 million        initiatives experienced schedule delays
  request)                   and/or cost increases. However, the third
                             plan did not address whether projects' prior
                             commitments for delivery of promised systems
                             capabilities (requirements) and
                             benefit/business value were being met. * IRS
                             used contractor-provided "rough
                             order-of-magnitude" estimates in preparing
                             the third expenditure plan. IRS planned to
                             validate the third plan's estimates as part
                             of its process to negotiate and definitize
                             contract task orders. Previously, this
                             process resulted in finalized contract costs
                             below the estimates, totalling $9 million.
                             (See Tax Systems Modernization: Results of
                             Review of IRS' Third Expenditure Plan,
                             GAO-01-227, January 22, 2001)

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                Appendix II:

          Spending Plan                   Results of GAO Review
                                * IRS' plan satisfied the conditions
                                specified in the appropriations acts. *
                                IRS continued to make important progress
                                in implementing modernization management
                                controls and capabilities. Nevertheless,
                                IRS' modernization management capacity is
                                still not where it should be, given (1)
                                the number of systems acquisition projects
                                that the March 2001 plan identifies as
                                underway and planned and (2) the fact that
                                several of the ongoing projects are
                                entering critical stages in their life
                                cycles. For example, IRS did not have a
                                sufficiently defined version of the
                                enterprise architecture to guide and
                                constrain projects, and employing rigorous
                                configuration management practices. * Due
                                to missing management capacity, key IRS
                                projects were beginning to experience
                                cost, schedule, and performance shortfalls
                                against the commitments the agency made in
                                its third expenditure plan. For example,
                                deployment of the Customer Communications
                                2001 project was three months behind
  4th Spending Plan (March      schedule, and promised system capabilities
  2001) ($128 million request)  and associated benefits had been deferred.
                                Also, a critical infrastructure project,
                                STIR, was reported to be 1.5 months late
                                in trying to complete its preliminary
                                design phase (Milestone 3); and the agency
                                was still working to finalize 6 of 19 work
                                products needed to complete the phase.
                                Thus, the project was actually almost five
                                months late. * IRS officials recognized
                                the need to address its modernization
                                management capacity before key ongoing
                                projects moved into critical life-cycle
                                phases, and before additional projects
                                were started. Accordingly, IRS planned or
                                had initiated steps to address these
                                weaknesses. In particular the Commissioner
                                had decided to slow ongoing and new
                                projects, giving priority to putting in
                                place missing management capacity. We
                                believed this decision was prudent and
                                appropriate and made recommendations to
                                ensure IRS followed through on this
                                decision. (See Business Systems
                                Modernization: Results of Review of IRS'
                                March 2001 Expenditure Plan, GAO-01-716,
                                June 29, 2001)

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

Appendix I
Briefing Slides from December 10 and 11,
2001, Briefings to the Senate and House
Appropriations Subcommittee Staffs

                                Appendix II

                 Comments from the Internal Revenue Service

Appendix II Comments from the Internal Revenue Service

Appendix II Comments from the Internal Revenue Service

Appendix II Comments from the Internal Revenue Service

Appendix III

                   GAO Contacts and Staff Acknowledgments

GAO Contact Gary N. Mountjoy, (202) 512-6367

Acknowledgments   In  addition  to the  individual  named  above, other  key
contributors  were  Bernard  R. Anderson,  William  G.  Barrick, Timothy  D.
Hopkins, Ona  M. Noble, Pietro L.  Salatti, Aaron W. Thorne,  and William F.
Wadsworth.

GAO's Mission The General Accounting Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to good
government is reflected in its core values of accountability, integrity, and
reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents is through the
Internet. GAO's Web site (www.gao.gov) contains abstracts and full-text
files of current reports and testimony and an expanding archive of older
products. The Web site features a search engine to help you locate documents
using key words and phrases. You can print these documents in their
entirety, including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its Web
site daily. The list contains links to the full-text document files. To have
GAO E-mail this list to you every afternoon, go to www.gao.gov and select
"Subscribe to daily e-mail alert for newly released products" under the GAO
Reports heading.

Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out to
the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:

U.S. General Accounting Office
P.O. Box 37050
Washington, D.C. 20013

To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061

Visit GAO's Document GAO Building

Room 1100, 700 4th  Street, NW (corner of 4th and G Streets, NW)Distribution
Center Washington, D.C. 20013

To Report Fraud, Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm,Waste, and Abuse in E-mail:
[email protected], orFederal Programs 1-800-424-5454 or (202) 512-7470
(automated answering system).

Public Affairs Jeff Nelligan, Managing Director, [email protected] (202)
512-4800 U.S. General Accounting Office, 441 G. Street NW, Room 7149,
Washington, D.C. 20548

                             Presorted Standard
                             Postage & Fees Paid
                                     GAO
                               Permit No. GI00

United States
General Accounting Office
Washington, D.C. 20548-0001

Official Business
Penalty for Private Use $300

Address Correction Requested
*** End of document. ***