Tax Systems Modernization: Results of Review of IRS' Third Expenditure Plan (Letter Report, 01/22/2001, GAO/GAO-01-227). This report reviews the Internal Revenue Service's (IRS) third expenditure plan for its systems modernization project. GAO found that the plan satisfied the conditions specified in Treasury's 1998 and 1999 appropriations acts and that IRS was making progress towards satisfying Congress' direction on the Custodial Accounting Project (CAP) and Security and Technology Infrastructure Release (STIR) Project. Although IRS has made significant progress in establishing effective modernization management capability, important and challenging work remains to ensure that the systems work as intended. GAO also found out that five modernization initiatives experienced schedule delays or cost increases, each of which IRS disclosed in the third plan. However, the third plan did not address whether projects' prior commitments for delivery of promised systems capabilities and benefit/business value were being met. IRS used contractor-provided "rough order-of-magnitude" estimates in preparing its third expenditure plan. However, consistent with its established practice, IRS planned to validate the third plan's estimates as part of its negotiating and definitizing contract task orders. For IRS' second expenditure plan, this process resulted in finalized contract costs that were $9 million under the "rough order-of-magnitude" estimates in the plan. --------------------------- Indexing Terms ----------------------------- REPORTNUM: GAO-01-227 TITLE: Tax Systems Modernization: Results of Review of IRS' Third Expenditure Plan DATE: 01/22/2001 SUBJECT: Appropriation accounts Systems conversions Budget outlays Information resources management Tax administration systems Future budget projections IDENTIFIER: IRS Information Technology Investments Account IRS Custodial Accounting Project IRS Security and Technology Infrastructure Release Project IRS Tax Modernization Program ****************************************************************** ** This file contains an ASCII representation of the text of a ** ** GAO Testimony. ** ** ** ** No attempt has been made to display graphic images, although ** ** figure captions are reproduced. Tables are included, but ** ** may not resemble those in the printed version. ** ** ** ** Please see the PDF (Portable Document Format) file, when ** ** available, for a complete electronic file of the printed ** ** document's contents. ** ** ** ****************************************************************** GAO-01-227 A Report to Congressional Committees January 2001 TAX SYSTEMS MODERNIZATION Results of Review of IRS' Third Expenditure Plan GAO- 01- 227 Lett er January 22, 2001 The Honorable Ben Nighthorse Campbell Chairman The Honorable Byron L. Dorgan Ranking Minority Member Subcommittee on Treasury and General Government Committee on Appropriations United States Senate The Honorable Jim Kolbe Chairman The Honorable Steny H. Hoyer Ranking Minority Member Subcommittee on Treasury, Postal Service and General Government Committee on Appropriations House of Representatives Pursuant to the Department of Treasury's fiscal year 1998 and 1999 appropriations acts, 1 the Internal Revenue Service (IRS) submitted to the Congress in October 2000 its third expenditure plan, requesting $200 million from its systems modernization appropriations account, referred to as the Information Technology Investments Account (ITIA). As required by the acts, we reviewed the plan. Our objectives were to (1) determine whether the third plan satisfied the conditions specified in the acts; 2 (2) determine IRS progress in response to the subcommittees' 1 The fiscal year 1998 Treasury and General Government Appropriations Act (Public Law 105- 61) and the fiscal year 1999 Omnibus Consolidated and Emergency Supplemental Appropriations Act (Public Law 105- 277). 2 Per the acts, ITIA funds are unavailable until IRS submits to the Congress for approval a modernization expenditure plan that (1) implements IRS' Modernization Blueprint (IRS' enterprise architecture); (2) meets the Office of Management and Budget's (OMB) system investment guidelines; (3) meets IRS life- cycle management requirements; (4) is reviewed and approved by IRS, Treasury, and OMB, and is reviewed by GAO; and (5) meets federal acquisition requirements and management practices. September 28, 2000, direction on the Custodial Accounting Project 3 (CAP) and the Security and Technology Infrastructure Release (STIR) project; 4 and (3) provide any other observations about the third plan and the systems modernization program. On November 9, 2000, we briefed your offices on the results of our review. This report transmits our November 9, 2000, briefing and reiterates our recommendations to the Commissioner of Internal Revenue that were specified in the briefing. The full briefing, including our scope and methodology, is reprinted in appendix I. In summary, we made four major points: ? First, IRS' third expenditure plan satisfied the conditions specified in the appropriation acts, and IRS was making progress towards satisfying the subcommittees' direction on the CAP and STIR projects. ? Second, IRS was still making important progress in establishing effective modernization management capability, but important and challenging work remained. For example, IRS had defined its system life- cycle methodology, which IRS refers to as its Enterprise Life Cycle, and planned to have it implemented by early 2001. IRS had also created a modernization management program office and planned to have it fully functional by early 2001. In addition, IRS had developed a draft enterprise architecture but still needed to resolve significant issues concerning its completeness and accuracy. Until these and other modernization management weaknesses were fully addressed, we concluded, key modernization controls would be missing, putting IRS at risk of building systems that might not perform as intended, might cost more, and might take longer to complete. ? Third, five modernization initiatives experienced schedule delays and/ or cost increases, each of which IRS disclosed in the third plan. However, the third plan did not address whether projects' prior commitments for delivery of promised systems capabilities (requirements) and benefit/ business value were being met. 3 CAP is one of a collection of systems that make up IRS' Integrated Financial System Project. CAP is designed to provide tax receipt and receivable analysis and reporting. Standard general ledger and other financial and administrative reporting as required by federal management directives are to be provided by other future projects. 4 STIR is the common integrated infrastructure to support and enable modernized business systems applications. As designed, it consists of a combination of custom and commercialoff- the- shelf software, hardware, and security solutions, integrated to form the technical foundation upon which modernized business systems applications will operate. ? Last, IRS used contractor- provided “rough order- of- magnitude” estimates in preparing its third expenditure plan. However, consistent with its established practice, IRS planned to validate the third plan's estimates as part of its negotiating and definitizing contract task orders. For IRS' second expenditure plan, this process resulted in finalized contract costs that were $9 million under the “rough order- ofmagnitude” estimates in the plan. Recommendations for To ensure that IRS fully responds to congressional direction and addresses Executive Action modernization management weaknesses, we recommend that the Commissioner of Internal Revenue ? follow through on plans to satisfy IRS appropriations subcommittees' direction on CAP and STIR; ? expedite the completion of IRS' enterprise architecture releases and implementation of other missing modernization management controls; ? not approve and fund detailed design and development activities for any system before the requisite enterprise architecture definition is completed; ? report immediately to IRS' appropriations subcommittees on any changes to commitments made in IRS' second plan concerning system requirements/ capabilities to be delivered and the associated benefits to be realized, and continue to report such performance measures in future expenditure plans; and ? report to IRS' appropriations subcommittees on any variance from cost estimates in its third plan of 10 percent or more that result from definitization of contract task orders. Agency Comments In commenting on a draft of this report, the Commissioner of Internal Revenue agreed with our findings and recommendations. The Commissioner's comments are reprinted in appendix II. We are sending copies of this report to Senator Max Baucus, Senator Robert C. Byrd, Senator Orrin G. Hatch, Senator Joseph I. Lieberman, Senator Ted Stevens, and Senator Fred Thompson, and to Representative Dan Burton, Representative William J. Coyne, Representative Amo Houghton, Representative David R. Obey, Representative Charles B. Rangel, Representative Jim Turner, Representative Henry A. Waxman, and Representative C. W. Bill Young, in their capacities as Chairmen or Ranking Minority Members of Senate and House Committees and Subcommittees. We are also sending copies to the Commissioner of Internal Revenue; the Secretary of the Treasury; the Chairman of the IRS Oversight Board; and the Director of the Office of Management and Budget. Copies will also be made available to others upon request. Should you or your staff have any questions on matters discussed in this report, please contact me at (202) 512- 3439. I can also be reached by e- mail at hiter@ gao. gov. Key contributors to this report are listed in appendix III. Randolph C. Hite Director, Information Technology Systems Issues Appendi xes Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Appendi x I Subcommittee Staff Information Technology Results of Review of IRS' Third ITIA Expenditure Plan Briefing to Staffs of the Senate Committee on Appropriations, Subcommittee on Treasury and General Government and the House Committee on Appropriations, Subcommittee on Treasury, Postal Service, and General Government November 9, 2000 1 Briefing Overview Introduction ? Objectives ? Scope and Methodology ? Results in Brief ? Background ? Results ? Conclusions ? Recommendations 2 Introduction ? Per IRS' FY 1998 and 1999 appropriations acts, Information Technology Investment Account (ITIA) funds are unavailable until IRS submits to the Congress for approval, a modernization expenditure plan that: ? Implements IRS' Modernization Blueprint (IRS' enterprise architecture); ? Meets OMB IT investment guidelines; ? Meets IRS life cycle management requirements; 1 ? Is reviewed and approved by IRS, Treasury, and OMB, and is reviewed by GAO; and ? Meets federal acquisition requirements and management practices. 1 IRS refers to its life cycle management program as the Enterprise Life Cycle (ELC), which is 3 graphically depicted in appendix I. Introduction ? To date, about $506 million has been appropriated for ITIA, and $249 million has been released. ? IRS plans to submit a series of expenditure plans over the life of the modernization requesting release of ITIA appropriated funds. On October 10, 2000, IRS submitted its third expenditure plan. IRS plans to submit another plan in March 2001. 4 Introduction 3rd Spending Plan 10/ 00 1st Interim 2nd Interim 4th Spending Spending Spending Spending Spending Plan Plan Plan Plan Plan 5/ 99 12/ 99 3/ 00 8/ 00 3/ 01 10/ 98 10/ 99 10/ 00 10/ 01 ITIA Funds Requested $35 $33 $176 $33 $200 ? For Release (millions) ITIA Funds Released $35 $33 $148 $33 ? ? Per Plan (millions) Cumulative Release of $35 $68 $216 $249 ? ? ITIA Funds (millions) 5 Objectives ? As agreed, our objectives were to ? determine whether the third plan satisfies the legislative conditions; ? determine IRS progress in response to the subcommittees' September 28, 2000, direction on the Custodial Accounting Project (CAP) and the Security and Technology Infrastructure Release (STIR) project; and ? provide any other observations about the third plan and the systems modernization program. ? We agreed to provide our results to the subcommittee by November 9, 2000. 6 Scope and Methodology ? To accomplish our objectives, we ? Reviewed the third expenditure plan and met with IRS program officials to understand the scope and content of the plan; ? Analyzed the plan against the legislative conditions to identify variances; ? Assessed IRS' progress and plans for responding to congressional direction on CAP and STIR; ? Reviewed program and project management reports and briefings; ? Observed modernization executive steering committee and subcommittee meetings; and ? Interviewed program and project management officials. 7 Scope and Methodology ? Analyzed available evidence on recent efforts to address modernization management weaknesses. Specifically, we analyzed progress and plans for ? Business Systems Modernization Office (BSMO) implementation, ? enterprise architecture definition, ? ELC definition and implementation, ? investment management definition and implementation, and ? software acquisition maturity, as defined by the Software Engineering Institute's (SEI) acquisition model. This model was developed by the SEI at Carnegie Mellon University to evaluate an organization's software acquisition capability. 8 Scope and Methodology ? Collaborated with the Treasury Inspector General for Tax Administration (TIGTA) to avoid duplication of effort in reviewing program and project initiatives and incorporated TIGTA's results in this briefing where appropriate. Initiatives addressed by TIGTA included the Customer Communications and e- Services projects, ELC, BSMO, and enterprise architecture. ? To meet our agreed upon report date, we did not independently validate planned initiatives' cost estimates or confirm, through system and project management documentation, the validity of IRS- provided information on the initiatives' content and progress. 9 Scope and Methodology ? We provided a draft of this briefing on November 8, 2000, to IRS' Chief Information Officer (CIO), BSMO Director, and other executives and have incorporated their comments where appropriate. ? We performed our work from October through November 2000 in accordance with generally accepted government auditing standards. 10 Results in Brief Objective 1: IRS' third plan satisfies the legislative conditions. ! " #"$ " %& !% !% "$'( )*$ Objective 2: IRS is making progress in responding to congressional direction on CAP and STIR. * *+ ./"0001 *(, - * *+ 2 $ - ./"0001 " " * 2 . (1 ./"0001 3 4 ( ./"0001 * ( " ./"0001 4 2 These acquisition requirements and practices are intended to establish acquisition management rigor and discipline, such as those defined in the 11 Software Engineering Institute's acquisition model. Our analysis of the plan focused on satisfaction of this model's tenets. Results in Brief ? Objective 3: Other Observations ? Observation 1 ? IRS continues to make important progress in establishing effective modernization management capability, but important and challenging work remains. # $ !" % $ 56* 5 ! , " 12 Results in Brief ? One important and challenging task for IRS is completing and implementing its enterprise architecture (EA). ? IRS is developing its EA in 3 releases (1. 0, 1.1, and 2.0). Each release is intended to provide incrementally more architectural definition. For example, 1.0 is to provide infrastructure level definition, and 1.1 is to provide application level definition, including a sequencing plan for strategic acquisition of modern business process applications. ? Thus far, IRS has developed a draft of EA 1.0. However, it still needs to resolve significant issues concerning EA 1.0's completeness and accuracy. For example, the draft does not include all planned EA products and does not fully address how security and privacy needs will be met. 13 Results in Brief ? IRS then needs to have EA 1.0 independently verified and validated. IRS planned to have EA 1.0 completed by September 2000, then slipped the date to November 2000, and currently plans to approve and issue it in December 2000. This slippage will also delay completing EA versions 1.1 and 2.0, but revised issuance dates have not yet been established for either. ? These delays in issuing IRS' EA are significant because IRS' third plan calls for beginning detailed design and development (ELC Milestone III) on certain projects before requisite EA definition is completed. For example, the E- Services projects are scheduled to pass ELC milestone III in November 2000 and February 2001, respectively. However, EA 1.1, which is to provide the necessary business process application architectural definition, was not scheduled for completion until March 2001, and this date will slip. 14 Results in Brief ? Observation 2 ? Five initiatives have experienced schedule delays and/ or cost increases, each of which IRS reported in the plan. For example, ? Tax Vision and Strategy (TAVS) project has slipped 3 months and estimated cost has increased $6.2 million (27 percent). ? E- Services estimated cost increased $1.4 million (26 percent). ? The plan did not, however, address whether these five and other projects' prior commitments for systems capabilities (requirements) and benefit delivery have also been delayed or reduced. This is not consistent with our recommendation for IRS to report progress against all prior commitments in each plan. 3 3 Tax Systems Modernization: Results of Review of IRS' Initial Expenditure Plan( GAO/ AIMD- 99- 15 206, June 15, 1999). Results in Brief ? Observation 3 ? Costs in the third plan are contractor- provided “rough order- of- magnitude” estimates. ? Consistent with IRS' recent actions to address prior subcommittee direction on this issue, IRS plans to validate the third plan's estimates as part of its contract/ task order definitization process. Under this process, IRS assesses contractor task order proposals, develops independent cost estimates for each, and negotiates a task order cost. ? For IRS' second expenditure plan, this process resulted in cumulative contract costs that were $9 million under estimates in the second plan (5 percent), although individual project variances exceeded 10 percent. ? We are making recommendations to address our observations. In commenting on a draft of this briefing, IRS' CIO agreed with our recommendations and plans to implement them. 16 Background ? The third plan seeks approval to obligate about $200 million for both program- level and project- specific activities. 4 Third ITIA Expenditure Plan ($ 000) Program Level Activities PRIME Program Management Office $21, 856 Federally Funded Research and Development Center (MITRE) $18,750 ELC, Quality Assurance, and Configuration Management $7, 394 Architecture Engineering Office $17, 570 Vision and Strategy - Tax Administration $6, 200 Management Reserve $15,000 Subtotal $86,770 Project Level and Infrastructure Activities Customer Services Capabilities $11,663 Custodial Accounting Project $44,130 Core Financial System $3, 449 Security and Technology Infrastructure Release $25,185 Other Enabling Infrastructure $28,849 Subtotal $113, 276 Total Requested Release $200, 046 4 See appendix II for a detailed summary of planned activities. 17 Background ? The third plan continues ongoing initiatives and establishes 3 new efforts: ? BSMO Quality Assurance contractor support; ? Core Financial System; and ? a management reserve. ? Like its previous plans, IRS' third expenditure plan covers contractor costs such as the Prime Systems Integration Support (PRIME) contractor and the Federally Funded Research and Development Center (MITRE), and not IRS internal costs, such as IRS BSMO staff costs. 18 Background ? To date, GAO has reviewed and reported on 2 expenditure plans and 2 “stopgap” spending measures supporting requests for ITIA funding releases. Spending Plan Results of GAO Review 1 st Spending Plan The plan satisfied the legislative conditions for the use of ITIA funds and was (May 1999) consistent with our open recommendations. ($ 35 million request) The plan was an appropriate first step, but the key to success would be effective implementation of the plan. Future plans should specify progress against prior plan commitments, and the next plan should clarify IRS/ contractor roles and responsibilities. 1 st Interim Spending Plan The plan raised concerns about projects that were scheduled to begin detailed (Dec 1999) design and software development before, among other things, the enteprise ($ 33 million request) architecture was completed and the ELC was defined and implemented. IRS should expedite completion of the architecture and implementation of the ELC. Future plans should explain how IRS plans to manage the risk of performing detailed design or development work if the architecture is not sufficiently completed or the ELC is not sufficiently implemented. 19 Background Spending Plan Results of GAO Review 2 nd Spending Plan (Mar 2000) IRS met relatively few commitments in its $35 million first ITIA spending plan, ($ 176 million request) even though the Service later received an additional $33 million and nearly 5 months of extra time to accomplish the goals set forth in the first plan. The plan satisfied the legislative conditions for the use of ITIA funds, and was generally consistent with recommendations contained in our earlier reports. The key to success would be whether IRS effectively implements the plan. Until IRS completes its initiated actions to redirect and restructure its modernization effort, it would continue to lack key modernization and technical controls. 2 nd Interim Spending Plan IRS had not adhered to the approved and funded March 7, 2000, spending plan. (Aug 2000) On selected initiatives, IRS had not met cost and schedule commitments made ($ 33 million request) in its March 7, 2000 spending plan. Most modernization initiatives had nevertheless made important progress since March 2000. IRS fully addressed two of its modernization management capability weaknesses, and it was making progress in addressing others. One project, Custodial Accounting Project (CAP), had been approved for product development without sufficient definition and without a compelling business case. Further investment in CAP should be limited until IRS demonstrates sufficient business value and reports to the House and Senate committees on risk mitigation. Another project, Security and Technology Infrastructure Release (STIR), was being preliminarily designed without sufficient requirements definition and economic justification. The STIR project should be directed to complete a security risk assessment as soon as possible, and ensure that STIR requirements and the proposed design solution are economically justified through a business case. 20 Results Objective 1: Determine whether the third plan satisfies the conditions in IRS' FY 1998 and 1999 appropriations acts. &' & "789 ( ! % 5 +:% " 5(0; 5( : " $4 " : -) ; " .01 & "78 % 56*% % ! 56* 2%% % " "% " < 00 21 Results &' ! " : " !& " ($ 000 % & " 56* " & " #"$ " % -- =%000 & !% & !> ?%000 !% "$'( !> 0%000 '( >/"=%000 )*$ ( 56*% %% % $ 5 . 51 $( * & "2 # "% % "% % 2 " 4 $ < 00% " $ 5" 00% - 2% , "$ 24 5 These are Acquisition Planning, Solicitation, Requirements Development and Management, Project Management, 22 Contract Tracking and Oversight, Evaluation, and Transition to Support. Results Objective 2: Determine IRS progress responding to congressional direction on CAP and STIR CAP GAO Findings Congressional Direction IRS Response In our September 2000 The subcommittees directed In response, IRS: briefing and subsequent that no CAP funds be - is revising the business case with the intent of report, we stated that expended until IRS' CIO and showing compelling business value benefits IRS had prematurely CFO certify and report to the that will be derived from developing CAP, passed milestone III Subcommittees that a plans to have the business case certified by without a compelling compelling business case had the CIO and CFO once it is completed, and business case to justify been established for treating intends to report to the subcommittees in the decision. . In CAP as a near- term priority November 2000. addition, IRS had not and that the risks associated - incorporated CAP into the Integrated Master demonstrated that CAP with post- milestone III Schedul e; was integrated with development and the lack of - moved CAP under the Program Management other modernization program controls were being Process, with formal participation beginning projects. effectively mitigated. in November 2000; According to IRS officials, no CAP funds from the August 2000 funding release have or will be used for post- milestone III work until it has fulfilled congressional direction. 23 Results GAO Findings Congressional Direction IRS Response In our September 2000 The subcommittees directed In response, IRS: briefing and subsequent IRS to complete an SRA, - accelerated development of and completed an report, we stated that validate STIR project SRA and identified new risks: IRS did not have requirements against the - web interface vulnerabilities adequate assurance that resul ts of the SRA, and have - outsourcing internet service provider; it was properly the results certified by the - engaged IRS' security office and Federally designing STIR because Commi ssi oner. The Funded Research and Development (FFRDC) it had not assessed the subcommittees also directed contractor (MITRE) to verify t hat the SRA is project's security threats that the STIR funds approved complete and consistent with IRS' ELC and vulnerabilities, for release should only be guidance, draft EA 1. 0, and its Technology analyzed the resulting used for the risk assessment Model View logical design; risks in terms of until the Commissioner - is currently mapping the results of the SRA to probability and impact, certifies the assessment is the STIR project requirements to ensure that and used this security complete and the results the syst em requirements document is risk assessment (SRA) appli ed to the project. complete and the baseline business case is to develop and justify cost effective; cost effective - plans to obtain the Commissioner's countermeasures. certification of the completed SRA in November 2000. According to IRS officials, no STIR funds from the August2000fundingrelease have or willbe usedfor post- milestone III work until it has fulfilled Congressional direction. 24 Results Objective 3: Other observations about IRS' third plan and its systems modernization program Observation 1: Progress Occurring on Management Weaknesses, But Important and Challenging Work Remains ? Since our September briefing and subsequent report 6 on IRS' last plan, IRS has continued to make important progress in addressing its remaining management weaknesses, and is on schedule for meeting most of its commitments for correcting these weaknesses by January 2000 (e. g., having a fully functional BSMO). However, commitments made in this last plan for addressing lack of an enterprise architecture have slipped. 6 Tax Systems Modernization: Results of Review of IRS' August 2000 Interim Spending Plan 25 (GAO- 01- 91, November 8, 2000). Results # !" $ % $ & (*"2 "4 $ < 00 @" " 00 $ 5" -,5 A000 A "-B-" ,5 56* A"C 56*% 8)-?0 ," 56* < 00 & " 56* 26 Results # !" $ % 5 * #. 12 -/"000 0"$ 2 % 000 @" ,5 "$ @"&5 0 *! 5: " * "5(0 /"000 ! A" ? $ ! * ! % %% < 00 +" % ! , " A"% 4 A 4 , % " * 4 27 Results ? As noted in the prior table, one important and challenging task that remains, and for which commitments have slipped, is issuing the EA 1. 0. ? IRS is developing its EA in 3 releases (1. 0, 1.1, and 2.0). Each release is intended to provide incrementally more architectural definition. As shown in the following table, EA 1.0 is to provide infrastructure level definition, and 1.1 is to provide application level definition, including a sequencing plan for strategic acquisition of modern business process applications. 28 Results Enterprise Architecture Being Developed in Phases Enterprise Architecture 1. 0 Enterprise Architecture 1. 1 Enterprise Architecture 2. 0 ? Builds on and updates Blueprint 97 ? Builds on Enterprise Architecture 1. 0 ? Builds on Enterprise Architecture 1. 1 ? Foundation for incrementally defined, ? Rich in business- specific processes ? Includes IMVS migration strategy incrementally delivered enterprise ? Mid- and Long- Term Enterprise Transition transition specificity ? Rich in Technical Infrastructure and Str ategy ? Shared Services at same level as TAVS 1. 0 Applications Infrastructure (lower pyramid) ? Includes TAVS migration strategy ? Knowl edge Management components ? Lean in business process applications (top of transition specificity incorporated into architecture, requirements, pyramid) ? Includes IMVS content not included in transition strategy ? Provides structural framework for future Enterprise Architecture 1. 0 ? Incorporates other vision and strategy Enterprise Architecture releases ? Complete Data Warehouse strategy results and updates as needed ? Captures and conveys critical high- level ? Key Concept updates requirements ? Business processes fit together fromprefiling ? Knowledge Management to post- filing ? Systems Management ? All systems, applications, data and interfaces ? Software Development derived from business processes ? Networks ? Includes business processes for 2002 projects ? Data ? Subsystemdescriptions from Blueprint 97 ? Business Rules Engines are accounted for ? IMVS input limited to custodial accounting ? Business processes around Knowledge Management ? High- level data warehouse strategy ? Deals with Enterprise Architecture 1. 0 ? Each project scoped by business systems issues/ conditionals ? Directed by Key Concepts on location, portals, systems framework, TRM, data 29 Results ? IRS has developed a draft of EA 1. 0 and circulated it internally for comment (see appendix III for description of framework being used to develop the EA). ? IRS obtained about 950 issues (159 of which IRS reports it has resolved). Some of the remaining issues are significant. ? Once these issues are resolved, IRS still needs to have MITRE assess EA 1.0 completeness and correctness. We reviewed MITRE's planned assessment method and found it to be reasonable. ? IRS then needs to obtain modernization steering committee approval of EA 1.0. ? IRS' revised date for issuing EA 1.0 is December 18, 2000. According to IRS, EA 1.0 slippage will delay EA 1.1 and 2.0. 30 Results Area of Comment Summary of Comments Proposed Resolution Security Lacks a comprehensive security Develop a security overview that incorporates key concepts and identifies overview. security points throughout the architecture. Security needs are not fully reflected in Add security needs to business process definitions. the business process descriptions. Privacy No explicit privacy principles, Clarify/ add privacy principles, constraints and assumptions to enterprise constraints and assumptions. architecture. Lacks a comprehensive privacy view. Create a key concept for privacy. Map security functions to privacy needs. Consider adding Enterprise Requirement( s) for privacy. Business Processes Tax Administration and Vision Strategy Incorporate TAVS operating models into enterprise architecture concept of (TAVS) and enterprise architecture use operations. different process models. Assess need to incorporate Business Operating Division- specific tailoring. Excludes differences among Business Tailor Business Architecture work products where needed. Operating Division business processes. Data Lack of breadth, depth, and accuracy in Extend conceptual data model to address scope of near- term projects. conceptual data model. Complete mapping of data to business processes. Mapping ofdata to businessprocesses Add security details to conceptual data model. incomplete. Security not reflected in the conceptual data model. Business Systems Lacks complete definition of business Complete descriptions of business systems assigned to near- term projects. systems. Complete descriptions of all business systems. Interface definitions are incomplete. Add interface descriptions for business systems assigned to near- term Mapping of systems to business projects. processes incomplete. Complete mapping of systems to business processes. Enterprise Requirements Lacks clear traceability to architecture. No action planned because direct traceability not intended. EA Consistency Products not consistent with each other, Rework products to reflect Technical Reference Model structure and intent, key concepts, and Technical Reference reflect key concepts, and agree with each other. Model. EA Completeness Many work products are incomplete. Complete key work products to the extent necessary to support near- term projects and to resolve major issues impacting approval. Traceability from Blueprint 97 Elements of Blueprint 97 are not all Complete accountability tracing of Blueprint 97 elements into Enterprise traced to Enterprise Architecture. Architecture. 31 Results ? As shown in the following graphic, IRS' third plan calls for beginning detailed design and development (i. e., building) on selected projects in advance of planned EA version releases. It is important that IRS ensure that the relevant and important architectural definition that is planned for each version be completed and used on affected systems before IRS begins building these systems. If it does not, IRS risks building systems that are not architecturally compliant, which could sub- optimize agencywide mission performance. 32 Selected Projects Results Customer Account Data Engine Customer Communications e- Services release 1 e- Services release 2 STIR Core Financial System Enterprise Architecture 1.0 1.0 1.1 2.0 ?? Dece March- June- 01 September- mber- 01 September- 00 00 01 Planned Date For Beginning Detailed Design and Development 33 Results Observation 2: Third Plan Discloses Project Cost and Schedule Changes, But Does Not Report Any Changes to Promised System Capabilities and Benefits ? Five projects have experienced cost increases and/ or schedule delays against commitments made in prior plan. IRS reported such changes in its third plan. Program/ Project 8/ 2000 Commitment Revised Commitment Change (%) Management Initiative Date and Funding Date and Funding ($ 000) ($ 000) TAVS Milestone 1 12/ 31/ 00 3/ 31/ 01 +3 months $22,695 $28,895 $6, 200 (27%) CRM Exam Milestone 3 11/ 07/ 00 12/ 18/ 01 +1.5 months $2, 426 $2, 426 $0 E- Services Milestone 3 2/ 28/ 01 2/ 28/ 01 0 months $5, 480 $6, 918 $1, 438 (26%) Enterprise Systems 1/ 31/ 01 2/ 28/ 01 +1 month Management Milestone $5, 431 $6, 894 $1, 463 (27%) 3 Customer 11/ 30/ 00 2/ 28/ 01 +3 months Communications $3, 509 $3, 509 $0 Milestone 3 (release 2002) 34 Results ? In our June 1999 report on IRS' first plan, 7 we recommended that IRS, in future expenditure plans, report progress against incremental project commitments. ? However, the third plan does not address whether program and project initiatives' scopes (e. g., system requirements sets, benefit expectations) have changed. Such disclosure is critical to understanding progress against incremental project commitments. 7 Tax Systems Modernization: Results of Review of IRS' Initial Expenditure Plan( GAO/ AIMD99- 35 206, June 15, 1999). Results Observation 3: Plan Is Based on Contractor Estimates That Have Not Been Validated by IRS ? The cost estimates in IRS' third plan are contractorprovided, “rough order of magnitude” estimates, and are not based on detailed work breakdown structures of tasks and deliverables. ? Consistent with IRS' recent practice established to address a similar concern we raised with IRS' second plan, IRS plans to validate these estimates as part of its task order definitization process with its PRIME and other contractors. Under this process, the contractor submits task order proposals that include costs, IRS assesses the proposals and develops independent cost estimates, and IRS negotiates a final task order cost. ? For IRS' second plan, this process resulted in some project negotiated costs being more than 10 percent above or below plan estimates. Cumulative negotiated costs were $9 million less than plan estimates (5 percent). 36 Conclusions ? IRS' third plan satisfies the legislative conditions, and IRS is in the process of satisfying its appropriations subcommittees' recent direction for the CAP and STIR projects. By doing so, IRS will be better positioned to make informed decisions about when or how to move forward with these two investments. ? IRS continues to make important progress in correcting modernization management weaknesses, such as implementing the ELC, making the BSMO fully functional, and developing an EA. However, until these weaknesses are fully addressed, key modernization controls will continue to be missing, putting IRS at risk of building systems that may not perform as intended, and/ or cost more and take longer to complete. 37 Conclusions ? As we have consistently reported, these risks are not as severe early in projects' life cycles when they are being planned (project definition and preliminary system design), but escalate as projects are built (detailed design and development). In the case of IRS and its ELC, this point of risk escalation is ELC milestone III. Consequently, we will remain concerned about projects that proceed beyond milestone III before these weaknesses are fully addressed. ? Given that IRS' third plan calls for several projects to pass ELC milestone III within the next several months, it is important for IRS to continue to make implementation of these program management controls a top priority. In particular, completing the EA releases relevant to these projects, fully implementing its ELC, and making its BSMO fully functional, are essential. 38 Conclusions ? While IRS is reporting on performance in meeting project cost and schedule commitments made in prior years, it is not disclosing whether projects' scope and expected benefit commitments have changed. Such information is critical to fully disclosing IRS modernization management performance and establishing accountability. ? The estimates in IRS' plan are contractor provided estimates that have yet to be subjected to government validation. IRS' established process for introducing such validation occurs as part of its contract task order definitization process. This process control is reasonable, but can result in project costs that are significantly different from what IRS' appropriations subcommittees are asked to approve. 39 Recommendations for Executive Action We recommend that the Commissioner of Internal Revenue: ? follow through on plans to satisfy IRS appropriations subcommittees' direction on CAP and STIR; ? expedite the completion of IRS' EA releases and implementation of other missing modernization management controls; ? do not approve and fund post- Milestone III detailed design and development activities for any system prior to completing requisite EA definition; 40 Recommendations for Executive Action ? report immediately to IRS' appropriations subcommittees on any changes to commitments made in IRS' second plan concerning system requirements/ capabilities to be delivered and the associated benefits to be realized, and continue to report such performance measures in future expenditure plans; and ? report to IRS' appropriations subcommittees on any variance from cost estimates in its third plan of 10 percent or more that result from definitization of contract task orders. 41 Agency Comments ? In commenting on a draft of this briefing, IRS' CIO agreed with our conclusions and recommendations and stated IRS plans to: ? satisfy the subcommittees' direction on CAP and STIR with reports on these projects to the subcommittees during November 2000. ? complete the EA 1. 0 and expedite the full implementation of modernization management controls. EA 1. 0 is to be completed and approved during December 2000. All other recommended management controls will be implemented as rapidly as possible. ? approve projects to move into detailed design (beyond milestone 3) only with the requisite EA definition. Project designs will be reviewed for EA compliance and approved by the Director of Architecture and Engineering before exiting milestone 3. ? report in November 2000 on any changes to the requirements/ capabilities/ benefits of projects from the baseline of the March 7, 2000, expenditure plan, and ? report any variance of 10 percent or more due to the definitization of task orders. These reports will be issued quarterly starting in January 2001. 42 Appendix I: Enterprise Life Cycle Support & Operations MS MS MS MS MS 1 2 3 4 5 ELC Milestones Strategy System Enterprise Post Concept Deployment Design Deployment Vision & Strategic Architecture Definition Evaluation 43 Development Integration Deployment Plan Appendix II: IRS' Expenditure Plan Business Systems Modernization - ITIA Spending Plan FY 2001 ($ 000) Milestone Amount Proposed Modernization Initiatives Milestone Date Requested Program Level Activities BSMO Quality Assurance FY Sep- 01 $1, 800 PRIME Program Management Office FY Sep- 01 $21,856 ELC Enhancements and Maintenance FY Sep- 01 $3, 686 FFRDC (MITRE) FY Sep- 01 $18,750 Architecture Engineering Office FY Sep- 01 $17,570 Configuration Management FY Sep- 01 $1, 908 Vision and Strategy - Tax Administration MS1 Mar- 01 $6, 200 Management Reserve FY Sep- 01 $15,000 $86,770 Business Systems Projects CRM Exam (1120 Replacement) FS2001 MS4 Sep- 01 $9, 917 e- Services (Window A) MS4 Oct- 01 $1, 746 Custodial Accounting Project / Taxpayer Account Subledger MS4 Jan- 03 $44,130 Core Financial System (CFS) MS2 Dec- 00 $2,829 MS3 Mar- 01 $620 $59,242 44 Appendix II: IRS' Expenditure Plan Infrastructure Projects Security and Technology Infrastructure Release (STIR) MS4 Aug- 01 $25,185 Enterprise Systems Management (ESM) MS3 Feb- 01 $6,894 MS4 Nov- 01 $9,184 Solutions Demonstration Lab (SDL) FY Sep- 01 $1,759 Virtual Development Environment (VDE) FY Sep- 01 $6,310 Enterprise Integration and Test Environment (EITE) FY Sep- 01 $4,702 $54,034 Total Business Systems Modernization Program $200, 046 45 Appendix III: EA Work Products Enterprise Business Direction Model View Location Model View Organization Model View 01- Enterprise 02- Enterprise 03- Enterprise 12- Location 13- Location- Type 14- Process Location 09- Organization 10- Role Definitions 11- Process Role Business Direction Context Business Concept PCAs Definitions Type Matrix Direction Model Matrix Model Diagrams of Operations System Engineering Model View Applications Model View 32- Application 33- Taxonomy of 34- Enterprise 35- Process/ 36- Enterprise 15- Systems 16- Taxonomy 17- Taxonomy of 18- Enterprise PCAs Application Types Application Application API Definitions Development of Tools Enterprise Standards Matrix Matrix PCAs Standards and Conventions Data Model View 19- Enterprise 20- Service 21- Taxonomy of 22- Taxonomy of Conventions Level PCAs Service Levels Interface Types 27- Data PCAs 28- Taxonomy of Data 29- Data Management 30- Enterprise 23- Reuse 24- Definition of 25- Solution 26- Process/ Systems Approach Conceptual Strategy Business Design Matrix 31- Process Data Model Systems Patterns Data Matrix Business Process Model View Infrastructure Model 04- Business Process 05- Enterprise 06- Business 07- Business 08- Process Thread Principles, Constraints, Process Hierarchy Process Flows Process Definitions Performance Models 45- Infrastructure 46- Taxonomy of 47- Infrastructure and Assumptions (PCAs) PCAs Infrastructure Strategy Elements Technology Model View 48- Infrastructure Concept of 37- Technology PCAs 38- Taxonomy of Technology 39- Technology Insertion 40- Process/ Technology Operations Capabilities Strategy Capability Matrix Enterprise Enterprise Requirements Requirements Security and Privacy Model View 53- Taxonomy of Requirements 54- Requirements Statements and Traceability Linkage 41- Security and 42- Taxonomy of Security 43- Verification and 44- Security Function Privacy PCAs and Privacy Functions Compliance Matrix Enterprise Transition Strategy Management and Other Work Products 55- System 56- Interface 57- Current 58- Near- Term 59- Medium- Term 60- Long- Term 49- Completeness 50- Completeness 51- Taxonomy 52- Enterprise Assignments Assignments Production Sequencing and Sequencing and Sequencing and and Adequacy and Adequacy of Baseline Architecture to Projects to Projects Environment Release Plan Release Plan Release Plan Assessment Assessment Content Risk Management Description (1 - 3 years) (4 - 6 years) (7+ years) Approach Plan 46 Appendi x II Comments From the Internal Revenue Service Appendi x I II GAO Contacts and Staff Acknowledgments GAO Contacts Gary Mountjoy, (202) 512- 6367 Steve Sebastian, (202) 512- 9521 Acknowledgments In addition to those named above, other key contributors were Bernard Anderson, Nancy DeFrancesco, Timothy Hopkins, Larry Korb, Sabine Paul, Jay Pelkofer, Karlin Richardson, Sherrie Russ, Tuyet Quan Thai, Aaron Thorne, and Teresa Tucker. (310205) Lett er GAO United States General Accounting Office Page 1 GAO- 01- 227 IRS' Third Expenditure Plan Contents Letter 3 Appendixes Appendix I: Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff 8 Appendix II: Comments From the Internal Revenue Service 54 Appendix III: GAO Contacts and Staff Acknowledgments 56 Page 2 GAO- 01- 227 IRS' Third Expenditure Plan Page 3 GAO- 01- 227 IRS' Third Expenditure Plan United States General Accounting Office Washington, D. C. 20548 Page 3 GAO- 01- 227 IRS' Third Expenditure Plan Page 4 GAO- 01- 227 IRS' Third Expenditure Plan Page 5 GAO- 01- 227 IRS' Third Expenditure Plan Page 6 GAO- 01- 227 IRS' Third Expenditure Plan Page 7 GAO- 01- 227 IRS' Third Expenditure Plan Page 8 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 9 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 10 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 11 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 12 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 13 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 14 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 15 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 16 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 17 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 18 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 19 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 20 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 21 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 22 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 23 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 24 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 25 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 26 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 27 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 28 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 29 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 30 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 31 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 32 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 33 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 34 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 35 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 36 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 37 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 38 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 39 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 40 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 41 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 42 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 43 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 44 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 45 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 46 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 47 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 48 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 49 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 50 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 51 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 52 GAO- 01- 227 IRS' Third Expenditure Plan Appendix I Briefing Slides From November 9, 2000, Briefing of Senate and House Appropriations Subcommittee Staff Page 53 GAO- 01- 227 IRS' Third Expenditure Plan Page 54 GAO- 01- 227 IRS' Third Expenditure Plan Appendix II Appendix II Comments From the Internal Revenue Service Page 55 GAO- 01- 227 IRS' Third Expenditure Plan Page 56 GAO- 01- 227 IRS' Third Expenditure Plan Appendix III Ordering Information The first copy of each GAO report is free. Additional copies of reports are $2 each. A check or money order should be made out to the Superintendent of Documents. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U. S. General Accounting Office P. O. Box 37050 Washington, DC 20013 Orders by visiting: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U. S. General Accounting Office Washington, DC Orders by phone: (202) 512- 6000 fax: (202) 512- 6061 TDD (202) 512- 2537 Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512- 6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. Orders by Internet: For information on how to access GAO reports on the Internet, send an e- mail message with “info” in the body to: info@ www. gao. gov or visit GAO's World Wide Web home page at: http:// www. gao. gov To Report Fraud, Waste, or Abuse in Federal Programs Contact one: ? Web site: http:// www. gao. gov/ fraudnet/ fraudnet. htm ? e- mail: fraudnet@ gao. gov ? 1- 800- 424- 5454 (automated answering system) United States General Accounting Office Washington, D. C. 20548- 0001 Official Business Penalty for Private Use $300 Address Correction Requested Presorted Standard Postage & Fees Paid GAO Permit No. GI00 *** End of document. ***