Information Technology: Agencies Need to Improve the Accuracy and
Reliability of Investment Information (12-JAN-06, GAO-06-250).
Each year, agencies submit to the Office of Management and Budget
(OMB) a Capital Asset Plan and Business Case--the exhibit 300--to
justify each request for a major information technology (IT)
investment. The exhibit's content should reflect controls that
agencies have established to ensure good project management, as
well as showing that they have defined cost, schedule, and
performance goals. It is thus a tool to help OMB and agencies
identify and correct poorly planned or performing investments. In
its budget and oversight role, OMB relies on the accuracy and
completeness of this information. GAO was asked to determine the
extent to which selected agencies have underlying support for the
information in their fiscal year 2006 exhibit 300s. From five
major departments having over $1 billion in IT expenditures in
that year, GAO chose for analysis 29 exhibits for projects that
supported a cross section of federal activities.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-250
ACCNO: A44677
TITLE: Information Technology: Agencies Need to Improve the
Accuracy and Reliability of Investment Information
DATE: 01/12/2006
SUBJECT: Data integrity
Documentation
Information technology
Internal controls
Investment planning
IT investment management
Performance management
Performance measures
Quality control
Regulatory agencies
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-250
* Report to Congressional Requesters
* January 2006
* INFORMATION TECHNOLOGY
* Agencies Need to Improve the Accuracy and Reliability of
Investment Information
* Contents
* Results in Brief
* Background
* Improvements in IT Management Are Goals of Laws and Guidance
* Exhibit 300 Supports OMB and Agency Oversight of IT
Management
* Exhibit 300s Were Generally Not Based on Adequate Support
* Underlying Documentation Was Lacking or Did Not Support the
Exhibit 300
* Agencies Did Not Always Comply or Provide Evidence of
Compliance with Federal Requirements and Policies
* Cost Data Supporting Business Cases Were Unreliable
* Conclusions
* Recommendations for Executive Action
* Agency Comments and Our Evaluation
* Objective, Scope, and Methodology
* Comments from the Office of Management and Budget
* Descriptions of Investments Reviewed
* Department of Agriculture
* System: Comprehensive Electronic Permit System (ePermits)
* Investment stage: New
* Business Reference Model category: Services for
Citizens
* System: Corporate Financial Management Systems (CFMS)
* System: Integrated Acquisition System (IAS)
* System: Phytosanitary Certificate Issuance and Tracking
System (PCIT)
* System: Processed Commodity Inventory Management System
(PCIMS)
* System: Store Tracking and Redemption System Redesign (STARS
II)
* Department of Commerce
* System: Advanced Weather Interactive Processing System
(AWIPS)
* System: Comprehensive Large Array-Data Stewardship System
(CLASS)
* System: Economic Census and Surveys (ECON)
* System: Improve the Automated Export System (AES)
* System: National Weather Service Telecommunication Gateway
(NWSTG) System
* System: Satellite Operations Control Center Command and Data
Acquisition (SOCC/CDA)
* Department of Energy
* System: Energy Sciences Network (ESnet)
* System: E-content Management System (eCMS)
* System: Integrated Planning, Accountability, and Budgeting
System Information System (IPABS- IS)
* System: Licensing Support Network (LSN)
* System: Los Alamos National Laboratory Enterprise Project
(LANL ERP)
* Department of Transportation
* System: Asset Supply Chain Management (ASCM)
* System: DOT Financial System Consolidation
* System: National Transit Database (NTD)
* System: Next Generation Air/Ground Communications (NEXCOM)
* System: System Approach for Safety Oversight (SASO)
* System: Wide-Area Augmentation System (WAAS)
* Department of the Treasury
* System: Customer Account Data Engine (CADE)
* System: Debt Management Accounting System (DMAS)
* System: Electronic Management System (EMS)
* System: Governmentwide Accounting and Reporting
Modernization (GWA)
* System: Service Center Recognition/Images Processing System
(SCRIPS)
* System: Secure Payment System (SPS)
* GAO Contact and Staff Acknowledgments
Report to Congressional Requesters
January 2006
INFORMATION TECHNOLOGY
Agencies Need to Improve the Accuracy and Reliability of Investment
Information
Contents
Tables
Figure
January 12, 2006Letter
The Honorable Tom Davis Chairman Committee on Government Reform House of
Representatives
The Honorable Adam H. Putnam House of Representatives
Each year, the Office of Management and Budget (OMB) plays a central role
in determining how much the government plans to spend for information
technology (IT) and how these funds are allocated. The IT budget is not
insignificant: federal agencies requested over $65 billion in fiscal year
2006. While these investments are critical to achieving the goals of the
federal government, for the past 3 years OMB has highlighted in the
President's Budget tens of billions of dollars of IT investments that are
at risk. In the most recent budget, for fiscal year 2006, nearly 25
percent of the funds requested, totaling about $15 billion, were
considered by OMB to be at risk.
A key component of OMB's management and oversight of the IT budget process
is the exhibit 300, also known as the Capital Asset Plan and Business
Case, which is developed by agencies and reviewed by both agencies and
OMB. OMB sets forth requirements for the exhibit 300 in its circular
A-11.1 According to this guidance, agencies are required to perform
analyses and provide documentation to support decisions on proposed major
IT investments. The exhibit 300 is the means to accomplish this task: it
is a reporting mechanism intended to enable an agency to demonstrate to
its own management, as well as OMB, that it has employed the disciplines
of good project management; developed a strong business case for the
investment; and met other Administration priorities in defining the cost,
schedule, and performance goals proposed for the investment. The exhibit
300 comprises eight key sections, which cover spending, performance goals
and measures, analysis of alternatives,2 risk inventory and assessment,
acquisition strategy, project (investment) and funding plan, enterprise
architecture, and security and privacy. When considering IT investments to
recommend for funding, OMB relies on the accuracy and completeness of the
information reported in the exhibit 300s.
This report responds to your request that we determine the extent to which
selected agencies have underlying support for the information in their
fiscal year 2006 exhibit 300s. To accomplish this objective, we reviewed
exhibit 300s from the fiscal year 2006 budget submission, as well as
supporting documentation, for 29 major IT investments at five
departments.3 We compared information in each exhibit 300 with the
supporting documentation on the corresponding investment. Further details
on our objective, scope, and methodology are provided in appendix I. Our
work was conducted between March and November 2005 in accordance with
generally accepted government auditing standards.
Results in Brief
Underlying support for the information provided in the exhibit 300 was
often inadequate. Examination of the exhibit 300s and the supporting
materials revealed three types of weaknesses.
o All exhibit 300s had documentation weaknesses. Either documentation did
not exist for specific areas of the exhibit 300, or it did not fully agree
with the exhibit 300. For 23 of the 29 investments, for example,
information in the performance goals and measures section was not
supported by explanations of how agencies had initially measured their
baseline levels of performance (from which they determine progress) or how
they determined the actual progress that was reported in the exhibit 300.
In the case of risk assessment, supporting documentation for about 75
percent of the investments did not address OMB's required risk categories.
Additionally, the analysis of alternatives for most investments either
lacked supporting documentation to justify the calculations of financial
benefits in the exhibit 300, or the documentation did not agree with what
was reported.
o Agencies did not always demonstrate that they complied with federal or
departmental requirements or policies with regard to management and
reporting processes. For example, 21 investments were required to use
an earned value management (EVM) system4 as the basis for the cost,
schedule, and performance information provided in the exhibit 300, but
only 6 investments used an EVM process that followed OMB-required
standards. In addition, none of the investments under review had cost
analyses that fully complied with OMB requirements for completing
cost-benefit and cost-effectiveness analyses. An exception was the
information security section, for which most investments had security
plans and indications that security awareness training had been conducted.
o For those sections that required actual cost data, including the summary
of spending and project and funding plan sections, the data were
unreliable because they were not derived from cost-accounting systems with
adequate controls. In the absence of adequate cost-accounting systems,
agencies generally derived cost information from ad hoc processes used by
project managers.
Agency officials attributed the shortcomings in support to lack of
understanding of a requirement or how to respond to it. Agency officials
mentioned in particular insufficient guidance or training, as well as lack
of familiarity with particular requirements, such as the EVM process. If
underlying support is inadequate in key areas, OMB and agency executives
are depending on unreliable information to monitor the management of major
IT projects and to make critical decisions on their funding, thus putting
at risk millions of dollars in investments. These weaknesses and their
causes are also consistent with problems in project and investment
management that are pervasive governmentwide, including at such agencies
as the Departments of Defense (DOD), Health and Human Services (HHS), and
Homeland Security (DHS), as documented in reports by GAO and others.
We are making recommendations to OMB aimed at improving guidance and
training in exhibit 300 requirements and at ensuring that limitations on
the reliability of information in exhibit 300s are disclosed and
mitigated.
In written comments, OMB accepted the findings of the draft report, while
expressing concern that, by directing our recommendations to OMB rather
than to the agencies, we were suggesting that OMB rather than the agencies
is responsible for data accuracy and employee training. We do not intend
to make this suggestion; we place significant responsibility on agencies,
as reflected in our recommendation that OMB instruct agencies to determine
the extent to which the information contained in each exhibit 300 is
accurate and reliable, to disclose weaknesses, and to describe their
approach to mitigating these weaknesses. This recommendation clearly
places responsibility on the agencies for assessing the quality of their
budget information and the processes that produced this information. Our
recommendations are directed to OMB because of its responsibility for
providing governmentwide leadership in information resources management,
pursuant to the Clinger-Cohen Act, and because they address findings
relating to OMB-required budget documents.
In addition, OMB expressed concern that the recommendations do not focus
on how well agencies fulfill their underlying information resources
management responsibilities. Our view is that our recommendation on
disclosing and mitigating weaknesses does address these underlying
responsibilities. The report specifically addresses the reliability of
exhibit 300s as support for agency and OMB decision making and clearly
states that the lack of documentation and inadequate support may indicate
underlying management weaknesses. Requiring agencies to disclose and
mitigate associated weaknesses will necessitate that agencies examine and
address their approach to fulfilling information resources management
responsibilities.
Technical comments from the Departments of Energy and Transportation were
incorporated as appropriate. The Departments of Agriculture, Commerce, and
the Treasury provided no comments on a draft of this report.
We address these comments more fully in the Agency Comments and Our
Evaluation section of this report. We have reproduced the written comments
in appendix II.
Background
The President's Budget for Fiscal Year 2006 included 1,087 IT projects,
totaling about $65 billion. The planned expenditures at the civilian
agencies comprised about $35 billion of that total cost. In particular,
the five departments in our review made up about one-third of the civilian
planned expenditures (see fig. 1).
Figure 1: Breakdown by Civilian Agencies of Planned $35 Billion in Fiscal
Year 2006 IT Investments
OMB plays a key role in overseeing these IT investments and how they are
managed, stemming from its predominant mission: to assist the President in
overseeing the preparation of the federal budget and to supervise budget
administration in executive branch agencies. In helping to formulate the
President's spending plans, OMB is responsible for evaluating the
effectiveness of agency programs, policies, and procedures; assessing
competing funding demands among agencies; and setting funding priorities.
To carry out these responsibilities, OMB depends on agencies to collect
and report accurate and complete information; these activities depend in
turn on agencies having effective IT management practices.
Improvements in IT Management Are Goals of Laws and Guidance
To drive improvement in the implementation and management of IT projects,
the Congress enacted the Clinger-Cohen Act in 1996, which expanded the
responsibilities of OMB and the agencies that had been set
under the Paperwork Reduction Act.5 The Clinger-Cohen Act requires that
agencies engage in capital planning and performance- and results-based
management.6 The act also requires OMB to establish processes to analyze,
track, and evaluate the risks and results of major capital investments in
information systems made by executive agencies. OMB is also required to
report to the Congress on the net program performance benefits achieved as
a result of major capital investments in information systems that are made
by executive agencies.7
With regard to OMB's responsibilities in this area, we recently issued a
report8 that provided recommendations to improve OMB's processes for
monitoring high-risk IT investments. Since that report was released, OMB
has issued additional guidance outlining steps that agencies must take for
all high-risk projects to better ensure improved execution and performance
as well as promote more effective oversight.9
In response to the Clinger-Cohen Act and other statutes, OMB developed
policy for planning, budgeting, acquisition, and management of federal
capital assets. This policy is set forth in OMB Circular A-11 (section
300) and in OMB's Capital Programming Guide (supplement to Part 7 of
Circular A-11), which directs agencies to develop, implement, and use a
capital programming process to build their capital asset portfolios. Among
other things, OMB's Capital Programming Guide directs agencies to
o evaluate and select capital asset investments that will support core
mission functions that must be performed by the federal government and
demonstrate projected returns on investment that are clearly equal to or
better than alternative uses of available public resources;
o institute performance measures and management processes that monitor
actual performance and compare to planned results; and
o establish oversight mechanisms that require periodic review of
operational capital assets to determine how mission requirements might
have changed and whether the asset continues to fulfill mission
requirements and deliver intended benefits to the agency and customers.
Among OMB's goals in requiring the use of a capital programming process is
to assist agencies in complying with a number of results-oriented
requirements. Key requirements include those set by
o the Federal Acquisition Streamlining Act of 1994,10 which (1) requires
agencies to establish cost, schedule, and measurable performance goals for
all major acquisition programs and (2) establishes that agencies should
achieve on average 90 percent of those goals;
o the Government Performance and Results Act of 1993,11 which establishes
the foundation for budget decision making to achieve strategic goals in
order to meet agency mission objectives; and
o the Federal Information Security Management Act of 2002,12 which
requires agencies to integrate IT security into their strategic and
operational planning processes, such as the capital planning and
enterprise architecture processes at the agency.
OMB is aided in its responsibilities by the Chief Information Officers
(CIO) Council as described by the E-Government Act of 2002.13 The council
is designated the principal interagency forum for improving agency
practices related to the design, acquisition, development, modernization,
use, operation, sharing, and performance of federal government information
resources. Among the specific functions of the CIO Council are the
development of recommendations for the Director of OMB on government
information resources management policies and requirements and the sharing
of experiences, ideas, best practices, and innovative approaches related
to information resources management. The CIO Council has issued several
guides on capital planning and investment management over the past several
years.14
To further support the implementation of IT capital planning practices, we
have developed an IT investment management (ITIM) framework15 that
agencies can use in developing a stable and effective capital planning
process, as required by statute and directed in OMB's Capital Programming
Guide. Consistent with the statutory focus on selecting,16
controlling,1817 and evaluating investments, this framework focuses on
these processes in relation to IT investments specifically. It is a tool
that can be used to determine both the status of an agency's current IT
investment management capabilities and the additional steps that are
needed to establish more effective processes. Mature and effective
management of IT investments can vastly improve government performance and
accountability. Without good management, such investments can result in
wasteful spending and lost opportunities for improving delivery of
services to the public.
The ITIM framework lays out a coherent collection of key practices that,
when implemented in a coordinated manner, can lead an agency through a
robust set of analyses and decision points that support effective IT
investment management. The framework explicitly calls for consideration of
cost, schedule, benefit, and risk objectives, including the development of
analyses such as return on investment and a risk management plan. The
framework also describes the criticality of tracking progress using valid
and complete data. The guidance laid out in the ITIM framework is
consistent with the requirements of OMB's Circular A-11 and matches it in
many instances. For example, among the requirements on the exhibit 300 is
that agencies indicate that the investment has been reviewed and approved
by the responsible oversight entity. The agency investment review board is
a critical element of the ITIM framework, and the expectation for the
board to select and oversee IT investments is explicit.
In previous work using our IT investment management framework, we reported
that the use of IT investment management practices by agencies was mixed.
For example, a few agencies that have followed the ITIM framework in
implementing capital planning processes have made significant
improvements.19 In contrast, however, we and others have continued to
identify weaknesses at agencies in many areas, including immature
management processes to support both the selection and oversight of major
IT investments and the measurement of actual versus expected performance
in meeting established IT performance measures.20 For example:
o We recently reported that the HHS senior investment board does not
regularly review component agencies' IT investments, leaving close to 90
percent of its discretionary investments without an appropriate level of
executive oversight.21 To remedy this weakness, we recommended that the
department (1) establish a process for the investment board to regularly
review and track the performance of a defined set of component agency IT
systems against expectations and (2) take corrective actions when these
expectations are not being met.
o At DHS, we determined that the department's draft information resources
management strategic plan did not include fully defined goals and
performance measures.22 To address this weakness, we recommended that the
department establish IT goals and performance measures that, at a minimum,
address how information and technology management contributes to program
productivity, the efficiency and effectiveness of agency operations, and
service to the public.
o A recent review by the DOD Inspector General23 determined that over 90
percent of the business cases submitted to OMB in support of the DOD
fiscal year 2006 budget request did not completely respond to one or more
data elements addressing security funding, certification and
accreditation, training and security plans, and enterprise architecture.
The DOD Inspector General concluded that, as a result, these submissions
continued to have limited value and did not demonstrate that the
department was effectively managing its proposed IT investments for fiscal
year 2006.
Exhibit 300 Supports OMB and Agency Oversight of IT Management
Besides providing policy for planning, budgeting, acquisition, and
management of federal capital assets, section 300 of OMB's Circular A-11
instructs agencies on budget justification and reporting requirements for
major IT investments.24 Section 300 defines the budget exhibit 300, also
called the Capital Asset Plan and Business Case, as a document that
agencies submit to OMB to justify resource requests for major IT
investments. According to OMB, only priority capital asset investments
that comply with the policies for good capital programming, as described
in the Capital Programming Guide, will be recommended for funding in the
President's Budget.
The exhibit 300 was established as a source of information on which to
base both quantitative decisions about budgetary resources consistent with
the Administration's program priorities and qualitative assessments about
whether the agency's planning, acquisition, management, and use of capital
assets (investments) are consistent with OMB policy and guidance. The
types of information included in the exhibit 300 are intended, among other
things, to help OMB and the agencies identify and correct poorly planned
or performing investments (i.e., investments that are behind schedule,
over budget, or not delivering expected results) and real or potential
systemic weaknesses in federal information resource management (such as a
shortage of sufficiently qualified project managers).
According to Circular A-11, the information in the exhibit 300 allows the
agency and OMB to review and evaluate each agency's IT spending and to
compare IT spending across the federal government. Further, the
information helps the agency and OMB to provide a full and accurate
accounting of IT investments for the agency, as required by the Paperwork
Reduction Act and the Clinger-Cohen Act.
The exhibit 300 is required for all assets, though certain sections apply
only to information technology. Table 1 provides a description of the key
sections of the exhibit 300, as well as examples of the types of
documentation that provide support for the data summarized in the exhibit
300 (although the supporting documentation may take other forms). This
support may be derived from a variety of sources, including financial
management systems and management processes that agencies carry out to
comply with federal requirements and guidelines (such as the Federal
Information Security Management Act of 2002 and the Federal Enterprise
Architecture),25 as well as from analyses carried out specifically in
support of the management of the investment.
Table 1: Description of Key Sections in the Exhibit 300 and General
Documentation Typically Used as Support
Section name Description Examples of supporting
documentation
Summary of Provides an overview of the Financial reports
Spending table costs for planning,
acquisition, maintenance,
and full-time employees for
the previous, current, and
budget fiscal years; it
also includes a summary of
these costs for earlier
years and estimated costs
for future years.
Performance Describes the link between Annual performance plan
Measures and Goals the agency's annual goals and/or annual performance
and mission and how the budget; IT strategic plan
investment will meet those
goals. This section
illustrates the performance
measures and results of the
investments.
Analysis of Provides a summary of the Cost-benefit analysis or
Alternatives comparison of viable cost-effectiveness analysis
alternative solutions that
includes a general
rationale and analysis of
the monetized benefits for
each alternative presented.
Risk Inventory and Provides a summary of the Risk management plan, risk
Assessment investment's risk reports
assessment, showing the
active management of 19
risk elements that OMB
requires to be considered.
Acquisition Provides a description of Acquisition plan
Strategy the acquisition strategy
used and mitigation efforts
to ensure minimal risk to
the government.
Project Provides a summary of the Cost performance reports,
(Investment) and investment's status in integrated baseline review,
Funding Plan accomplishing baseline cost time-phased performance
and schedule goals through measurement baseline, work
the use of an earned value breakdown structure, and
management (EVM) system or operational analysis
operational analysis,
depending on the life-cycle
stage.
Enterprise Demonstrates that the Investment-specific
Architecture investment is included in artifacts, including as-is
the agency's enterprise and to-be architectures,
architecture and capital migration plan, documented
planning investment control approval by an enterprise
process. Illustrates the architecture review
agency's capability to committee
align the investment to the
Federal Enterprise
Architecture.
Security and Provides a description of Certification and
Privacy an investment's security accreditation packages,
and privacy issues. It security plans, security
summarizes the agency's training logs, and
ability to manage security system-level incident
at the system or handling procedures
application level.
Additionally, it
demonstrates compliance
with the certification and
accreditation process, as
well as the mitigation of
IT security weaknesses.
Source: GAO analysis based on OMB data.
According to OMB guidance, the life-cycle stage of the asset affects what
is reported on the exhibit 300:
o New investments (i.e., proposed for budget year or later, or in
development) must be justified based on the need to fill a gap in the
agency's ability to meet strategic goals and objectives with the lowest
life-cycle costs of all possible alternatives and provide risk-adjusted
cost and schedule goals and measurable performance benefits.
o Mixed life-cycle investments (i.e., investments that are operational but
include some developmental effort, such as a technology refresh) must
demonstrate satisfactory progress toward achieving baseline cost,
schedule, and performance goals using an EVM system.26
o Operational investments (i.e., steady state) must demonstrate, among
other things, how close actual annual operating and maintenance costs are
to the original life-cycle cost estimates; whether the technical merits of
the investment continue to meet the needs of the agency and customers; and
that an analysis of alternatives was performed with a future focus.
OMB requires agencies to transmit exhibit 300s electronically, using a
predefined format. To meet this requirement and facilitate the aggregation
of elements of the exhibits from various sources throughout the
organization, many agencies use software applications to compile their
exhibits 300s. Besides aggregating portions of the exhibit 300, these
tools are designed to also perform certain calculations, such as return
on investment and those required for earned value analysis.
Exhibit 300s Were Generally Not Based on Adequate Support
Although the agencies reported that all 29 exhibit 300s had been approved
by their investment review boards (as required), in many instances,
support for the information provided was not adequate. (Details on the 29
investment projects described in the exhibit 300s that we reviewed are
provided in app. III.) Three types of problems were evident.
o First, all exhibit 300s had documentation weaknesses. For example, each
investment lacked documentary support for one or more of the following:
Analysis of Alternatives, Risk Inventory and Assessment, and Performance
Measures and Goals. In other cases, the supporting material that was
provided to us did not match the information in the exhibit 300.
o Second, agencies did not always demonstrate (for example, in the
Security and Privacy and the Project and Funding Plan sections) that they
complied with federal requirements or policies with regard to management
and reporting processes.
o Finally, information in some sections (such as the Summary of Spending
table and the Project and Funding Plan) could not be relied upon because
the numbers were not derived using repeatable processes or reliable
systems.
Agency officials attributed the absence of adequate support for their
exhibit 300s to lack of understanding of the requirements or of how to
respond to them. Agency officials mentioned in particular insufficient
guidance or training, as well as lack of familiarity with particular
requirements, such as the EVM process. If underlying support is inadequate
in key areas, OMB and agency executives are depending on unreliable
information to monitor the management of major IT projects and to make
critical decisions on their funding, thus putting at risk millions of
dollars in investments.
Underlying Documentation Was Lacking or Did Not Support the Exhibit 300
OMB Circular A-11 states that agencies must justify funding requests for
major acquisitions by demonstrating, among other things, measurable
performance benefits, comprehensive risk mitigation and management
planning, and positive return on investment for the planned investment.
Agencies are instructed to establish performance metrics (including
baselines from which progress can be measured) to ensure that project
managers are accountable in meeting expected performance goals and that
projects are aligned with the agencies' strategic goals. Agencies are also
expected to manage investment risk through a robust risk management
program; according to OMB's guidance, agencies need to actively manage
risks from initial concept throughout the life cycle of each investment.
To demonstrate a positive return on investment for the selected
alternative and identify a project's total lifetime cost, OMB requires
agencies to compare alternatives and report summary cost information for
investments (including calculations for payback period and net present
value). Documents produced in the performance of these activities provide
evidence that they were carried out as required.
o Performance measures. The investments did not usually demonstrate the
basis for the performance measure information provided in the exhibit 300.
Only 6 of the 29 investments had documentation to support how agencies
initially measured their baseline levels of performance, from which they
measured progress toward the agency's strategic goals. In most cases, the
investments lacked documentation describing the levels of performance that
had been achieved or how these results actually helped meet agency
strategic needs. The absence of documentation in these cases could
indicate a systemic weakness in agency performance management practices,
since well-developed practices should provide the expected support. This
finding is consistent with our prior work where we determined that
agencies were generally not measuring actual versus expected performance
in meeting IT performance goals.27 Weak performance management practices
reduce the ability of agency executives to track investment performance in
meeting performance objectives and raise the risk that investments will
not be well aligned with agency strategic objectives.
o Risk management. About 75 percent of the investments were unable to
demonstrate that they were actively addressing the risk elements that OMB
specifies in Circular A-11, or how they had determined that any of those
risks were not applicable. In addition, documentation of risk management
that was provided had significant weaknesses. In one case, a risk
management plan was approximately 9 years old and had not been updated,
and for three investments, the risk documentation addressed only the
project development phase, even though the systems had exited that phase
and were in full operation.
o Analysis of alternatives. All 29 investments reported cost information
in the analysis of alternatives section of the exhibit 300. However, in
about 72 percent of the exhibit 300s reviewed, either supporting
documentation was missing for this cost information, or information in the
documentation did not agree with that in the exhibit 300.
In cases where investments lacked documentation to support information
reported in the performance and risk areas, project officials frequently
told us that they had filled out these sections of the exhibit 300 to
satisfy the reporting requirement, relying on their own knowledge of the
investment rather than any project documentation. However, such an
approach is not consistent with the requirement for providing accurate
information in compliance with OMB capital programming and capital
planning and investment control policies. In addition, several project
officials told us that they believed some of the 19 risk management areas
required in the exhibit 300 were not applicable to their investment, but
they reported on those categories nonetheless to fulfill the requirement.
Although the guidance instructs agencies to indicate whether the risk
category was not applicable, officials stated that their impression is
that "not applicable" responses might lower the evaluation of their
investments and reduce or eliminate their funding.
Further, agency officials generally responded that the training they
received for preparing the exhibit 300 was not sufficient. For example,
one agency commented that agencies would benefit from targeted OMB
training that would address agency-specific questions. Several agencies
stressed that OMB training should occur earlier in the budget cycle. In
addition, one agency said that it needed OMB training on preparing each
section of the exhibit 300.
Overall, the lack of documentation supporting the exhibit 300s raises
questions regarding the sufficiency of the business case for the
investment and the quality of the projects' management.
Agencies Did Not Always Comply or Provide Evidence of Compliance with
Federal Requirements and Policies
Compliance with OMB and other federal guidance and related federal laws
helps ensure that agency investments are managed in a manner consistent
with the intent of the Congress and that key information is available to
OMB and agency managers on which they can base informed decisions.
o The security section of the exhibit 300 requires that agencies
demonstrate that they have developed information security plans in
accordance with the Federal Information Security Management Act of 2002
(FISMA); according to FISMA, these plans must include rules of behavior
for system use, technical security controls, and procedures for incident
handling-that is, how to respond to system security breaches. In addition,
agencies ensure that employees and contractors receive
security awareness training.28 Guidance from the National Institute of
Standards and Technology (NIST) supports FISMA by outlining the necessary
components of key security documentation, including security plans,
certification and accreditation packages, and security controls testing.
o For the analysis of alternatives section, OMB's instructions for the
exhibit 300 cite the Clinger-Cohen Act, which requires agencies to
complete a cost-benefit analysis for new IT investments, and OMB Circular
A-94, which outlines requirements for completing cost-benefit and
cost-effectiveness analyses, including the comparison of at least three
alternatives, a discussion of assumptions for each alternative, and an
analysis of uncertainty (a sensitivity assessment to raise awareness of
the potential for unforeseen impacts on the investment).
o For the project and funding plan section, OMB Circular A-11 provides
guidance that requires an agency to have in place a process for monitoring
the investment's status in accomplishing baseline cost and schedule goals.
For the 29 investments, agency compliance with the FISMA and NIST
requirements described above was mixed. For example, about 86 percent of
all investments could demonstrate, based on documentation, that security
awareness training had been conducted for employees and contractors and
that a mechanism for tracking completion of security awareness training
had been established. In addition, 21 of the 22 operational investments
(for which information security plans are required) had security plans
that addressed areas such as the rules of behavior for system use and
technical security controls. In contrast, about 77 percent of these 22
investments did not provide support describing how incident handling
activities would be performed at a system level, such as detecting,
reporting, and mitigating risks associated with security incidents.
While the compliance of security documentation with federal requirements
was mixed, the documented support for the analysis of alternatives and the
project and funding plan areas of the exhibit 300 provided little
assurance that investments complied with applicable guidance and laws.
None of the investments had cost analysis documentation that fully
complied with Circulars A-94 and A-11 criteria (lacking, for example, a
comparison of at least three alternatives, a discussion of assumptions for
each alternative, or an analysis of uncertainty).
Project officials attributed deficiencies in the analysis of alternatives
to, among other things, a lack of understanding of what was expected for
reporting in the exhibit 300. In a few instances, officials noted that
they believed that their investments were excluded from meeting the
federal requirements because the investments were near the end of their
operational or, in some cases, useful life cycles. OMB guidance on
analysis of alternatives does not differentiate between operational and
developmental investments; nonetheless, one agency's internal guidance
explicitly states that no analysis of alternatives is necessary for
investments in the steady state (that is, operational). However, a
forward-looking analysis of alternatives for operational investments can
help agencies recognize when an alternative solution may be more efficient
or effective than the current investment, thereby freeing scarce resources
to be reallocated.
The agencies' lack of compliance with OMB guidelines for analysis of
alternatives, including the cost-benefit analysis, leaves senior executive
managers at risk of making poor investment management decisions on
incomplete and sometimes inaccurate information.
For the project and funding plan section of the exhibit 300, OMB Circular
A-11 provides guidance on the information to be provided, which depends
upon the state of the investment (i.e., new, mixed life cycle, or steady
state). According to this guidance, information presented in the project
and funding plan is to be derived from one of two types of analysis: for
steady state investments, an operational analysis, and for new and mixed
life-cycle investments, an analysis based on an EVM process that is
compliant with ANSI/EIA-748-A.29 Operational analysis is a method for
assessing the technical merits of an existing investment in meeting user
needs, while EVM is a method for assessing the value of work performed
compared to its actual cost during development of an investment.
Of the eight steady state investments we reviewed, only two had conducted
an operational analysis. Furthermore, only one of those had documented
procedures that were in accordance with OMB's Capital Programming Guide
criteria, such as addressing user needs and technical performance. In most
cases for which no operational analyses were in place, agency officials
commented that OMB guidance describing how to perform an operational
analysis was at such a high level of generality that they found it
difficult to follow. Instead of attempting to devise and perform an
operational analysis, therefore, they implemented variations on an EVM
process. However, these implementations of EVM did not address topics
required for the operational analysis, such as user needs and technical
performance. Unless they address these topics, agencies may not have the
information they need to determine, among other things, whether
investments are performing as intended and meeting user needs.
Similarly, of the 21 new and mixed life-cycle investments required to use
EVM, only 6 used an EVM process that generally followed the ANSI
standard.30 Since fiscal year 2002, OMB has required the use of EVM as a
project management tool. The ANSI standard is intended to ensure that data
produced by an EVM process are reliable so as to allow objective reports
of project status, produce early warning signs of impending schedule
delays and cost overruns, and ultimately provide unbiased estimates of
anticipated costs at completion. If agencies do not implement EVM
processes that follow the ANSI standard, they have reduced assurance that
the information used for tracking the cost, schedule, and performance of
the investment is reliable.
For the remaining 15 investments that did not have EVM processes following
the required standard, project officials commented that EVM was relatively
new to them and that they did not understand how to implement an
ANSI-compliant process at the time of the fiscal year 2006 submission. At
the time of our review, all five departments stated that they were working
toward implementing compliant processes.
To OMB's credit, it recognized the need for improvement in the execution
of agencies' IT projects and has issued clarifying guidance on the
implementation of EVM.31 This guidance, issued in August 2005, could be
expected to have an impact on the exhibit 300s prepared for fiscal year
2008. Under this guidance, agencies are instructed, among other things, to
o develop comprehensive agency policies for using EVM to plan and manage
development activities for major IT investments no later than December 31,
2005;
o include a provision and clause in major acquisition contracts or agency
in-house project charters directing the use of an EVM system compliant
with the required standard; and
o provide documentation demonstrating that the contractor's or agency's
in-house EVM system complies with the required standard and conduct
periodic surveillance reviews.
Additionally, the Civilian Agency Acquisition Council and the Defense
Acquisition Regulations Council published in the Federal Register a
proposed amendment32 to the Federal Acquisition Regulation (FAR Case
2004-019) to standardize EVM contract policy across the government.
In previous work,33 we have reported that EVM can have a significant
impact on the success of an IT acquisition because it heightens visibility
into whether a program is on target with respect to cost, schedule, and
technical performance. Therefore, it is important that the process is
implemented properly to maximize its value as a project management tool.
If it is not implemented effectively, agency executives and OMB risk
making poor investment decisions based on inaccurate and potentially
misleading EVM information.
Cost Data Supporting Business Cases Were Unreliable
Accurate and timely cost management information is critical for federal
managers to understand the progress of major projects and vital in
developing meaningful links among budget, accounting, and performance. The
Federal Financial Management Improvement Act of 1996 emphasizes the need
for agencies to have systems that are able to generate reliable, useful,
and timely information for decision-making purposes and to ensure
accountability on an ongoing basis.on the lack of adherence to federal
accounting standards throughout the federal government and have made
recommendations that agencies improve cost-accounting systems. 34 In
previous work, we have reported 35
At every agency, cost information reported in the 29 exhibit 300s was
derived from ad hoc processes rather than from cost-accounting systems
with adequate controls to ensure accountability. This condition had impact
in two particular areas of the exhibit 300-the summary of spending table
and the project and funding plan section:
o Figures for dollars expended for the prior year (in this case, fiscal
year 2004) were not reliable. In all cases, documentation provided to
support prior year cost figures in the summary of spending table showed
that the information was derived from ad hoc sources, such as spreadsheet
estimates, handwritten figures, or e-mails. Therefore, the cost data
reported in the exhibit 300 are not verifiable.
o Information in the project and funding plans was also unreliable for the
21 new and mixed life-cycle investments required to use EVM. As discussed
earlier, 15 of these investments reported cost figures based on EVM
processes that did not follow the ANSI standard; because the standard was
not followed, these processes did not have the controls necessary to
ensure that the data they produced were reliable. The other 6 investments
had ANSI-compliant EVM processes in place for the contractor component of
the investment costs, but the government component of the investment costs
was derived from ad hoc systems (such as tracking government costs in
spreadsheets based on project managers' own records); thus, that portion
of the data was not reliable, lending a degree of unreliability to the
overall EVM reports.
The lack of accurate cost figures limits decision makers' ability to
determine the actual resources expended on an investment, and therefore
inhibits their ability to make fully informed decisions on whether to
proceed. Without reliable systems that meet federal standards, government
agencies cannot produce reliable cost-based performance information.
Conclusions
The usefulness of the exhibit 300 business case as a mechanism to support
the selection and oversight of federal IT investments is undercut by the
kinds of weaknesses displayed in the 29 business cases that we reviewed.
Although we cannot directly project these examples to the more than one
thousand business cases developed each year across the federal government,
our results suggest that the issues raised need attention. The
shortcomings in guidance and training are likely to be widespread, and so
the weaknesses may extend beyond the specific examples identified here.
The kinds of weaknesses displayed and the causes behind them are
consistent with the pervasive problems with project and investment
management that we have documented in numerous prior reports.
The absence of documentary support in the cases reviewed raises questions
regarding the sufficiency of the justification provided for the investment
and undermines the management decisions being made based on it. More
troubling, it may indicate an underlying weakness in the management of the
investment, particularly since several sections of the exhibit are
specifically designed to capture information from systems used in project
management, such as those that support EVM and financial management. In
many cases, inadequate support raises questions regarding the adequacy of
an agency's management processes and internal controls, which strongly
affect the reliability of the information presented to decision makers.
Further, in view of the inaccuracies in the cases reviewed, it is evident
that agencies are not taking sufficient actions to ensure the accuracy of
the information in the exhibit 300s. To make reasonable decisions,
management needs to be aware of limitations in the data on which they rely
and thus be able to take steps to mitigate the risks involved.
Collectively, our findings raise questions on whether fundamental project
management processes are in place, whether project managers are adequately
trained in these processes, and whether they receive sufficient guidance
on these processes and on preparing all areas of the exhibit 300. At a
minimum, this situation undermines the usefulness of the exhibit 300 as a
mechanism to communicate to OMB and agency executives that the project
team has employed the disciplines of good project management. By reporting
information that is not supported by documentation, the exhibit 300 can
create the misleading appearance that investments are being managed
properly, when in fact they are not. In addition, OMB has relied on these
exhibits to identify and oversee high-risk projects; thus, our finding
that the data being presented to OMB may not be reliable or accurate
further complicates its oversight.
While OMB is applying more rigor to its oversight processes through such
processes as its tracking of high-risk investments, these advances may be
undermined by inaccurate or unreliable data used in decision making.
Unless these weaknesses are addressed, OMB, agency executives, and
Congress will not have assurance that key decisions to pursue and oversee
the $65 billion in IT investments are being made based on accurate and
reliable information.
Recommendations for Executive Action
To improve the accuracy and validity of exhibit 300s for major IT
investments and to increase the value of using the information they
provide in selection, oversight, and resource allocation decisions, we are
making three recommendations.
1.Because decision makers should be aware of any weaknesses in the
processes used to develop the information in the exhibit 300s, we are
recommending that the Director of OMB direct agencies to determine the
extent to which the information contained in each exhibit 300 is accurate
and reliable. Where weaknesses in accuracy and reliability are identified,
the agency should be required to disclose them and explain the agency's
approach to mitigating them.
In addition, to help ensure that agency personnel completing exhibit 300s
better understand their responsibilities, we recommend that the Director
of OMB take the following additional actions:
2.In advance of OMB's next issuance of the Circular A-11 update, develop
and promulgate clearer and more explicit guidance for sections of the
exhibit 300 business case that cause confusion, including addressing
weaknesses identified in this report (as indicated below) and consulting
with agency personnel having responsibility for completing exhibit 300s
across government to identify other areas of confusion. At a minimum, the
guidance should do the following:
o Provide a more detailed description of the requirements for completing
an operational analysis, as referred to in the supplement to Part 7 of
Circular A-11, the Capital Programming Guide.
o Address or clarify possible flexibilities and alternative approaches
available to agencies in completing their exhibit 300s: for example,
whether the analysis of alternatives section of the exhibit 300 needs to
be updated every year for steady state investments and whether all risk
areas are relevant for all investments.
3.Provide for training of agency personnel responsible for completing
exhibit 300s. This training should go beyond a description of changes from
prior years' guidance and include working through examples for a variety
of investments. In developing the training, OMB should consult with
agencies to identify deficiencies that the training should address.
In implementing these recommendations, OMB should work with the CIO
Council to develop the necessary guidance and implement an effective
training program to ensure governmentwide acceptance of these changes.
Because we have outstanding recommendations aimed at enhancing OMB's audit
guidance related to federal cost-accounting systems,36 we are not making
any new recommendations in this report regarding federal cost accounting.
Agency Comments and Our Evaluation
We provided a draft of this report to OMB and the five agencies whose
exhibit 300s we reviewed.
In written comments received on December 23, 2005, the Administrator of
OMB's Office for E-Government and Information Technology accepted the
findings of the draft report. OMB described two of our three
recommendations and expressed three concerns: first, that our report does
not address the need for agencies to ensure the accuracy of their IT
investment requests; second, that the report focuses on the way agency
employees fill out OMB's exhibit 300s and not on the underlying management
responsibilities; and third, that by directing our recommendations to OMB
rather than to the agencies, we could be seen as suggesting that OMB and
not the agencies are responsible for data accuracy and employee training.
OMB's concern regarding data accuracy is addressed by our first
recommendation: that the Director of OMB instruct agencies to determine
the extent to which the information contained in each exhibit 300 is
accurate and reliable, to disclose weaknesses, and to describe the
agency's approach to mitigating these weaknesses. This recommendation
clearly places responsibility on the agencies for assessing the quality of
their budget information and the processes that produced this information.
With respect to OMB's concern that the recommendations do not focus on how
well agencies fulfill their underlying information resources management
responsibilities, our view is that our recommendation on disclosing and
mitigating weaknesses does address these underlying responsibilities. The
report specifically addresses the exhibit 300s and the reliability of
these documents when used as support in the agencies' and OMB's
decision-making processes. As our report clearly states, the lack of
documentation may indicate an underlying weakness in the management of the
investment. In many cases inadequate support raises questions about the
investments' program management and internal controls. Requiring agencies
to disclose and mitigate associated weaknesses presupposes that agencies
examine and address their approach to fulfilling information resources
management responsibilities.
Regarding OMB's third concern, we do not intend to suggest that agencies
are not responsible and accountable for the weaknesses we describe. We
place significant responsibility on agencies to manage their information
assets effectively, as reflected in our first recommendation and in the
large number of evaluations that we have previously conducted at
individual agencies and the recommendations resulting, some of which are
still outstanding. In this report, however, our recommendations are
directed to OMB because they address findings relating to OMB-required
budget documents, and OMB has statutory responsibility for providing
information resources management guidance governmentwide.
Regarding OMB's comment that agencies be held responsible for employee
training in information resources management, we agree that agencies are
responsible for such training. However, as agencies indicated during the
review, additional training by OMB would be helpful, especially in the
understanding of OMB's requirements for the exhibit 300. This is also
consistent with OMB's responsibility under the E-Government Act of 200237
to identify where current training does not satisfy the personnel needs
related to information technology and information resource management.
The Deputy Associate Chief Information Officer for Information Technology
Reform of the Department of Energy provided largely technical comments,
which we incorporated as appropriate. The Director of Audit Relations of
the Department of Transportation also provided technical comments that
were incorporated as appropriate. The Departments of Agriculture,
Commerce, and the Treasury provided no comments.
The written comments from OMB are reproduced in appendix II.
As agreed with your offices, unless you publicly announce the contents of
this report earlier, we plan no further distribution until 30 days from
the report date. At that time, we will send copies of this report to the
Secretaries of the Departments of Agriculture, Commerce, Energy,
Transportation, and the Treasury and the Director of Office of Management
and Budget. We will also make copies available to others upon request. In
addition, this report will be available at no charge on the GAO Web site
at http://www.gao.gov.
If you have any questions on matters discussed in this report, please
contact me at (202) 512-9286. I can also be reached by e-mail at
[email protected] . Contact points for our Offices of Congressional
Relations and Public Affairs may be found on the last page of this report.
Other contacts and key contributors to this report are listed in appendix
IV.
David A. Powner Director, Information Technology Management Issues
Objective, Scope, and Methodology Appendix I
Our objective was to ascertain the extent to which selected agencies have
underlying support for the information described in their fiscal year 2006
exhibit 300s as submitted to the Office of Management and Budget (OMB) in
September 2004. To address our objective, we reviewed the supporting
documentation for 29 exhibit 300s from agencies and components from the
Departments of Agriculture, Commerce, Energy, Transportation, and the
Treasury.1
We selected the five departments for our review on the basis of two
criteria. First, to ensure that we examined significant investments, we
selected departments that expected to spend $1 billion or more on
information technology (IT) investments in fiscal year 2006. Second, of
those agencies with significant investments, we further narrowed our
selection of agencies to those with the first and second largest number of
IT investments in each of three categories of the federal government's
Business Reference Model (BRM): Services for Citizens,2 Support Delivery
of Services,3 and Management of Government Resources.4 We did this to
ensure that the agencies under review reflect the primary business
operations performed by the federal government. We excluded the Mode of
Delivery Business Area because we found investments in this area to be
largely from one agency, the Department of Defense (DOD). (In general,
Mode of Delivery describes the mechanisms the government uses to achieve
its purposes: Services for Citizens.) (We excluded DOD and the Department
of Homeland Security (DHS) from our selection, because the Defense
Inspector General recently performed an extensive review of exhibit 300s,5
and we have both completed and ongoing work on several major IT
investments at DHS).6 This process resulted in the selection of the five
departments mentioned above.
To make our selection of IT investments from the five departments, we used
OMB capital planning and budget documentation to identify a mix of
investments. Specifically, we chose IT investments that (1) supported
government operations across each of the three BRM business areas
identified above and (2) reflected different stages of investment (e.g.,
new, mixed life cycle, and steady state). Initially, we selected a total
of 30 investments (i.e., 6 investments from each department). However, one
IT investment was dropped from our total of 30 selected investments
because we determined during our review that OMB and the agency had
cancelled its funding.
To determine the extent of each investment's underlying support, we
developed a set of questions regarding the types of analysis and
documentation that were associated with the information provided in each
of the major sections of OMB's exhibit 300.7 Using our set of questions,
we met with agency officials for each selected investment to collect and
analyze investment documentation associated with each exhibit 300 area in
our evaluation. We further compared the documentation against the exhibit
300 to ascertain whether the documentation agreed with what the investment
reported in the exhibit 300. Where federal requirements, laws, and other
guidelines were cited in Circular A-11, we also used these to assess the
extent to which agencies and components had complied with specific
documentation requirements as prescribed in these sources (including
National Institute of Standards and Technology (NIST) guidance, OMB
circulars, and OMB memorandums).
In areas where federal directives were cited in the exhibit 300, we
conducted limited reliability testing; these areas included security,
analysis of alternatives, and the project and funding plan. In our
evaluation of security documentation, we used criteria set forth in NIST
guidance to assess whether the major components were present in key
documents, which included the security plan and system-level incident
handling procedures. For security awareness training, we identified
whether training was conducted and tracked but did not assess its content.
In our evaluation of the analysis of alternatives, we used criteria from
OMB Circular A-94 to assess whether the major components were present in
the cost-benefit or cost-effectiveness analysis. In cases where investment
managers told us that their earned value management (EVM) processes were
in conformance with ANSI/EIA-748-A (for our evaluation of the project and
funding plan sections), we used criteria from ANSI/EIA-748-A to assess
whether key EVM processes were in place. We did not test the quality of
the documentation in these areas of evaluation.
Regarding the reliability of cost data, we did not test the adequacy of
agency or contractor cost-accounting systems. Our evaluation was based on
what we were told by the agencies and the information they could provide
(to the extent to which they had information).
We performed our work at the agencies' offices in the Washington, D.C.,
metropolitan area. We conducted our review between March and November 2005
in accordance with generally accepted government auditing standards.
Comments from the Office of Management and Budget Appendix II
Descriptions of Investments Reviewed Appendix III
The following provides additional detail on the agencies and investments
that we reviewed as part of this audit. We reviewed a total of 29
investments at five departments: Agriculture, Commerce, Energy,
Transportation, and the Treasury.1 The selected departments account for
the first and second largest number of IT investments in each of three
categories of the federal government's Business Reference Model: Services
for Citizens,2 Support Delivery of Services,3 and Management of Government
Resources.4
According to OMB guidance, the life-cycle stage of the asset affects what
is reported on the exhibit 300:
o New investments (i.e., proposed for budget year or later, or in
development) must be justified based on the need to fill a gap in the
agency's ability to meet strategic goals and objectives with the lowest
life-cycle costs of all possible alternatives and must provide
risk-adjusted cost and schedule goals and measurable performance benefits.
o Mixed life-cycle investments (i.e., investments that are operational but
include some developmental effort, such as a technology refresh) must
demonstrate satisfactory progress toward achieving baseline cost,
schedule, and performance goals using an EVM system.5
o Operational investments (i.e., steady state) must demonstrate, among
other things, how close actual annual operating and maintenance costs are
to the original life-cycle cost estimates, whether the technical merits of
the investment continue to meet the needs of the agency and customers, and
that an analysis of alternatives was performed with a future focus.
Department of Agriculture
System: Comprehensive Electronic Permit System (ePermits)
Brief description: This system is expected to automate processes to allow
the Department of Agriculture to issue, track, and rapidly verify the
validity of a federal permit allowing the importation of plants and
animals. It is also expected to assist the public by allowing applicants
to apply for permits, check the status of permit applications, and receive
permits online.
Investment stage: New
Business Reference Model category: Services for Citizens
Table 2: Financial Funding Data for ePermits
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $4.8 $4.8 $0.0
FY 05 $3.3 3.3 0.0
FY 06 $2.2 $2.2 $0.0
Source: OMB FY2006 Exhibit 53.
System: Corporate Financial Management Systems (CFMS)
Brief description: This investment is designed to represent the entire
portfolio of current corporate financial management and administrative
payment systems for the department. It is a corporatewide solution for
financial management reform and systems integration that provides tools
for program and financial managers to manage and evaluate federal
programs.
Investment stage: Steady state
Business Reference Model category: Management of Government Resources
Table 3: Financial Funding Data for CFMS
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $64.6 $0.0 $64.6
FY 05 $62.0 0.0 62.0
FY 06 $61.7 $0.0 $61.7
Source: OMB FY2006 Exhibit 53.
System: Integrated Acquisition System (IAS)
Brief description: This system is intended to be a single enterprisewide
acquisition management system to support a strategic and more standardized
acquisition management process for Agriculture. It is expected to provide
a real-time interface to the department's core financial system, reliable
data, and a shortened time for acquiring goods and services.
Investment stage: Mixed life cycle
Business Reference Model category: Management of Government Resources
Table 4: Financial Funding Data for IAS
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $35.9 $20.5 $15.4
FY 05 $27.1 11.6 15.5
FY 06 $30.6 $14.5 $16.1
Source: OMB FY2006 Exhibit 53.
System: Phytosanitary Certificate Issuance and Tracking System (PCIT)
Brief description: This system is expected to establish a new process to
collect and track phytosanitary certificates issued by the department,
which attest to compliance with import regulations of importing countries.
It is also intended to provide better service to users by reducing the
need for repetitive data entry from applicants and enabling certifying
officials to deliver certificates in a timelier manner.
Investment stage: New
Business Reference Model category: Services for Citizens
Table 5: Financial Funding Data for PCIT
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $0.9 $0.9 $0.0
FY 05 $3.6 3.6 0.0
FY 06 $2.5 $2.5 $0.0
Source: OMB FY2006 Exhibit 53.
System: Processed Commodity Inventory Management System (PCIMS)
Brief description: This system is designed to support the annual
acquisition, tracking, and distribution of commodities acquired by
Agriculture for domestic and foreign food assistance programs by providing
financial and program management, reporting, and control to track
commodity requests against purchases and distributions from inventory.
Investment stage: Steady state
Business Reference Model category: Management of Government Resources
Table 6: Financial Funding Data for PCIMS
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $12.9 $4.3 $8.6
FY 05 $12.2 3.7 8.6
FY 06 $12.7 $2.8 $9.9
Source: OMB FY2006 Exhibit 53.
System: Store Tracking and Redemption System Redesign (STARS II)
Brief description: This system is intended to support the department's
Food-Stamp program mission by tracking and monitoring food
coupon/electronic benefit redemption activities and regulatory violations
by businesses and associated administrative actions related to enforcement
of penalties, among other things. This initiative is expected to replace
the current legacy system, which has been in place since 1993.
Investment stage: New
Business Reference Model category: Support Delivery of Services
Table 7: Financial Funding Data for STARS II
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $12.7 $12.4 $0.3
FY 05 $7.0 5.2 1.8
FY 06 $6.4 $0.0 $6.4
Source: OMB FY2006 Exhibit 53.
Department of Commerce
System: Advanced Weather Interactive Processing System (AWIPS)
Brief description: This system is designed to be an interactive computer
system that integrates all meteorological and hydrological data and all
satellite and radar data to enable the forecaster to prepare and issue
more accurate and timely forecasts and warnings.
Investment stage: Mixed life cycle
Business Reference Model category: Services for Citizens
Table 8: Financial Funding Data for AWIPS
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $49.2 $14.0 $35.2
FY 05 $49.5 12.7 36.8
FY 06 $52.2 $14.1 $38.1
Source: OMB FY2006 Exhibit 53.
System: Comprehensive Large Array-Data Stewardship System (CLASS)
Brief description: This system is expected to provide an integrated
solution to weather and water data archive and access, including an access
portal with search, browse, and geospatial capabilities for users to
obtain environmental data, contributing to improvements in prediction
capabilities.
Investment stage: New
Business Reference Model category: Services for Citizens
Table 9: Financial Funding Data for CLASS
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $3.2 $3.2 $0.0
FY 05 $9.6 9.6 0.0
FY 06 $7.6 $7.6 $0.0
Source: OMB FY2006 Exhibit 53.
System: Economic Census and Surveys (ECON)
Brief description: This investment is designed to provide statistical
programs that count and profile U.S. businesses and government
organizations through the gathering of surveys and principal economic
indicators in order to conduct research and technical studies.
Investment stage: Steady state
Business Reference Model category: Support Delivery of Services
Table 10: Financial Funding Data for ECON
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $53.6 $0.0 $53.6
FY 05 $34.0 0.0 34.0
FY 06 $51.4 $0.0 $51.4
Source: OMB FY2006 Exhibit 53.
System: Improve the Automated Export System (AES)
Brief description: The current system is designed to expedite monthly
statistics on international trade, remedy shortcomings in export
statistics, and help to control the export of weapons or other hazardous
items that could be a threat to U.S. national security or public welfare.
The proposed initiative is designed to improve the current system to
handle electronic filing of all export transactions, incorporate an
electronic manifest system, and provide for verification of export
information reported on export transactions.
Investment stage: Mixed life cycle
Business Reference Model category: Support Delivery of Services
Table 11: Financial Funding Data for AES
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $0.0 $0.0 $0.0
FY 05 $0.0 0.0 0.0
FY 06 $6.6 $6.6 $0.0
Source: OMB FY2006 Exhibit 53.
System: National Weather Service Telecommunication Gateway (NWSTG) System
Brief description: The current system is designed to collect and
distribute raw and processed hydrometeorological data and products,
disseminating weather observations and guidance to a national and
international community of customers. Improvements to current system are
expected to provide sufficient performance, capacity, and catastrophic
backup capability to meet current and future demands for data.
Investment stage: Mixed life cycle
Business Reference Model category: Support Delivery of Services
Table 12: Financial Funding Data for NWSTG
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $13.2 $5.5 $7.7
FY 05 $13.0 5.0 8.0
FY 06 $11.1 $0.8 $10.3
Source: OMB FY2006 Exhibit 53.
System: Satellite Operations Control Center Command and Data Acquisition
(SOCC/CDA)
Brief description: This system is designed to command and control
Commerce's operational environmental satellites and to acquire and manage
the weather and water data the satellites collect, in order to provide
support functions that are not available commercially, such as real-time
hurricane support.
Investment stage: Mixed life cycle
Business Reference Model category: Services for Citizens
Table 13: Financial Funding Data for SOCC/CDA
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $11.7 $11.7 $0.0
FY 05 $10.5 6.1 4.4
FY 06 $12.6 $7.7 $5.0
Source: OMB FY2006 Exhibit 53.
Department of Energy
System: Energy Sciences Network (ESnet)
Brief description: This project is designed to support scientific research
by providing an interoperable, effective, and reliable communications
infrastructure and network services to the Department of Energy research
facilities.
Investment stage: Mixed life cycle
Business Reference Model category: Services for Citizens
Table 14: Financial Funding Data for ESnet
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $18.9 $11.3 $7.6
FY 05 $18.3 9.5 8.8
FY 06 $18.3 $9.3 $9.0
Source: OMB FY2006 Exhibit 53.
System: E-content Management System (eCMS)
Brief description: This system is expected to be an enterprisewide,
integrated document and records management system that will include portal
accessibility and integration with knowledge management tools in order to
improve decision and service delivery quality and serve as a resource for
operations management.
Investment stage: New
Business Reference Model category: Support Delivery of Services
Table 15: Financial Funding Data for eCMS
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $2.3 $2.3 $0.0
FY 05 $2.7 2.7 0.0
FY 06 $6.4 $4.1 $2.3
Source: OMB FY2006 Exhibit 53.
System: Integrated Planning, Accountability, and Budgeting System
Information System (IPABS-IS)
Brief description: This system is designed to support the routine
collection and reporting needs of Energy for life-cycle planning, budget
formulation, and project and budget execution.
Investment stage: Steady state
Business Reference Model category: Support Delivery of Services
Table 16: Financial Funding Data for IPABS-IS
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $3.1 $0.0 $3.1
FY 05 $2.9 0.0 2.9
FY 06 $2.8 $0.0 $2.8
Source: OMB FY2006 Exhibit 53.
System: Licensing Support Network (LSN)
Brief description: This is a Web-based system that is intended to make
relevant documentary material supporting the Nuclear Regulatory License
Application available to users, as part of the requirements of the Nuclear
Waste Policy Act.
Investment stage: Steady state
Business Reference Model category: Services for Citizens
Table 17: Financial Funding Data for LSN
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $39.6 $39.6 $0.0
FY 05 $13.6 0.0 13.6
FY 06 $9.6 $0.0 $9.6
Source: OMB FY2006 Exhibit 53.
System: Los Alamos National Laboratory Enterprise Project (LANL ERP)
Brief description: This investment is intended to identify, design, and
implement the systems, processes, and controls related to financial
management, human resources, supply chain management, facilities
maintenance, information management, project management, and manufacturing
in order to lower costs and provide more efficient operations and improved
management.
Investment stage: Mixed life cycle
Business Reference Model category: Management of Government Resources
Table 18: Financial Funding Data for LANL ERP
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $43.2 $39.3 $3.9
FY 05 $45.5 41.3 4.2
FY 06 $43.7 $41.3 $2.4
Source: OMB FY2006 Exhibit 53.
Department of Transportation
System: Asset Supply Chain Management (ASCM)
Brief description: This investment is intended to provide the Department
of Transportation (DOT) with asset management and supply chain management
information systems to track and manage over $21 billion in federal
government assets. Reducing the number of information systems, optimizing
supply chain operations, and streamlining business operations of employees
are expected to result in reduced costs to the agency.
Investment stage: Mixed life cycle
Business Reference Model category: Management of Government Resources
Table 19: Financial Funding Data for ASCM
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $5.0 $5.0 $0.0
FY 05 $6.0 6.0 0.0
FY 06 $13.2 $13.2 $0.0
Source: OMB FY2006 Exhibit 53.
System: DOT Financial System Consolidation
Brief description: This program is expected to consolidate several major
and nonmajor DOT financial systems to interface or integrate all related
systems in order to eliminate redundant data and processes.
Investment stage: New
Business Reference Model category: Management of Government Resources
Table 20: Financial Funding Data for DOT Financial System Consolidation
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $51.0 $6.2 $44.8
FY 05 $64.5 11.1 53.3
FY 06 $44.1 $0.2 $44.0
Source: OMB FY2006 Exhibit 53.
System: National Transit Database (NTD)
Brief description: This system is designed to collect performance data
from over 640 local transit agencies for the purpose of reporting
statistical data on the U.S. transit industry.
Investment stage: Steady state
Business Reference Model category: Services for Citizens
Table 21: Financial Funding Data for NTD
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $2.2 $0.0 $2.2
FY 05 $3.7 0.0 3.7
FY 06 $3.7 $0.0 $3.7
Source: OMB FY2006 Exhibit 53.
System: Next Generation Air/Ground Communications (NEXCOM)
Brief description: This system is intended to provide air pilot/controller
voice and data communications by utilizing a digital-based air/ground
communication system.
Investment stage: New
Business Reference Model category: Services for Citizens
Table 22: Financial Funding Data for NEXCOM
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $28.7 $28.7 $0.0
FY 05 $29.5 29.5 0.1
FY 06 $33.8 $33.5 $0.3
Source: OMB FY2006 Exhibit 53.
System: System Approach for Safety Oversight (SASO)
Brief description: This system is expected to consolidate the agency's 28
oversight systems on aviation regulatory compliance into 5 integrated
aviation safety risk management systems. Its intended purpose is to allow
applicable government agencies and the aviation industry to use common
system safety applications and databases for managing and overseeing
flight safety.
Investment stage: New
Business Reference Model category: Services for Citizens
Table 23: Financial Funding Data for SASO
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $7.2 $7.2 $0.0
FY 05 $7.3 7.3 0.0
FY 06 $10.7 $10.7 $0.0
Source: OMB FY2006 Exhibit 53.
System: Wide-Area Augmentation System (WAAS)
Brief description: This is a navigation system that is designed to provide
navigation across the entire United States for all classes of aircraft in
all flight operations, including en-route navigation, airport departures,
and airport arrivals including precision landing approaches in all weather
conditions.
Investment stage: Mixed life cycle
Business Reference Model category: Services for Citizens
Table 24: Financial Funding Data for WAAS
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $105.6 $99.4 $6.2
FY 05 $122.6 99.2 23.4
FY 06 $124.4 $100.0 $24.4
Source: OMB FY2006 Exhibit 53.
Department of the Treasury
System: Customer Account Data Engine (CADE)
Brief description: This system is part of a modernization program that is
expected to provide the Department of the Treasury with the capability to
manage its tax accounts utilizing new technology, applications, and
databases. This system is designed to create applications for daily
posting, settlement, maintenance, refund processing, and issue detection
for taxpayer tax account and return data to improve customer service and
compliance.
Investment stage: Mixed life cycle
Business Reference Model category: Support Delivery of Services
Table 25: Financial Funding Data for CADE
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $100.6 $100.6 $0.0
FY 05 $109.9 109.9 0.0
FY 06 $109.9 $109.9 $0.0
Source: OMB FY2006 Exhibit 53.
System: Debt Management Accounting System (DMAS)
Brief description: This system is designed to be a financial accounting
system for activities associated with Treasury's debt collection program
to track funds recovered by the agency, post these funds to the proper
account in an accurate and timely manner, and transfer moneys due to the
appropriate government agencies. The system is also designed to record the
general ledger activity and produce operational, management, and standard
external reports.
Investment stage: Mixed life cycle
Business Reference Model category: Support Delivery of Services
Table 26: Financial Funding Data for DMAS
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $4.4 $2.3 $2.1
FY 05 $4.1 1.8 2.3
FY 06 $4.2 $1.9 $2.4
Source: OMB FY2006 Exhibit 53.
System: Electronic Management System (EMS)
Brief description: This system is designed to be a front-end processing
system that receives, validates, stores, forwards to mainframe electronic
filing systems, and acknowledges electronic files containing tax
documents. The system is intended to receive returns from third parties,
acknowledge the receipt of information, format the information for
mainframe processing, provide acknowledgements to the third parties, and
send state return data to participating states.
Investment stage: Steady state
Business Reference Model category: Management of Government Resources
Table 27: Financial Funding Data for EMS
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $11.8 $3.1 $8.8
FY 05 $9.1 2.0 7.1
FY 06 $10.3 $2.0 $8.2
Source: OMB FY2006 Exhibit 53.
System: Governmentwide Accounting and Reporting Modernization (GWA)
Brief description: This system is designed to produce accurate,
accessible, and timely governmentwide financial information through the
streamlining of reports and the reduction of the reconciliation burden on
government agencies in order to minimize the amount of labor necessary to
transfer financial information.
Investment stage: Mixed life cycle
Business Reference Model category: Management of Government Resources
Table 28: Financial Funding Data for GWA
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $7.7 $5.6 $2.1
FY 05 $7.8 5.5 2.3
FY 06 $7.8 $5.3 $2.5
Source: OMB FY2006 Exhibit 53.
System: Service Center Recognition/Images Processing System (SCRIPS)
Brief description: This system is intended to be a data capture,
management, and storage system used to process tax documents automatically
in order to meet mandated timelines and processing requirements for
various tax forms and the Federal Tax Deposits, which directly impacts
revenue brought into the federal treasury.
Investment stage: Steady state
Business Reference Model category: Support Delivery of Services
Table 29: Financial Funding Data for SCRIPS
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $13.0 $0.0 $13.0
FY 05 $13.7 0.0 13.7
FY 06 $15.0 $0.0 $15.0
Source: OMB FY2006 Exhibit 53.
System: Secure Payment System (SPS)
Brief description: This system is designed to be a browser-based Internet
version of the current Electronic Certification System, which will allow
federal program agencies to submit certified requests for payment
disbursement online. It is intended to provide a more secure payment
process, increase the ability to protect sensitive financial and privacy
data, and improve the financial performance of federal program agencies by
providing program agencies a method of providing financial data to
Treasury.
Investment stage: Steady state
Business Reference Model category: Support Delivery of Services
Table 30: Financial Funding Data for SPS
Millions of dollars
Fiscal year Total Development Steady state
FY 04 $4.3 $0.0 $4.3
FY 05 $3.1 0.0 3.1
FY 06 $3.1 $0.0 $3.1
Source: OMB FY2006 Exhibit 53.
GAO Contact and Staff Acknowledgments Appendix IV
David A. Powner, (202) 512-9286, [email protected] .
In addition to the contact named above, the following people made key
contributions to this report: Carol Cha, Barbara Collier, Joseph Cruz,
Lester Diamond, Valerie Hopkins, Sandra Kerr, Linda Lambert, Tammi Nguyen,
Chris Owens, Mark Shaw, Kevin Walsh, and Martin Yeung.
(310479)
www.gao.gov/cgi-bin/getrpt? GAO-06-250 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Dave Powner at (202) 512-9286 or
[email protected].
Highlights of GAO-06-250 , a report to congressional requesters
January 2006
INFORMATION TECHNOLOGY
Agencies Need to Improve the Accuracy and Reliability of Investment
Information
Each year, agencies submit to the Office of Management and Budget (OMB) a
Capital Asset Plan and Business Case-the exhibit 300-to justify each
request for a major information technology (IT) investment. The exhibit's
content should reflect controls that agencies have established to ensure
good project management, as well as showing that they have defined cost,
schedule, and performance goals. It is thus a tool to help OMB and
agencies identify and correct poorly planned or performing investments. In
its budget and oversight role, OMB relies on the accuracy and completeness
of this information. GAO was asked to determine the extent to which
selected agencies have underlying support for the information in their
fiscal year 2006 exhibit 300s. From five major departments having over $1
billion in IT expenditures in that year, GAO chose for analysis 29
exhibits for projects that supported a cross section of federal
activities.
What GAO Recommends
To improve the accuracy and value of exhibit 300s, GAO is making
recommendations aimed at improving guidance and training in exhibit 300
requirements and at ensuring limitations on reliability are disclosed and
mitigated. In response to a draft of this report, the agencies agreed with
the findings or had no comment. OMB accepted the findings but stated that
ultimate responsibility for the accuracy and reliability of this
information lies with the agencies.
Underlying support was often inadequate for information provided in the
exhibit 300s reviewed. Three general types of weaknesses were evident:
o All exhibit 300s had documentation weaknesses. Documentation
either did not exist or did not fully agree with specific areas of
the exhibit 300. For example, both these problems occurred in
relation to calculations of financial benefits for most
investments. In addition, for 23 of the 29 investments,
information on performance goals and measures was not supported by
explanations of how agencies had initially measured their baseline
levels of performance (from which they determine progress) or how
they determined the actual progress reported in the exhibit 300.
o Agencies did not always demonstrate that they complied with
federal or departmental requirements or policies with regard to
management and reporting processes. For example, 21 investments
were required to use a specific management system as the basis for
the cost, schedule, and performance information in the exhibit
300, but only 6 did so following OMB-required standards. Also,
none had cost analyses that fully complied with OMB requirements
for cost-benefit and cost-effectiveness analyses. In contrast,
most investments did demonstrate compliance with information
security planning and training requirements.
o In sections that required actual cost data, these data were
unreliable because they were not derived from cost-accounting
systems with adequate controls. In the absence of such systems,
agencies generally derived cost information from ad hoc processes.
Officials from the five agencies (the Departments of Agriculture,
Commerce, Energy, Transportation, and the Treasury) attributed these
shortcomings in support to lack of understanding of a requirement or how
to respond to it. Agency officials mentioned in particular insufficient
guidance or training, as well as lack of familiarity with particular
requirements.
The weaknesses in the 29 exhibit 300s raise questions regarding the
sufficiency of the business cases for these major investments and the
quality of the projects' management. Without adequate support in key
areas, OMB and agency executives may be depending on unreliable
information to make critical decisions on IT projects, thus putting at
risk millions of dollars.
Further, although the 29 examples cannot be directly projected to the over
one thousand business cases developed each year across the federal
government, the results suggest that the underlying causes for the
weaknesses identified need attention. These weaknesses and their causes
are also consistent with problems in project and investment management
that are pervasive governmentwide, including at such agencies as the
Departments of Defense, Health and Human Services, and Homeland Security,
as documented in reports by GAO and others.
*** End of document. ***