Financial Management Systems: Lack of Disciplined Process Puts
Effective Implementation of Treasury's Governmentwide Financial
Report System at Risk (21-APR-06, GAO-06-413).
For the past 9 years, since the first audit of the consolidated
financial statements of the U.S. government (CFS), one of the
major impediments to our ability to render an opinion on the CFS
is that the federal government has not had adequate system,
controls, and procedures to properly prepare the CFS. To address
some of the internal control weaknesses identified in our audit
report, Treasury began developing the Governmentwide Financial
Report System (GFRS). The goal of this new system is to directly
link information from federal agencies' audited financial
statements to amounts reported in the CFS, a concept that we
strongly support. We reported internal control weaknesses and GAO
recommendations regarding the preparation of the CFS, along with
progress made in this area in a separate report. This report
provides our assessment of Treasury's ongoing effort to develop
and implement GFRS and makes recommendations for reducing the
risks associated with the development of GFRS.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-413
ACCNO: A52218
TITLE: Financial Management Systems: Lack of Disciplined Process
Puts Effective Implementation of Treasury's Governmentwide
Financial Report System at Risk
DATE: 04/21/2006
SUBJECT: Financial management systems
Financial records
Financial statement audits
Financial statements
Internal controls
Risk assessment
Program goals or objectives
Governmentwide Financial Reporting
System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-413
* Results in Brief
* Background
* Scope and Methodology
* Effective Implementation of the Disciplined Processes Are Ke
* Lack of Disciplined Processes Puts GFRS at Risk of Not Meeti
* Treasury Has Not Developed a Concept of Operations
* Treasury Has Not Developed a Detailed Project Plan and Sched
* Impacts of Not Following OMB Guidance
* GFRS Has Some Usability Issues
* Conclusions
* Recommendations for Executive Action
* Agency Comments and Our Evaluation
* Appendix I: Comments from the Department of Treasury
* Order by Mail or Phone
Report to the Secretary of the Treasury
United States Government Accountability Office
GAO
April 2006
FINANCIAL MANAGEMENT SYSTEMS
Lack of Disciplined Processes Puts Effective Implementation of Treasury's
Governmentwide Financial Report System at Risk
GAO-06-413
Contents
Letter 1
Results in Brief 2
Background 3
Scope and Methodology 5
Effective Implementation of the Disciplined Processes Are Key to Reducing
Project Risks 6
Lack of Disciplined Processes Puts GFRS at Risk of Not Meeting Its
Performance, Schedule, and Cost Goals 10
Conclusions 17
Recommendations for Executive Action 18
Agency Comments and Our Evaluation 18
Appendix I Comments from the Department of Treasury 21
Figures
Figure 1: Percent of Effort Associated with Undisciplined Projects 8
Figure 2: Software Development Trade-off Triangle 9
Figure 3: Relationship between Requirements Development and Testing 12
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office
Washington, DC 20548
April 21, 2006
The Honorable John W. Snow The Secretary of the Treasury
For the past 9 years, we have reported that the federal government did not
have adequate systems, controls, and procedures to properly prepare its
consolidated financial statements. Many of the weaknesses in internal
control that contributed to our continuing disclaimers of opinion were
identified by federal agency financial statement auditors during their
audits of federal agencies' financial statements and were reported in
detail with recommendations to the federal agencies in separate reports.
However, some of the internal control weaknesses were identified during
our tests of the Department of the Treasury's (Treasury) process for
preparing the consolidated financial statements of the U.S. government
(CFS). The ability to prepare the CFS has been a long-standing challenge
for Treasury's Financial Management Service. To address some of the
internal control weaknesses identified in our audit report,1 Treasury
began developing the Governmentwide Financial Report System (GFRS). The
goal of this new system is to directly link information from federal
agencies' audited financial statements to amounts reported in the CFS, a
concept that we strongly support and is a necessary prerequisite for our
reliance on the individual agency audits in auditing the CFS. Once
Treasury is able to achieve adequate systems, controls, and procedures to
properly prepare its consolidated financial statements, a major impediment
to our ability to audit the CFS would be eliminated.
This report provides our assessment of Treasury's ongoing effort to
develop and implement GFRS. Although the implementation of any information
system, such as GFRS, is never risk-free, organizations that follow and
effectively implement accepted best practices in systems development and
implementation (commonly referred to as disciplined processes) have been
shown to reduce these risks to acceptable levels. The term acceptable
levels acknowledges the fact that any system acquisition has risks and
will suffer the adverse consequences associated with defects. A
disciplined software development and acquisition process can maximize the
likelihood of achieving the intended results (performance) within
established resources (costs) on schedule. Effective implementation of the
disciplined processes reduces the potential for risks to occur and helps
prevent those that do occur from having any significant adverse impact on
the cost, timeliness, and performance of the project. Our reviews have
found, like those performed by others, that the effectiveness of the
processes used to manage the project provide valuable insight into whether
a project will meet its cost, schedule, and performance objectives.
1Department of the Treasury, 2005 Financial Report of the United States
Government (Washington, D.C.: December 2005). This report includes GAO's
audit report.
We last briefed your staff on our initial assessment of this development
effort in September 2004. This report updates our work and focuses on
whether Treasury has effectively implemented disciplined processes
necessary to reduce the risks to the GFRS project to acceptable levels and
accordingly provide reasonable assurance that GFRS will meet its
performance, schedule, and cost goals. We reported internal control
weaknesses and our recommendations regarding the preparation of the CFS,
along with progress made in this area in a separate report.2
Results in Brief
For fiscal years 2004 and 2005 Treasury's GFRS was able to capture certain
agency financial information from agencies' audited financial statements.
At the same time, the system was not yet at the stage of development that
it could be used to compile the consolidated financial statements from the
information that was captured. Therefore, for fiscal years 2004 and 2005
Treasury continued to have to primarily use manual procedures to prepare
the CFS as GFRS continued to be developed. We identified some risks in
Treasury's development of GFRS.
We found that Treasury had not yet effectively implemented the disciplined
processes necessary to provide reasonable assurance that GFRS will meet
its performance, schedule, and cost goals. Specifically, Treasury had not
o developed a concept of operations or any other document that
adequately defines or documents the expected performance of GFRS;
o developed a detailed project plan and schedule through
completion of GFRS;
o developed a budget justification for GFRS in its Capital Asset
Plan and Business Case (commonly referred to as the Exhibit 300),
as called for in Office of Management and Budget (OMB) Circular
No. A-11;3 and
o implemented the disciplined processes necessary to effectively
manage the GFRS project, which has contributed to usability
problems encountered by its users.
2 GAO, Financial Audit: Significant Internal Control Weaknesses Remain in
Preparing the Consolidated Financial Statements of the U.S. Government,
GAO-06-415 (Washington, D.C.: Apr. 21, 2006).
In our work at certain other federal agencies, we have found that not
effectively implementing disciplined processes has led to a range of
problems, from increased cost and reduced functionality to system failure.
Going forward, it will be important that Treasury better mitigate its
risks so that long-standing internal control weaknesses regarding the
preparation of the CFS can be eliminated and, more importantly, Treasury
ends up with a system that fully meets its and agencies' needs. This
report includes three recommendations to the Secretary of the Treasury and
the Fiscal Assistant Secretary focused on reducing the risks associated
with the development of GFRS.
In commenting on a draft of this report, Treasury stated that it concurs
with our recommendations and is working to adopt them. Treasury also
stated that it plans to develop a concept of operations for GFRS and
recently started to develop a budget justification for GFRS in a Capital
Asset Plan and Business Case. Further, Treasury stated that although it
uses manual procedures to prepare the CFS, it does not agree that manual
processes necessarily create internal control weaknesses. We agree. Our
discussion of Treasury's manual processes was not intended to imply that
using manual processes will by itself lead to internal control weaknesses;
rather, it was to highlight that Treasury, as a user of GFRS, has to
perform manual compilation procedures because GFRS was not yet at the
stage of development that it could be used to compile the consolidated
financial statements from the agency financial information that was
captured.
Background
Treasury began implementing a new process for preparing the CFS using GFRS
during fiscal year 2004. GFRS is an application that is being developed to
meet the business needs of Treasury to prepare the CFS. Treasury has
stated that the goal of the new system is to be able to directly link
information from federal agencies' audited financial statements to amounts
reported in the CFS and to address our recommendations4 regarding the
preparation of the CFS. For the fiscal year 2004 and 2005 reporting
process, GFRS had enough functionality to capture certain federal agency
financial information from federal agencies' audited financial statements.
We also found that Treasury made progress in demonstrating that amounts in
the Balance Sheet and the Statement of Net Cost were consistent with
federal agencies' audited financial statements prior to eliminating
intragovernmental activity and balances. GFRS, though, was not yet at the
stage of development that it could be used to compile the CFS from the
information that was captured. Treasury's plans had been to evaluate and
consider the manual processes used during fiscal year 2004 to develop the
necessary requirements needed for the development of the compilation
module of GFRS. Although some enhancements to GFRS were implemented for
fiscal year 2005, Treasury continued to primarily use manual procedures to
prepare the most recently issued CFS.
3Section 300 of OMB Circular No. A-11, Preparation, Submission, and
Execution of the Budget (Nov. 2, 2005), sets forth requirements for
federal agencies for planning, budgeting, acquiring, and managing
information technology capital assets.
As part of the concept of directly linking information from agencies'
audited financial statements to amounts reported in the CFS, Treasury has
classified 35 federal agencies as verifying5 agencies and approximately
125 as nonverifying agencies and required these agencies to enter data in
GFRS. Verifying agencies are required to reclassify certain of their
audited financial statement line items to the appropriate CFS line items
within GFRS. Both verifying and nonverifying agencies are required to
enter certain of their financial statement notes and other requested data
into GFRS. Treasury then uses this information to prepare the CFS. GFRS
users at verifying and nonverifying agencies include financial statement
preparers, Chief Financial Officers (CFOs), Inspectors General (IGs), and
contracted independent public accountants. GFRS grants certain permissions
to different user roles which limit each type of user to designated
locations within the system. Only certain roles have permission to input,
modify, and approve data.
4GAO, Financial Audit: Process for Preparing the Consolidated Financial
Statements of the U.S. Government Continues to Need Improvement,
GAO-05-407 (Washington, D.C.: May 4, 2005).
5Treasury defines the verifying agencies as the 24 Chief Financial Officer
(CFO) Act agencies, Export-Import Bank of the United States, Farm Credit
System Insurance Corporation, Federal Communications Commission, Federal
Deposit Insurance Corporation, National Credit Union Administration, U.S.
Postal Service, Pension Benefit Guaranty Corporation, Railroad Retirement
Board, Securities and Exchange Commission, Smithsonian Institution, and
Tennessee Valley Authority. Treasury also refers to the verifying agencies
as "significant" agencies.
Scope and Methodology
As part of our annual audit of the CFS, we evaluate Treasury's financial
reporting procedures and related internal control, including the
development of GFRS. Our work was performed in accordance with U.S.
generally accepted government auditing standards. In fiscal years 2004 and
2005, GFRS was used to collect federal agencies' audited financial
statement data compiled in the CFS.
To assess Treasury's implementation of disciplined processes in the
development of GFRS, we reviewed industry standards and best practices
from the Institute of Electrical and Electronics Engineers (IEEE),
Software Engineering Institute (SEI), OMB Circular No. A-11, and prior GAO
reports. Beginning in fiscal year 2004, we reviewed and analyzed available
GFRS documentation and made inquiries of Treasury's GFRS project team and
other Treasury personnel to determine whether Treasury has effectively
implemented disciplined processes necessary to reduce the risks to the
GFRS project to acceptable levels and accordingly provide reasonable
assurance that GFRS will meet its performance, schedule, and cost goals.
We briefed Treasury on our initial assessment of this development effort
in September 2004. We reported internal control weaknesses related to GFRS
in our fiscal year 2004 audit report6 and made a recommendation related to
GFRS in our fiscal year 2004 internal control report.7 In fiscal year
2005, we reviewed documentation related to the planned enhancements for
GFRS for 2005 and made inquiries of Treasury's GFRS project team and other
Treasury personnel to further determine whether Treasury has effectively
implemented disciplined processes. We again reported internal control
weaknesses related to GFRS in our fiscal year 2005 audit report.
We requested comments on a draft of this report from the Secretary of the
Treasury or his designee. Treasury's comments are reprinted in appendix I
and are also discussed in the Agency Comments and Our Evaluation section.
6Department of the Treasury, 2004 Financial Report of the United States
Government (Washington, D.C.: December 2004). This report includes GAO's
audit report.
7 GAO-05-407 .
Effective Implementation of the Disciplined Processes Are Key to Reducing
Project Risks
Disciplined processes, which are fundamental to successful systems
development and implementation efforts, have been shown to reduce the
risks associated with software development and acquisition to acceptable
levels. A disciplined software development and acquisition process can
maximize the likelihood of achieving the intended results (performance)
within established resources (costs) on schedule. Although there is no
standard set of practices that will ever guarantee success, several
organizations, such as SEI8 and IEEE,9 as well as individual experts, have
identified and developed the types of policies, procedures, and practices
that have been demonstrated to reduce development time and enhance
effectiveness. The key to having a disciplined system development effort
is to have disciplined processes in multiple areas, including project
planning and management, requirements management, configuration
management, risk management, quality assurance, and testing. Effective
processes should be implemented in each of these throughout the project
life cycle because change is constant. Effectively implementing the
disciplined processes necessary to reduce project risks to acceptable
levels is hard to achieve because a project must effectively implement
several best practices, and inadequate implementation of any one may
significantly reduce or even eliminate the positive benefits of the
others.
As part of the disciplined processes, acquiring and implementing a new
financial management system such as GFRS requires a methodology that
starts with a clear definition of the organization's mission and strategic
objectives and ends with a system that meets specific information needs.
We have seen many system efforts fail because federal agencies started
with a general need, but did not define in precise terms (1) the specific
problems they were trying to solve, (2) what their operational needs were,
and (3) what specific information requirements flowed from these
operational needs. Instead, they plunged into the acquisition and
implementation process in the belief that these specifics would somehow be
defined along the way. The typical result was that systems were delivered
well past anticipated milestones; failed to perform as expected; and,
accordingly, were over budget because of required costly modifications.
8SEI is a federally funded research and development center operated by
Carnegie Mellon University and sponsored by the Department of Defense. The
SEI objective is to provide leadership in software engineering and in the
transition of new software engineering technologies into practice.
9IEEE is a nonprofit, technical professional association that develops
standards for a broad range of global industries including the information
technology and information assurance industries. The IEEE standards
program serves the global needs of industry, government, and the public.
It also works to assure the effectiveness and high visibility of the
standards program both within IEEE and throughout the global community.
Figure 1 shows how organizations that do not effectively implement the
disciplined processes lose the productive benefits of their efforts as a
project continues through its development and implementation cycle.
Although undisciplined projects show a great deal of productive work at
the beginning of the project, the rework associated with defects begins to
consume more and more resources. In response, processes are adopted in the
hopes of managing what later turns out, in reality, to have been
unproductive work. Generally, these processes are "too little, too late"
and rework begins to consume more and more resources because sufficient
foundations for building the systems were not established or not
established adequately. Experience has shown that projects for which
disciplined processes are not implemented at the beginning are forced to
implement them later when it takes more time and they are less
effective.10
10Steve McConnell, Rapid Development: Taming Wild Software Schedules
(Redmond, Wash: Microsoft Press, 1996).
Figure 1: Percent of Effort Associated with Undisciplined Projects
As shown in figure 1, a major consumer of project resources in
undisciplined efforts is rework (also known as thrashing). Rework occurs
when the original work has defects or is no longer needed because of
changes in project direction. Disciplined organizations focus their
efforts on reducing the amount of rework because it is expensive. Fixing a
requirements defect after the system is released costs anywhere from 10 to
100 times as much as fixing it when the requirements are defined.11 As
shown in figure 1, projects that are unable to successfully address their
rework will eventually only be spending their efforts on rework and the
11Steve McConnell, Code Complete, Second Edition (Redmond, Wash: Microsoft
Press, 2004).
associated processes rather than on productive work. In other words, the
project will continually find itself reworking items.
A disciplined software development and acquisition process can maximize
the likelihood of achieving the intended results (performance) within
established resources (costs) on schedule. Because of this relationship,
these factors can be viewed as an equilateral triangle as shown in figure
2.
Figure 2: Software Development Trade-off Triangle
The performance corner of the triangle is also referred to as the product
corner. This corner includes quality and all other product-related
attributes including complexity, usability, maintainability, and defect
rate. For example, beyond a certain point, it is difficult to increase
performance without changing the cost and/or schedule attributes.
Similarly, a schedule cannot be drastically compressed without degrading
the quality of the software project or increasing the cost of development.
Schedule, cost, and performance must be in balance for a project to
succeed. The following sections of this report address our observations
regarding the performance, schedule, and cost of Treasury's GFRS
development effort.
Lack of Disciplined Processes Puts GFRS at Risk of Not Meeting Its Performance,
Schedule, and Cost Goals
Treasury's development of GFRS is a positive initiative and Treasury has
made progress towards preparing the CFS based on agencies' audited
financial statements, which is one of our principal concerns. While we
support the goals of GFRS, we are concerned that Treasury has not yet
effectively implemented a number of disciplined processes that have been
shown to reduce project risks to acceptable levels. Specifically, we noted
that Treasury has not yet implemented the types of project management
processes necessary to provide the information required to assess the
progress of GFRS and whether it is meeting its cost, schedule, and
performance objectives. First, Treasury has not described its vision for
the project in a manner that can be used to guide the development efforts.
Although disciplined projects use a concept of operations document to
serve this purpose, Treasury has not yet developed such a document.
Treasury also has not developed a detailed project plan or schedule
through the completion of GFRS or followed OMB's guidance for major
investment planning, budgeting, and acquisition. Some adverse impacts
associated with these process weaknesses have already been realized.
Specifically, significant usability issues have arisen and little
meaningful project management information is available to assess the
project performance.
Treasury Has Not Developed a Concept of Operations
Treasury has not adequately defined or documented the expected
functionality for GFRS. One approach for describing the functionality that
is expected from a system is to document the vision in a concept of
operations. According to IEEE Standard 1362-1998, a concept of operations
document is normally one of the first documents produced during a
disciplined development effort because it describes system characteristics
for a proposed system from the user's viewpoint. This is important because
a good concept of operations document can be used to communicate overall
quantitative and qualitative system characteristics to the user,
developer, and other organizational elements typically involved in a
systems development effort. This allows the reader to understand the user
organizations, missions, and organizational objectives from an integrated
systems point of view. Until such a document is developed, it is virtually
impossible for the project to develop a firm foundation for its
requirements and testing activities. Experience has shown that poor
requirements12 are a major factor associated with projects that do not
meet their cost, schedule, and performance objectives. Furthermore,
without good requirements, it is impossible to develop a disciplined
testing program. Figure 3 shows the relationship between requirements and
testing and how the concept of operations provides the foundation for the
requirements.
12Requirements are the specifications that system developers and program
managers use to design, develop, and acquire a system. They need to be
unambiguous, consistent with one another, verifiable, and directly
traceable to higher level business or functional requirements. It is
critical that requirements flow directly from the organization's concept
of operations.
Figure 3: Relationship between Requirements Development and Testing
The Treasury project team stated they consider the November 2001 Working
Group Report on the Financial Report of the United States Government
Preparation Process to be their concept of operations. Our analysis noted
at least two reasons why this document would not be acceptable for that
purpose. First, the November 2001 report resembles an alternatives
analysis where Treasury weighed several options to change its current
process for preparing the CFS. The report does not address the objectives
of a concept of operations because it does not outline the business
processes that are expected to be implemented in the new system. For
example, when we compared the November 2001 report to the current
functionality of GFRS, we found that the November 2001 report stated that
federal agencies would be required to submit an analysis of changes in net
position and to split net position between intragovernmental and
operations with the public. However, over 2 years ago Treasury stated to
GAO that they would not be requiring federal agencies to split their net
position and did not provide an alternative solution for addressing this
key net position weakness affecting the preparation of the CFS. Second,
the November 2001 report was issued prior to the Federal Accounting
Standards Advisory Board's issuance of Statement of Federal Financial
Accounting Standard No. 24, Selected Standards for the Consolidated
Financial Report of the United States Government, which requires two
additional financial statements to be included in the Financial
Report-Reconciliation of Net Operating Cost and Unified Budget Deficit and
Statement of Changes in Cash Balance from Unified Budget and Other
Activities. Because the November 2001 report could not contemplate the
preparation of these two statements, it is at least partially obsolete in
terms of reporting requirements for two of the five primary consolidated
financial statements.
Going forward, an effective concept of operations could help Treasury
reach its goal of having GFRS, when fully implemented, resolve several
continuing weaknesses that we have identified during our CFS audit work,
including providing additional functionality that would allow Treasury to
eliminate some of the manual processes and provide improvements from its
previous process. Treasury has not yet developed and documented in a
concept of operations document or any other document (1) the corrective
actions for financial reporting weaknesses that the system is expected to
address and (2) the functionality it needs GFRS to provide when it is
completed to implement these corrective actions. Accordingly, Treasury
does not have adequate assurance that the resulting system will have the
needed functionality to address the weaknesses that Treasury management
may expect the system to eliminate or reduce the time it takes Treasury to
produce the CFS. This basic information is critical for the project team
to effectively develop a complete set of high-level requirements for GFRS
and reach agreement with Treasury management on the functionality GFRS is
expected to provide and how to determine when GFRS is complete and has
ultimately met management's expectations.
Treasury Has Not Developed a Detailed Project Plan and Schedule through
Completion of GFRS
Treasury has not clearly defined what GFRS will encompass and when that
functionality is expected to be delivered. This basic information should
serve as the foundation to support realistic project scheduling efforts,
so that Treasury could (1) determine the work needed to complete the
project and (2) make informed decisions on when that work can be expected
to be completed.
While we understand that GFRS will take several years to develop, a
"blueprint" guiding these efforts has not been developed. Rather than
using best practices to schedule out the functionality to be developed and
implemented over the entire development period, each year Treasury decides
on the high-level requirements to be developed and implemented only during
that given year. These requirements are not "placed in context" to the
overall system goals and needs. This approach presents the following
issues.
o It is unclear how the requirements selected for implementation
each year move GFRS closer to providing the necessary
functionality to address the material weaknesses Treasury expects
the system to correct.
o Without a detailed project plan and schedule through completion
of GFRS, it is unclear whether the project is capable of meeting
its established time frames and goals, how to determine the
progress made with a given year's efforts, or how to estimate the
funding necessary to develop the system.
These weaknesses have already had adverse project impacts. Specifically,
even once the requirements for a given year have been "defined", we found
that little meaningful project scheduling was used. For example, Treasury
provided us with a list of its milestones for the fiscal year 2005
development of GFRS; however, this documentation did not provide
information such as the detailed tasks to be performed, duration of the
tasks, person responsible for completing each task, or how these
development efforts link to the overall functionality required for GFRS.
Accordingly, Treasury did not have the basic project management
information necessary to determine whether its efforts were meeting the
project's cost, schedule, and performance objectives.
In our September 2004 briefing to Treasury on our initial assessment of
the GFRS development effort, we noted that Treasury faced the risk of
rework by using a single-year scheduling approach rather than developing a
schedule for the entire development period. We stated that implementing
any of management's high-level requirements for the current year of
development could cause significant rework of the functionality that was
implemented in the prior year. Disciplined organizations focus their
efforts on reducing the amount of rework because it is expensive. Without
a detailed project plan and schedule, the program does not have a process
that is designed to avoid doing things twice and runs a much greater risk
that it will cost additional time, money, and resources.
Impacts of Not Following OMB Guidance
Treasury has not yet developed a budget justification for GFRS in a
Capital Asset Plan and Business Case (commonly referred to as the Exhibit
300), called for in OMB Circular No. A-11. The OMB circular provides
instructions on how to prepare the necessary budget justification and
reporting requirements for major information technology investments,13
such as GFRS. The Exhibit 300 helps demonstrate to federal agency
management and OMB that the federal agency has employed the disciplines of
good project management, represented a strong business case for the
investment, and met other administration priorities to define the proposed
cost, schedule, and performance goals for the investment if funding
approval is obtained. The business case should include security, privacy,
and enterprise architecture, and provide the effectiveness and efficiency
gains planned by the business lines and functional operations. According
to Treasury personnel, it was a Treasury policy decision not to prepare an
Exhibit 300 for submission to OMB.
The circular states that an "exhibit 300 must be submitted for all major
investments in accordance with this section." The circular also notes that
"[g]ood budgeting requires appropriations for the full risk adjusted costs
of asset acquisition be enacted in advance to help ensure that all costs
and benefits are fully taken into account when decisions are made about
providing resources." We found that Treasury has not yet determined the
cost of GFRS to date or estimated its cost through completion. Instead,
Treasury's project team stated that, to date, they have used funds
remaining from other Treasury projects to pay for the development of
GFRS.14 According to Treasury's project team, the development of GFRS is
guided, limited, and constrained by the funds that they can "beg, borrow,
and steal" from other Treasury development efforts. As noted in OMB
Circular No. A-11, when "capital assets are funded in increments, without
certainty if or when future funding will be available, it can and
occasionally does result in poor planning, acquisition of assets not fully
justified, higher acquisition costs, project (investment) delays,
cancellation of major investments, the loss of sunk costs, or inadequate
funding to maintain and operate the assets."
13OMB Circular No. A-11, section 300 defines a major investment as a
system or project requiring special management attention because of its
importance to the mission or function of the agency, a component of the
agency or another organization; is for financial management and obligates
more than $500,000 annually; has significant program or policy
implications; has high executive visibility; has high development,
operating, or maintenance costs; or is defined as major by the agency's
capital planning and investment control process.
14We did not review Treasury's alternative funding sources.
GFRS Has Some Usability Issues
Effectively implementing disciplined processes helps ensure the system
meets the users' needs. To date, GFRS has had some significant usability
issues. In February 2005, Treasury held a forum with GFRS federal agency
users, including agency CFO, IG, and independent public accountant staff,
to discuss lessons learned from the fiscal year 2004 GFRS reporting
process and planned changes to GFRS for fiscal year 2005. At the forum,
the users noted the lack of user-friendly reporting capabilities in GFRS.
As we noted in our September 2004 briefing to Treasury, user-friendly
reporting capabilities were not available to federal agency users for
fiscal year 2004 and such functionality was critical to the ultimate
usefulness of the system. The process of viewing the information entered
into GFRS is especially burdensome for the 32 federal agencies15 required
to have this information audited. GFRS does not provide the ability for an
agency to develop one report containing all of the information required to
be entered into GFRS. Instead, users are required to run over 35 separate
reports in GFRS, which has proven to be time-consuming. Compounding this
is that agencies have only a very short period of time to enter the data
into GFRS. Specifically, federal agencies are required to have their
audited financial information in the GFRS closing package by November
18-just 3 days after their audited financial statements are due to OMB.
We believe the lack of user-friendly reporting capabilities is a result of
Treasury not effectively implementing disciplined processes. Treasury had
not defined types of reporting that would be available to the users and
when it would be available. Treasury's project team stated that it is now
trying to address this issue and has acknowledged that a lack of an
adequate understanding of users' reporting needs contributed to the
problem. They also noted the lack of funding as another reason this issue
has not been fully addressed. In our view, if Treasury had implemented
disciplined processes during the planning of GFRS, such as developing a
concept of operations and implementing an effective requirements
management process, this issue (1) would have been identified and (2)
actions could have been taken at the outset to develop a viable solution
earlier in the process rather than waiting until significant amounts of
development work had been performed.
15The Treasury Financial Manual states that the Inspector General or their
contracted independent public accountant for each verifying agency, except
those agencies with a year end other than September 30 (Federal Deposit
Insurance Corporation, National Credit Union Administration, and Farm
Credit System Insurance Corporation), must opine on the closing package
data, entered by the Chief Financial Officer into GFRS, as to its
consistency with the comparative, audited, consolidated, department-level
financial statements. Of the 35 verifying agencies, 32 are currently
required to have an audit of their closing package.
In our discussions with the Treasury project team over the reporting
issue, we were told that they are concerned that providing more robust
reporting capabilities to users could cause the system to fail and "crash"
should all of the GFRS users run these reports at the same time. In our
September 2004 briefing, we noted that one of the challenges to Treasury
was adequately testing GFRS to ensure that it could function properly when
all federal agencies use the system at the same time, which includes
allowing users to run all needed GFRS reports. Treasury's project team
told us that they were unable to test GFRS capacity because they had not
identified or developed all the needed user-friendly reporting
capabilities.
We believe other usability issues exist that directly affect Treasury as a
user of GFRS as well. For example, currently, Treasury cannot rely on the
system to compile the CFS from the information that is captured.
Therefore, for 2 years in a row, Treasury primarily used manual procedures
to prepare the CFS. While Treasury's initial plans were to evaluate and
consider the manual processes used during fiscal year 2004 to develop the
necessary requirements needed for the development of the compilation
module of GFRS, this capability has not been added and it is unclear when
or if it will be available.
Conclusions
The implementation of any major system, such as GFRS, is not a risk-free
proposition. However, organizations that follow and effectively implement
accepted best practices in systems development and implementation have
been shown to reduce these risks to acceptable levels. Treasury has not
yet done so. If it continues on this path, it runs an unnecessary risk of
building a system that may be more costly and take longer to deploy, while
not providing all of the intended system functionality. Going forward, it
will be important that Treasury take the actions necessary to implement
the disciplined processes that provide reasonable assurance that GFRS will
achieve Treasury's ultimate goal of preparing the CFS with a direct link
to federal agencies' audited financial statements and addressing
long-standing internal control weaknesses. A key first step will be for
Treasury to develop and document its view of how GFRS will operate,
including how it will be used to address significant internal control
weaknesses.
Recommendations for Executive Action
We recommend that the Secretary of the Treasury direct the Treasury Fiscal
Assistant Secretary to take the following three actions to reduce the
risks associated with the development of GFRS:
o Follow the budget justification and reporting processes set out
in OMB Circular No. A-11 and ensure that funding needs are fully
addressed.
o Develop and adopt a concept of operations that fully describes
the functionality expected to be provided by GFRS and the material
internal control weaknesses that are expected to be addressed by
the system.
o Develop and effectively implement the disciplined processes
necessary to properly manage the development of GFRS, including
the development of a detailed project plan and schedule through
the completion of the system.
Agency Comments and Our Evaluation
In written comments on a draft of this report, which are reprinted in
appendix I, Treasury stated that our report clearly points out
recommendations for reducing the risks associated with the continued
development of GFRS. Treasury also stated that it concurs with our
recommendations and is working to adopt them. Further, Treasury stated
that it plans to develop a concept of operations for GFRS and recently
started to develop a budget justification for GFRS in a Capital Asset Plan
and Business Case.
Treasury provided additional comments regarding Treasury's manual
procedures used in preparing the CFS. In particular, Treasury stated that
although it uses manual procedures to prepare the CFS, it does not agree
that manual processes necessarily create internal control weaknesses. We
agree. Our discussion of Treasury's manual processes was not intended to
imply that using manual processes will lead to internal control
weaknesses, but rather to highlight that Treasury, as a user of GFRS, has
to perform manual compilation procedures because GFRS was not yet at the
stage of development that it could be used to compile the consolidated
financial statements from the agency financial information that was
captured. Our understanding is that one of the automated functions
Treasury expected GFRS to perform was the compilation of the CFS from the
information captured rather than performing these functions manually. This
capability has not yet been added to GFRS, and it is unclear when or if it
will be available because, as we have reported, Treasury does not
currently have a concept of operations that defines or documents the
expected functionality for GFRS.
This report contains recommendations to the Secretary of the Treasury. The
head of a federal agency is required by 31 U.S.C. 720 to submit a written
statement on actions taken on these recommendations. You should submit
your statement to the Senate Committee on Homeland Security and
Governmental Affairs and the House Committee on Government Reform within
60 days of the date of this report. A written statement must also be sent
to the House and Senate Committees on Appropriations with the agency's
first request for appropriations made more than 60 days after the date of
the report.
We are sending copies of this report to the Chairmen and Ranking Minority
Members of the Senate Committee on Homeland Security and Governmental
Affairs; the Subcommittee on Federal Financial Management, Government
Information, and International Security, Senate Committee on Homeland
Security and Governmental Affairs; the House Committee on Government
Reform; and the Subcommittee on Government Management, Finance, and
Accountability, House Committee on Government Reform. In addition, we are
sending copies to the Fiscal Assistant Secretary of the Treasury. Copies
will be made available to others upon request. This report is also
available at no charge on GAO's Web site at www.gao.gov .
We appreciate the courtesy and cooperation extended to us by your staff
throughout our work. We look forward to continuing to work with your
offices to help improve systems development efforts in the federal
government. If you have any questions about the contents of this report,
please contact Gary Engel, Director, Financial Management and Assurance,
who may be reached at (202) 512-3406 or by e-mail at
[email protected] , or Keith Rhodes, Chief Technologist, Applied Research and
Methods, who may be reached at (202) 512-6412 or by e-mail at
[email protected]. Key contributors to this letter were Chris Martin, Lynda
Downing, and Katherine Schirano.
Gary T. Engel Director Financial Management and Assurance
Keith A. Rhodes Chief Technologist Applied Research and Methodology Center
for Engineering and Technology
Appendix I: Comments from the Department of Treasury Appendix I: Comments
from the Department of Treasury
(198415)
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts newly
released reports, testimony, and correspondence on its Web site. To have
GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: [email protected]
Automated answering system: (800) 424-5454 or (202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202) 512-4800 U.S.
Government Accountability Office, 441 G Street NW, Room 7149 Washington,
D.C. 20548
www.gao.gov/cgi-bin/getrpt? GAO-06-413 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Gary Engel at (202) 512-3406 or
[email protected].
Highlights of GAO-06-413 , a report to the Secretary of the Treasury
April2006
FINANCIAL MANAGEMENT SYSTEMS
Lack of Disciplined Processes Puts Effective Implementation of Treasury's
Governmentwide Financial Report System at Risk
For the past 9 years, since the first audit of the consolidated financial
statements of the U.S. government (CFS), one of the major impediments to
our ability to render an opinion on the CFS is that the federal government
has not had adequate system, controls, and procedures to properly prepare
the CFS. To address some of the internal control weaknesses identified in
our audit report, Treasury began developing the Governmentwide Financial
Report System (GFRS). The goal of this new system is to directly link
information from federal agencies' audited financial statements to amounts
reported in the CFS, a concept that we strongly support. We reported
internal control weaknesses and GAO recommendations regarding the
preparation of the CFS, along with progress made in this area in a
separate report. This report provides our assessment of Treasury's ongoing
effort to develop and implement GFRS and makes recommendations for
reducing the risks associated with the development of GFRS.
What GAO Recommends
GAO is making three recommendations focused on reducing the risks
associated with the development of GFRS. Treasury stated that it concurs
with the recommendations in this report and is working to adopt them.
Treasury's development of GFRS is a positive initiative and Treasury has
made progress towards preparing the CFS based on agencies' audited
financial statements, which has been one of our principal concerns.
Treasury, though, has not yet effectively implemented the disciplined
system development processes necessary to provide reasonable assurance
that GFRS will meet all of its performance, schedule, and cost goals and
result in the most efficient and effective means of preparing the CFS.
Specifically, Treasury has not
o developed a concept of operations or any other document that
adequately defines or documents the expected performance of GFRS;
o developed a detailed project plan and schedule through
completion of GFRS;
o developed a budget justification for GFRS in its Capital Asset
Plan and Business Case (commonly referred to as the Exhibit 300),
as called for in Office of Management and Budget (OMB) Circular
No. A-11; and
o implemented the disciplined processes necessary to effectively
manage the GFRS project, which has contributed to usability
problems encountered by its users.
A disciplined software development and acquisition process can maximize
the likelihood of achieving the intended results (performance) within
established resources (costs) on schedule. Because of this relationship,
these factors can be viewed as an equilateral triangle as shown below.
Software Development Trade-Off Triangle
In our work at certain other federal agencies, we have found that project
deficiencies such as those we have identified with the GFRS project have
led to a range of problems, from increased cost and reduced functionality
to system failure. Going forward, it will be important that Treasury
better mitigate its risks so that long-standing internal control
weaknesses regarding the preparation of the CFS can be eliminated and,
more importantly, Treasury ends up with a system that fully meets its and
agencies' needs.
*** End of document. ***