Financial Management Systems: HHS Faces Many Challenges in
Implementing Its Unified Financial Management System (30-SEP-04,
GAO-04-1089T).
GAO has previously reported on systemic problems the federal
government faces in achieving the goals of financial management
reform and the importance of using disciplined processes for
implementing financial management systems. As a result, the
Subcommittee on Government Efficiency and Financial Management,
House Committee on Government Reform, asked GAO to review and
evaluate the agencies' plans and ongoing efforts for implementing
financial management systems. The results of GAO's review of the
Department of Health and Human Services' (HHS) ongoing effort to
develop and implement the Unified Financial Management System
(UFMS) are discussed in detail in the report Financial Management
Systems: Lack of Disciplined Processes Puts Implementation of
HHS' Financial System at Risk (GAO-04-1008).
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-04-1089T
ACCNO: A12856
TITLE: Financial Management Systems: HHS Faces Many Challenges
in Implementing Its Unified Financial Management System
DATE: 09/30/2004
SUBJECT: Financial management
Information technology
Internal controls
Risk management
Taxpayers
HHS Unified Financial Management System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-04-1089T
United States Government Accountability Office
GAO Testimony
Before the Subcommittee on Government Efficiency and Financial Management,
Committee on Government Reform, House of Representatives
For Release on Delivery Expected at 2:00 p.m. EDT Thursday, September 30,
2004
FINANCIAL MANAGEMENT SYSTEMS
HHS Faces Many Challenges in Implementing Its Unified Financial Management
System
Statement of Jeffrey C. Steinhoff
Managing Director, Financial Management and Assurance
Keith A. Rhodes
Chief Technologist, Applied Research and Methodology
Center for Engineering and Technology
a
GAO-04-1089T
Highlights of GAO-04-1089T, a testimony before the Subcommittee on
Government Efficiency and Financial Management, Committee on Government
Reform, House of Representatives.
GAO has previously reported on systemic problems the federal government
faces in achieving the goals of financial management reform and the
importance of using disciplined processes for implementing financial
management systems. As a result, the Subcommittee on Government Efficiency
and Financial Management, House Committee on Government Reform, asked GAO
to review and evaluate the agencies' plans and ongoing efforts for
implementing financial management systems.
The results of GAO's review of the Department of Health and Human
Services' (HHS) ongoing effort to develop and implement the Unified
Financial Management System (UFMS) are discussed in detail in the report
Financial Management Systems: Lack of Disciplined Processes Puts
Implementation of HHS' Financial System at Risk
(GAO-04-1008). In this report, GAO makes 34 recommendations focused on
mitigating risks associated with the project. In light of this report, the
Subcommittee asked GAO to testify on the challenges HHS faces in
implementing UFMS.
www.gao.gov/cgi-bin/getrpt?GAO-04-1089T.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Jeffrey Steinhoff (202)
512-2600, [email protected] or Keith Rhodes (202) 512-6412,
[email protected].
September 30, 2004
FINANCIAL MANAGEMENT SYSTEMS
HHS Faces Many Challenges in Implementing Its Unified Financial Management
System
HHS had not effectively implemented several disciplined processes, which
are accepted best practices in systems development and implementation, and
had adopted other practices, that put the project at unnecessary risk.
Although the implementation of any major system is not a risk-free
proposition, organizations that follow and effectively implement
disciplined processes can reduce these risks to acceptable levels. While
GAO recognized that HHS had adopted some best practices related to senior
level support, oversight, and phased implementation, GAO noted that HHS
had focused on meeting its schedule to the detriment of disciplined
processes.
GAO found that HHS had not effectively implemented several disciplined
processes to reduce risks to acceptable levels, including
o requirements management,
o testing,
o project management and oversight using quantitative measures, and
o risk management.
Compounding these problems are departmentwide weaknesses in information
technology management processes needed to provide UFMS with a solid
foundation for development and operation, including
o investment management,
o enterprise architecture, and
o information security.
GAO also identified human capital issues that significantly increase the
risk that UFMS will not fully meet one or more of its cost, schedule, and
performance objectives, including
o staffing and
o strategic workforce planning.
HHS stated that it had an aggressive implementation schedule, but
disagreed that a lack of disciplined processes is placing the UFMS program
at risk. GAO firmly believes if HHS continues to follow an approach that
is schedule-driven and shortcuts key disciplined processes, it is
unnecessarily increasing its risk. GAO stands by its position that
adherence to disciplined processes is crucial, particularly with a project
of this magnitude and importance.
HHS indicated that it plans to delay deployment of significant
functionality associated with its UFMS project for at least 6 months. This
decision gives HHS a good opportunity to effectively implement disciplined
processes to enhance the project's opportunity for success.
Mr. Chairman and Members of the Subcommittee:
We are pleased to be here today to discuss the efforts by the Department
of Health and Human Services (HHS) to develop and implement its Unified
Financial Management System (UFMS). We would like to thank the
Subcommittee for having this hearing. Hearings such as this one today
foster meaningful financial management reform. Our work focused on whether
the UFMS project was being managed in a way that best ensures long-term
success of this important project. At the time of our review, the complete
implementation of UFMS was targeted for 2007 and the estimated total
project cost was over $700 million.1 Not only must the system ultimately
replace 5 accounting systems, but it must also interface with about 110
other systems. By any measure, this is a major undertaking that brings
with it a degree of risk. Risk, though, can be managed and reduced to
acceptable levels through the use of disciplined processes, which in
short, represent best practices in system development and implementation
that have proven their value in the past.
Our report,2 which was prepared at the request of the Subcommittee and is
being released at this hearing, discusses in detail the issues we
identified with the UFMS project and includes 34 recommendations that
focus on mitigating project risk. Our testimony today3 will (1) highlight
the importance of adhering to disciplined processes for a system
development and implementation effort such as UFMS, (2) summarize our
findings on HHS' management of the UFMS project, and (3) provide our
perspective on actions needed for HHS to mitigate the risk to this project
and move forward.
1The costs for this financial management system improvement effort can be
broken down into four broad areas: (1) National Institutes of Health
(NIH); (2) Centers for Medicare and Medicaid Services (CMS); (3) all other
HHS entities including the Centers for Disease Control and Prevention
(CDC); and (4) a system to consolidate the results of HHS' financial
management operations. HHS estimated that it would spend about $110
million for NIH, $393 million for CMS, and $210 million for the remaining
HHS organizations. HHS has not yet developed an estimate of the costs
associated with integrating these efforts into a unified financial
management system.
2GAO, Financial Management Systems: Lack of Disciplined Processes Puts
Implementation of HHS' Financial System at Risk, GAO-04-1008 (Washington,
D.C.: Sept. 23, 2004).
3This testimony is based on our report and does not assess HHS' other
financial management improvement efforts at the National Institutes of
Health (NIH) and Centers for Medicare and Medicaid Services (CMS).
Disciplined Processes Are Key to Successful System Implementation
The ability to produce the information needed to efficiently and
effectively manage the day-to-day operations of the federal government and
provide accountability to taxpayers and the Congress has been a
long-standing challenge for federal agencies. To help address this
challenge, many agencies are in the process of replacing their core
financial systems as part of their financial management system improvement
efforts. Although the implementation of any major system is not a
risk-free proposition, organizations that follow and effectively implement
disciplined processes can reduce these risks to acceptable levels. The use
of the term acceptable levels acknowledges the fact that any systems
acquisition has risks and will suffer the adverse consequences associated
with defects. However, effective implementation of the disciplined
processes reduces the potential for risks to occur and helps prevent those
that do occur from having any significant adverse impact on the cost,
timeliness, and performance of the project. A disciplined software
development and acquisition process can maximize the likelihood of
achieving the intended results (performance) within established resources
(costs) on schedule.
Although there is no standard set of practices that will ever guarantee
success, several organizations, such as the Software Engineering Institute
(SEI)4 and the Institute of Electrical and Electronic Engineers (IEEE),5
as well as individual experts have identified and developed the types of
policies, procedures, and practices that have been demonstrated to reduce
development time and enhance effectiveness. The key to having a
disciplined system development effort is to have disciplined processes in
multiple areas, including project planning and management, requirements
management, configuration management, risk management, quality assurance,
and testing. Effective processes should be implemented in each of these
areas throughout the project life cycle because change is constant.
Effectively implementing the disciplined processes necessary to reduce
project risks to acceptable levels is hard to achieve because a project
must effectively implement several best practices, and inadequate
4SEI is a federally funded research and development center operated by
Carnegie Mellon University and sponsored by the U.S. Department of
Defense. The SEI objectives are to provide leadership in software
engineering and in the transition of new software engineering technologies
into practice.
5IEEE develops standards for a broad range of global industries including
the information technology and information assurance industries.
implementation of any one practice may significantly reduce or even
eliminate the positive benefits of the others.
Successfully acquiring and implementing a new financial management system
requires a process that starts with a clear definition of the
organization's mission and strategic objectives and ends with a system
that meets specific information needs. We have seen many system efforts
fail because agencies started with a general need, such as improving
financial management, but did not define in precise terms (1) the specific
problems they were trying to solve, (2) what their operational needs were,
and (3) what specific information requirements flowed from these
operational needs. Instead, they plunged into the acquisition and
implementation process in the belief that these specifics would somehow be
defined along the way. The typical result was that systems were delivered
well past anticipated milestones; failed to perform as expected; and,
accordingly, were overbudget because of required costly modifications.
Undisciplined projects typically show a great deal of productive work at
the beginning of the project, but the rework associated with defects
begins to consume more and more resources.6 In response, processes are
adopted in the hopes of managing what later turns out to have been
unproductive work. Generally, these processes are "too little, too late"
because sufficient foundations for building the systems were not
established or not established adequately. Experience has shown that
projects for which disciplined processes are not implemented at the
beginning are forced to implement them later when it takes more time and
the processes are less effective.7
A major consumer of project resources in undisciplined efforts is rework
(also known as thrashing). Rework occurs when the original work has
defects or is no longer needed because of changes in project direction.
Disciplined organizations focus their efforts on reducing the amount of
rework because it is expensive. Experts have reported that fixing a defect
during the testing phase costs anywhere from 10 to 100 times the cost of
fixing it during the design or requirements phase.8 Projects that are
unable
6Steve McConnell, Rapid Development: Taming Wild Software Schedules
(Redmond, Wash.: Microsoft Press, 1996).
7McConnell, Rapid Development: Taming Wild Software Schedules.
8McConnell, Rapid Development: Taming Wild Software Schedules.
to successfully address their rework will eventually only be spending
their time on rework and the associated processes rather than on
productive work. In other words, the project will continually find itself
reworking items.
HHS Had Not Effectively Implemented Disciplined Processes, Information
Technology Management Practices, and Human Capital Planning
We found that HHS had adopted some best practices in its development of
UFMS. The project had support from senior officials and oversight by
independent experts, commonly called independent verification and
validation (IV&V) contractors. We also view HHS' decision to follow a
phased implementation to be a sound approach.
However, at the time of our review, HHS had not effectively implemented
several disciplined processes essential to reducing risks to acceptable
levels and therefore key to a project's success, and had adopted other
practices that put the project at unnecessary risk. HHS officials told us
that they had carefully considered the risks associated with implementing
UFMS and that they had put in place strategies to manage these risks and
to allow the project to meet its schedule within budget. However, we found
that HHS had focused on meeting its schedule to implement the first phase
of the new system at the Centers for Disease Control and Prevention (CDC)
in October 2004, to the detriment of disciplined processes and thus had
introduced unnecessary risks that may compromise the system's cost,
schedule, and performance. We would now like to briefly highlight a few of
the key disciplined processes that HHS had not fully implemented at the
time of our review. These matters are discussed in detail in our report.
o Requirements management. Requirements are the specifications that
system developers and program managers use to design, develop, and acquire
a system. Requirements management deficiencies have historically been a
root cause of systems that do not meet their cost, schedule, and
performance objectives. Effective requirements management practices are
essential for ensuring the intended functionality will be included in the
system and are the foundation for testing. We found significant problems
in HHS' requirements management process and that HHS had not developed
requirements that were clear and unambiguous.
o Testing. Testing is the process of executing a program with the intent
of finding errors.9 Without adequate testing, an organization (1) is
taking a significant risk that substantial defects will not be detected
until after the system is implemented and (2) does not have reasonable
assurance that new or modified systems will function as planned. We found
that HHS faced challenges in implementing a disciplined testing program,
because, first of all, it did not have an effective requirements
management system that produced clear, unambiguous requirements upon which
to build its testing efforts. In addition, HHS had scheduled its testing
activities, including those for converting data from existing systems to
UFMS, late in the implementation cycle leaving little time to correct
defects identified before the scheduled deployment in October 2004.
o Project management and oversight using quantitative measures. We found
that HHS did not have quantitative metrics that allowed it to fully
understand (1) its capability to manage the entire UFMS effort; (2) how
problems in its management processes would affect the UFMS cost, schedule,
and performance objectives; and (3) the corrective actions needed to
reduce the risks associated with the problems identified with its
processes. Such quantitative measures are essential for adequate project
management oversight. Without such information, HHS management can only
focus on the project schedule and whether activities have occurred as
planned, not on whether the substance of the activities achieved their
system development objectives. As we note in our report, this is not an
effective practice.
o Risk management. We noted that HHS routinely closed its identified
risks on the premise that they were being addressed. Risk management is a
continuous process to identify, monitor, and mitigate risks to ensure that
the risks are being properly controlled and that new risks are identified
and resolved as early as possible. An effective risk management process is
designed to mitigate the effects of undesirable events at the earliest
possible stage to avoid costly consequences.
In addition, HHS' effectiveness in managing the processes associated with
its data conversion and UFMS interfaces will be critical to the success of
this project. For example, CDC's ability to convert data from its existing
systems to the new system will be crucial to helping ensure that UFMS will
9Glenford J. Myers, The Art of Software Testing (John Wiley & Sons, Inc.,
1979).
provide the kind of data needed to manage CDC's programs and operations.
The adage "garbage in garbage out" best describes the adverse impact.
Furthermore, HHS expects that once UFMS is fully deployed, the system will
need to interface with about 110 other systems, of which about 30 system
interfaces are needed for the CDC deployment. Proper implementation of the
interfaces between UFMS and the other systems it receives data from and
sends data to is essential for providing data integrity and ensuring that
UFMS will operate as it should and provide the information needed to help
manage its programs and operations.
Compounding these UFMS-specific problems are departmentwide weaknesses we
have previously reported in information technology (IT) investment
management,10 enterprise architecture,11 and information security.12
Specifically, HHS had not established the IT management processes needed
to provide UFMS with a solid foundation for development and operation.
Such IT weaknesses increase the risk that UFMS will not achieve planned
results within the estimated budget and schedule. We will now highlight
the IT management weaknesses that HHS must overcome:
o Investment management. HHS had weaknesses in the processes it uses to
select and control its IT investments. Among the weaknesses we previously
identified, HHS had not (1) established procedures for the development,
documentation, and review of IT investments by its review boards or (2)
documented policies and procedures for aligning and coordinating
investment decision making among its investment management boards. Until
HHS addresses weaknesses in its selection or control processes, IT
projects like UFMS will face an increased likelihood that the projects
will not be completed on schedule and within estimated costs.
o Enterprise architecture. While HHS is making progress in developing an
enterprise architecture that incorporates UFMS as a central component,
10GAO, Information Technology Management: Governmentwide Strategic
Planning, Performance Measurement, and Investment Management Can Be
Further Improved, GAO-04-49 (Washington, D.C.: Jan. 12, 2004).
11GAO, Information Technology: Leadership Remains Key to Agencies Making
Progress on Enterprise Architecture Efforts, GAO-04-40 (Washington, D.C.:
Nov. 17, 2003).
12GAO, Information Security: Agencies Need to Implement Consistent
Processes in Authorizing Systems for Operation, GAO-04-376 (Washington,
D.C.: June 28, 2004).
most of the planning and development of the UFMS IT investment had
occurred without the guidance of an established enterprise architecture.
An enterprise architecture is an organizational blueprint that defines how
an entity operates today and how it intends to operate in the future and
invest in technology to transition to this future state. Our experience
with other federal agencies has shown that projects developed without the
constraints of an established enterprise architecture are at risk of being
duplicative, not well integrated, unnecessarily costly to maintain and
interface, and ineffective in supporting missions.
o Information security. HHS had not yet fully implemented the key
elements of a comprehensive security management program and had
significant and pervasive weaknesses in its information security controls.
The primary objectives of information security controls are to safeguard
data, protect computer application programs, prevent unauthorized access
to system software, and ensure continued operations. Without adequate
security controls, UFMS cannot provide reasonable assurance that the
system is protected from loss due to errors, fraud and other illegal acts,
disasters, and incidents that cause systems to be unavailable.
Finally, we believe it is essential that an agency take the necessary
steps to ensure that it has the human capital capacity to design,
implement, and operate a financial management system. We found that staff
shortages and limited strategic workforce planning have resulted in the
project not having the resources needed to effectively design, implement,
and operate UFMS. We identified the following weaknesses:
o Staffing. HHS had not filled positions in the UFMS Program Management
Office that were identified as needed. Proper human capital planning
includes identifying the workforce size, skills mix, and deployment needed
for mission accomplishment and to create strategies to fill the gaps.
Scarce resources could significantly jeopardize the project's success and
have led to several key UFMS deliverables being significantly behind
schedule.
o Strategic workforce planning. HHS had not yet fully developed key
workforce planning tools, such as the CDC skills gap analysis, to help
transform its workforce so that it can effectively use UFMS. Strategic
workforce planning focuses on developing long-term strategies for
acquiring, developing, and retaining an organization's total workforce
(including full- and part-time federal staff and contractors) to meet the
needs of the future. Strategic workforce planning is essential for
achieving the mission and goals of the UFMS project. By not identifying
staff with the requisite skills to operate such a system and by not
identifying gaps in needed skills and filling them, HHS has not optimized
its chances for the successful implementation and operation of UFMS.
Action Is Needed to Mitigate Risk
To address the range of problems we have just highlighted, our report
includes 34 recommendations that focus on mitigating the risks associated
with this project. We made 8 recommendations related to the initial
deployment of UFMS at CDC that are specifically tied to implementing
critical disciplined processes. In addition, we recommended that until
these 8 recommendations are substantially addressed, HHS delay the initial
deployment. The remaining 25 recommendations were centered on developing
an appropriate foundation for moving forward and focused on (1)
disciplined processes, (2) IT security controls, and (3) human capital
issues.
In its September 7, 2004, response to a draft of our report, HHS disagreed
regarding management of the project and whether disciplined processes were
being followed. In its comments, HHS characterized the risk in its
approach as the result, not of a lack of disciplined processes, but of an
aggressive project schedule. From our perspective, this project
demonstrated the classic symptoms of a schedule-driven effort for which
key processes had been omitted or shortcutted, thereby unnecessarily
increasing risk. As we mentioned at the outset of our testimony, this is a
multiyear project with an estimated completion date in fiscal year 2007
and a total estimated cost of over $700 million.13 With a project of this
magnitude and importance, we stand by our position that it is crucial for
the project to adhere to disciplined processes that represent best
practices. Therefore, in order to mitigate its risk to an acceptable
level, we continue to believe it is essential for HHS to adopt and
effectively implement our 34 recommendations.
In commenting on our draft report, HHS also indicated that actions had
either been taken, were under way, or were planned that address a number
of our recommendations. In addition, HHS subsequently contacted us on
13This includes the eventual incorporation of CMS and NIH.
September 23, 2004, to let us know that it had decided to delay the
implementation of a significant amount of functionality associated with
the CDC deployment from October 2004 until April 2005 in order to address
the issues that had been identified with the project. HHS also provided us
with copies of IV&V reports and other documentation that had been
developed since our review. Delaying implementation of significant
functionality at CDC is a positive step forward given the risks associated
with the project. This delay, by itself, will not reduce the risk to an
acceptable level, but will give HHS a chance to implement the disciplined
processes needed to do so.
HHS will face a number of challenges in the upcoming 6 months to address
the weaknesses in its management of the project that were discussed in our
report. At a high level, the key challenge will be to implement an event
driven project based on effectively implemented disciplined processes,
rather than a schedule-driven project. It will be critical as well to
address the problems noted in the IV&V reports that were issued during and
subsequent to our review. If the past is prologue, taking the time to
adhere to disciplined processes will pay dividends in the long term.
Mr. Chairman, this concludes our statement. We would be pleased to answer
any questions you or other members of the Subcommittee may have at this
time.
Contacts and Acknowledgments
(193057)
For further information about this statement, please contact Jeffrey C.
Steinhoff, Managing Director, Financial Management and Assurance, who may
be reached at (202) 512-2600 or by e-mail at [email protected], or Keith
A. Rhodes, Chief Technologist, Applied Research and Methodology Center for
Engineering and Technology, who may be reached at (202) 5126412 or by
e-mail at [email protected]. Other key contributors to this testimony
include Kay Daly, Michael LaForge, Chris Martin, and Mel Mench.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
GAO's Mission The Government Accountability Office, the audit, evaluation
and investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of The fastest and easiest way to obtain copies of GAO
documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO postsGAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To Testimony have GAO e-mail you a list of newly posted products
every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected] Programs Automated answering system: (800)
424-5454 or (202) 512-7470
Congressional Gloria Jarmon, Managing Director, [email protected] (202)
512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125
Relations Washington, D.C. 20548
Public Affairs Jeff Nelligan, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***