Homeland Security: Departmentwide Integrated Financial Management
Systems Remain a Challenge (21-JUN-07, GAO-07-536).		 
                                                                 
Since the Department of Homeland Security (DHS) began operations 
in March 2003, it has faced the daunting task of bringing	 
together 22 diverse agencies and developing an integrated	 
financial management system to provide timely, reliable, and	 
useful financial information. GAO was asked to determine (1)	 
whether DHS has fully developed plans for implementing and/or	 
migrating to an integrated departmentwide financial management	 
system, (2) the potential usefulness of the work products	 
received for the funds spent on the financial modernization	 
effort, and (3) going forward, how DHS can incorporate best	 
practices into its plans for migrating to an integrated 	 
departmentwide financial management system. GAO interviewed key  
DHS officials, reviewed relevant DHS policy and procedure	 
documents, and analyzed work products related to the financial	 
modernization effort.						 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-07-536 					        
    ACCNO:   A71535						        
  TITLE:     Homeland Security: Departmentwide Integrated Financial   
Management Systems Remain a Challenge				 
     DATE:   06/21/2007 
  SUBJECT:   Best practices					 
	     Concept of operations				 
	     Federal agency reorganization			 
	     Financial management				 
	     Financial management systems			 
	     Homeland security					 
	     Human capital management				 
	     Internal controls					 
	     Requirements definition				 
	     Risk management					 
	     Standards						 
	     Strategic planning 				 
	     Systems conversions				 
	     Program implementation				 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-07-536

   

     * [1]Results in Brief
     * [2]Background
     * [3]DHS Lacks a Fully Developed Financial Management Strategy an
     * [4]eMerge2 Costs Are Unknown and Work Products Have Limited Use

          * [5]Actual Costs of eMerge2 Are Unknown
          * [6]eMerge2 Work Products Have Limited Future Usefulness

               * [7]The Concept of Operations Document Is Flawed
               * [8]eMerge2 System Requirements Are Deficient

     * [9]Four Key Building Blocks and Effective Human Capital Managem

          * [10]Concept of Operations Provides Foundation
          * [11]Standard Business Processes Promote Consistency
          * [12]Strategy for Consolidating and Migrating Financial Managemen
          * [13]Disciplined Processes Will Help Ensure Successful Implementa
          * [14]Strong Human Capital Management Needed at DHS

     * [15]Conclusions
     * [16]Recommendations for Executive Action
     * [17]Agency Comments and Our Evaluation
     * [18]Disciplined Processes Are Key to Successful Financial Manage

          * [19]Requirements Management
          * [20]Testing
          * [21]Data Conversion and System Interfaces
          * [22]Configuration Management
          * [23]Risk Management
          * [24]Project Management
          * [25]Quality Assurance

     * [26]GAO Contacts
     * [27]Acknowledgments
     * [28]GAO's Mission
     * [29]Obtaining Copies of GAO Reports and Testimony

          * [30]Order by Mail or Phone

     * [31]To Report Fraud, Waste, and Abuse in Federal Programs
     * [32]Congressional Relations
     * [33]Public Affairs

Report to the Subcommittee on Federal Financial Management, Government
Information, Federal Services, and International Security, Committee on
Homeland Security and Governmental Affairs, U.S. Senate

United States Government Accountability Office

GAO

June 2007

HOMELAND SECURITY

Departmentwide Integrated Financial Management Systems Remain a Challenge

GAO-07-536

Contents

Letter 1

Results in Brief 3
Background 6
DHS Lacks a Fully Developed Financial Management Strategy and Plan 9
eMerge2 Costs Are Unknown and Work Products Have Limited Usefulness 12
Four Key Building Blocks and Effective Human Capital Management Must Drive
DHS's Financial Management Transformation Efforts 21
Conclusions 26
Recommendations for Executive Action 27
Agency Comments and Our Evaluation 28
Appendix I Scope and Methodology 30
Appendix II Material Weaknesses/Reportable Conditions at DHS for Fiscal
Years 2003 through 2006 32
Appendix III Key Questions for the Department of Homeland Security to
Consider Based on the Four Building Blocks 34
Appendix IV Disciplined Processes 35
Appendix V Comments from the Department of Homeland Security 47
Appendix VI GAO Contacts and Staff Acknowledgments 53
Related GAO Products 54

Figures

Figure 1: eMerge2 Project Timeline 15
Figure 2: DHS Systems Inventory 17
Figure 3: Relationship between Requirements Development and Testing 38

Abbreviations

AICPA American Institute of Certified Public Accountants
BPMN business process modeling notation
CBP U.S. Customs and Border Protection
CFO Chief Financial Officer
COTS commercial-off-the-shelf
CRP conference room pilot
DHS Department of Homeland Security
DOT Department of Transportation
eMerge2 Electronically Managing Enterprise Resources for Government
  Effectiveness and Efficiency
EPR Emergency Preparedness and Response
ERP enterprise resource planning
FEMA Federal Emergency Management Agency
FFMIA Federal Financial Management Improvement Act of 1996
FLETC Federal Law Enforcement Training Center
ICE U.S. Immigration and Customs Enforcement
ICOFR Internal Control Over Financial Reporting
IEEE Institute of Electrical and Electronics Engineers
INS U.S. Immigration and Naturalization Service
IT information technology
OCFO Office of the Chief Financial Officer
OFM Office of Financial Management
OGC Office of the General Counsel
OMB Office of Management and Budget
OM&S operating materials and supplies
PP&E property, plant, and equipment
RMTO Resource Management Transformation Office
SEI Software Engineering Institute
TSA Transportation Security Administration
UDO undelivered order

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

United States Government Accountability Office
Washington, DC 20548

June 21, 2007

The Honorable Tom Carper
Chairman
The Honorable Tom Coburn, M.D.
Ranking Member
Subcommittee on Federal Financial Management, Government
  Information, Federal Services, and International Security
Committee on Homeland Security and Governmental Affairs
United States Senate

Since the Department of Homeland Security (DHS) began operations in March
2003, as mandated by the Homeland Security Act of 2002,1 it has faced the
daunting task of bringing together 22 diverse agencies and developing an
integrated financial management system. Since 2003, we have designated
implementing and transforming DHS as high risk2 because the agency has yet
to implement a corrective action plan that includes a comprehensive
transformation strategy, and because its management systems--especially
related to financial, information, acquisition, and human capital
management--are not yet integrated and wholly operational. DHS inherited
many financial management weaknesses and vulnerabilities from 22 agencies.
Auditors had identified 30 reportable
conditions,3 18 of which were considered material internal control
weaknesses4 in fiscal year 2003.

1Pub. L. No. 107-296, 116 Stat. 2135 (Nov. 25, 2002).

2GAO, High-Risk Series: An Update, [34]GAO-07-310 (Washington, D.C.:
January 2007).

DHS began implementation of the Electronically Managing Enterprise
Resources for Government Effectiveness and Efficiency (eMerge2) program in
January 2004 to integrate financial management systems across the entire
department and to address the financial management weaknesses. eMerge2 was
expected to establish the strategic direction for migration,
modernization, and integration of DHS financial, accounting, procurement,
personnel, asset management, and travel systems, processes, and policies.
DHS officials have stated that approximately $52 million in total was
spent on the eMerge2 project before it was halted in December 2005. DHS
officials are considering other options to provide integrated financial
management systems and are assessing the capabilities of financial
management systems at various internal components. In March 2006, we
reported5 that DHS was at an important crossroads in implementing a
financial management system, and we discussed the necessary building
blocks that form the foundation for successful financial management system
implementation efforts. As DHS moves forward, periodic independent updates
on the status of financial management modernization that aligns with a
comprehensive transformation strategy are important to help key
congressional leaders and DHS management provide effective oversight.
Moreover, DHS must be able to provide reliable, useful, and timely
financial management information, so that DHS leadership and the Congress
are well positioned to make fully informed decisions to secure America's
homeland.

3Under standards issued by the American Institute of Certified Public
Accountants (AICPA), "reportable conditions" are matters coming to the
auditors' attention relating to significant deficiencies in the design or
operation of internal controls that, in the auditors' judgment, could
adversely affect the department's ability to record, process, summarize,
and report financial data consistent with the assertions of management in
the financial statements. The AICPA recently revised its guidance for
audits of financial statements beginning on or after December 15, 2006;
the term "reportable condition" has been replaced by "significant
deficiency."

4A material weakness was previously defined as a reportable condition in
which the design or operation of one or more of the internal control
components does not reduce to a relatively low level the risk that
misstatements caused by error or fraud in amounts that would be material
in relation to the financial statements being audited may occur and not be
detected within a timely period by employees in the normal course of
performing their assigned functions. The new definition of a material
weakness is a significant deficiency, or combination of significant
deficiencies, that results in more than a remote likelihood that a
material misstatement of the financial statements will not be prevented or
detected.

5GAO, Financial Management Systems: DHS Has an Opportunity to Incorporate
Best Practices in Modernization Efforts, [35]GAO-06-553T (Washington,
D.C.: Mar. 29, 2006).

You asked us to establish baseline information on DHS's financial
management system modernization and to periodically update the status of
these efforts. This report, our first in response to your request,
provides an assessment of the status of DHS's efforts to modernize its
financial management systems. Because of your concern about DHS's
successful implementation of an integrated financial management system,
you also asked us to determine (1) whether DHS has fully developed plans
for implementing and/or migrating to an integrated departmentwide
financial management system, (2) the potential usefulness of the work
products received for the funds spent on eMerge2, and (3) going forward,
how DHS can incorporate key building blocks and human capital best
practices into its plans for implementing and/or migrating to an
integrated departmentwide financial management system.

This report incorporates lessons learned and best practices from our prior
work that focused on federal government financial management system
implementation efforts. We interviewed key DHS officials and reviewed
their existing policies and procedures related to financial management
systems. We analyzed and reviewed eMerge2 work products as well as related
current financial management initiatives under way. Our work on this
report was performed in Washington, D.C., from September 2006 through
April 2007 in accordance with generally accepted government auditing
standards. Details on our scope and methodology are included in appendix
I. Related GAO reports are listed at the end of this report.

Results in Brief

While DHS officials have recognized the need for an integrated financial
management system, no financial strategy or integrated financial
management systems effort that includes financial management policies and
procedures, standard business processes, a human capital strategy, and
effective internal controls has been developed. Moreover, DHS has
experienced significant turnover in leadership, has yet to address the
root causes of existing financial management problems, and still lacks a
financial management strategy that includes a formal strategic financial
management plan to implement or migrate to an integrated system.

In early March 2007, DHS officials issued a high-level plan with a stated
purpose of addressing the existing internal control weaknesses. While a
positive step, the plan has a policy and process focus and does not
comprise a strategy for financial systems modernization. More detailed
implementation strategies will be necessary to fully address the financial
management system integration efforts. DHS recognizes that there is an
urgent need for an integrated financial management system, and told us
that after assessing the capabilities of existing financial management
systems at several of its components, it has decided to consolidate its
financial management systems. In commenting on a draft of this report, DHS
indicated that it plans to leverage its current investments by migrating
components to internal service providers using the financial management
systems models currently in place at either the Transportation Security
Administration (TSA)6 or U.S. Customs and Border Protection (CBP). Our
concern is that these components have numerous financial management
weaknesses. For example, the financial statement auditors for TSA
reported7 that the agency was unable to provide sufficient evidential
matter or make knowledgeable representations to support fiscal year 2005
and 2006 transactions and account balances, particularly for budgetary
accounting; undelivered orders; and property, plant, and equipment, among
others.

According to DHS officials, several of the work products developed for
eMerge2 will be useful as they move forward with their financial
management modernization efforts, regardless of the strategic financial
management direction ultimately selected by DHS. However, our review
indicated that the usefulness of many of the eMerge2 work products is
questionable. The work products developed include a core set of financial
management system requirements and various other qualitative financial
management plans, including a concept of operations document. Our review
of the core set of financial management requirements and concept of
operations developed for the eMerge2 project found that DHS had not fully
incorporated best practices in this effort, and therefore it is not
surprising that the results were significantly flawed and the work
products were not very useful. For example, the concept of operations
document lacked critical elements called for by the Institute of
Electrical and Electronics Engineers, Inc. (IEEE) standards, 8 such as
providing a detailed description of the existing system(s) that DHS
planned to replace. In addition, our review of key eMerge2 requirements
identified requirements that were unclear and incomplete when compared
with the attributes called for in the IEEE standards.9 DHS has little to
show for the $18 million in contractor costs and $52 million overall it
reported to us that it spent on eMerge2. DHS did not to provide
documentation to support these reported costs. DHS's decision to end the
project before spending an estimated $229 million on a financial
management system that would not provide the expected system functionality
and desired performance was prudent, and we support the decision to cut
its losses. However, the agency has made little progress since that time
and has missed an invaluable opportunity to address existing financial
management problems.

6The U.S. Coast Guard operates TSA's financial management system.

7Department of Homeland Security, Performance and Accountability Report
Fiscal Year 2006 (Washington, D.C.: November 2006).

8IEEE Guide for Information Technology - System Definition - Concept of
Operations Document, Std.1362-1998. The IEEE is a nonprofit, technical
professional association that develops standards for a broad range of
global industries, including the information technology and information
assurance industries, and is a leading source for defining best practices.

As we previously reported,10 consolidation of an entity as large and
diverse as DHS poses significant management challenges, including
integrating a myriad of redundant financial management systems and
addressing existing and newly identified weaknesses in the inherited
components. The federal government has long been plagued by financial
management system modernization efforts that have failed to meet their
cost, schedule, and performance goals. In order for DHS to avoid these
long-standing problems that have plagued financial management system
improvement efforts and avoid repeating the mistakes it made with eMerge2,
it must adopt solutions that reduce the risks associated with these
efforts to acceptable levels. In our March 2006 testimony,11 we identified
four key concepts that will be critical to DHS's ability to successfully
complete the implementation of an integrated financial management system
or migration to shared service providers. Careful consideration of these
concepts, each one building upon the next, will be integral to the success
of DHS's strategy. The four building blocks are (1) developing a concept
of operations, (2) defining standard business processes, (3) developing an
implementation or migration strategy for DHS components, and (4) defining
and effectively implementing disciplined processes necessary to properly
manage the specific projects. Effective human capital management, such as
strategic workforce planning and change management, is also identified as
critical to successfully implementing a new financial management system.
DHS officials recognize the importance of having sufficient staff on board
to execute a financial management strategy, but because DHS does not
currently have a financial management system project in place, it has not
yet developed human capital plans and activities. As DHS develops a
financial management plan or strategy, careful consideration of key human
capital practices will be a critical success factor.

9IEEE Recommended Practice for Software Requirements Specifications, Std.
830-1998. This recommended practice is aimed at specifying requirements of
software to be developed but also can be applied to assist in the
selection of in-house and commercial software products.

10GAO, Financial Management: Department of Homeland Security Faces
Significant Financial Management Challenges, [36]GAO-04-774 (Washington,
D.C.: July 19, 2004).

11 [37]GAO-06-553T .

We are making six recommendations focused on the need for DHS to develop a
financial management plan or strategy and to fully adopt the building
blocks and human capital practices that are vital to minimizing the risk
related to modernizing its financial management systems. In written
comments on a draft of this report, DHS concurred with our recommendations
and described the approach and steps that are planned to improve DHS's
financial management systems. DHS's comments are discussed in the Agency
Comments and Our Evaluation section and reprinted in appendix V. DHS also
provided several technical comments, which we incorporated as appropriate.

Background

When DHS was created in March 2003 and merged 22 diverse agencies, there
were many known financial management weaknesses and vulnerabilities in the
inherited agencies. For 5 of the agencies that transferred to DHS--Customs
Service (Customs),12 TSA, Immigration and Naturalization Service (INS),13
Federal Emergency Management Agency (FEMA), and Federal Law Enforcement
Training Center (FLETC)--auditors had identified 30 reportable conditions,
18 of which were considered material internal control weaknesses. Further,
of the four component agencies--Customs, TSA, INS, and FEMA--that had
previously been subject to stand-alone financial statement audits, all
four agencies' systems were found not to be in substantial compliance with
the requirements of the Federal Financial Management Improvement Act of
1996 (FFMIA).14

12The Bureau of Customs and Border Protection is now the U.S. Customs and
Border Protection component of DHS.

13Bureau of Immigration and Customs Enforcement is now the U.S.
Immigration and Customs Enforcement component of DHS.

Most of the 22 components that transferred to DHS had not been subjected
to significant financial statement audit scrutiny prior to their transfer,
so the extent to which additional significant internal control
deficiencies existed was unknown. For example, conditions at the Coast
Guard surfaced because of its greater relative size and increased audit
scrutiny at DHS as compared to its former legacy agency, the Department of
Transportation (DOT). As part of DOT's financial statement audit, the
Coast Guard had no specifically attributable reported weaknesses
identified. However, identified weaknesses related to the Coast Guard were
one of the main reasons that the independent auditors were unable to
provide an opinion on DHS's consolidated balance sheets as of September
30, 2006 and 2005. The auditors identified numerous material weaknesses
related to fund balance with treasury; property, plant, and equipment; and
budgetary accounting. Moreover, the auditors reported that the Coast Guard
did not have an organizational structure that fully supported the
development and implementation of effective policies, procedures, and
internal controls. The Coast Guard's personnel rotation policy, among
other issues, made it difficult for the Coast Guard's Chief Financial
Officer to institutionalize internal controls related to financial
management and reporting.

As noted above, material internal control weaknesses have been an ongoing
problem at DHS since its inception, and these material internal control
weaknesses and financial reporting problems continued in fiscal year 2006.
We previously reported15 that for fiscal year 2003, the DHS financial
statement auditors reported 14 total reportable conditions, 7 of which
were considered to be material weaknesses. In fiscal year 2006, while the
total number of reportable conditions decreased to 12, the number of
reportable conditions considered to be material weaknesses increased to
10. A description of the material weaknesses as identified by the auditors
in fiscal years 2003 through 2006 can be found in appendix II. Some of the
more recent material weaknesses identified by the auditors include
problems with fund balance with treasury, budgetary accounting, and
intergovernmental balances.

14Federal Financial Management Improvement Act of 1996, Pub. L. No.
104-208, div. A, S 101(f), title VIII, 110 Stat. 3009, 3009-389 (Sept. 30,
1996), requires agencies to implement financial management systems that
substantially comply with (1) federal financial management systems
requirements, (2) federal accounting standards, and (3) the U.S. Standard
General Ledger at the transaction level.

15 [38]GAO-04-774 .

The DHS Financial Accountability Act of 200416 made DHS subject to the
Chief Financial Officers Act of 1990 (CFO Act),17 which requires DHS to
issue audited financial statements, among other things. In fiscal year
2006, the DHS financial statement auditors issued a disclaimer of opinion
because the scope of their work was not sufficient to express an opinion
given the seriousness of DHS's financial management problems. DHS's
Inspector General engaged the auditors to audit the balance sheet and
statement of custodial activity for the fiscal year that ended September
30, 2006. The auditors were not engaged to audit DHS's statements of net
costs, changes in net position, budgetary resources, and financing for the
years ended September 30, 2006 and 2005, because the Office of Financial
Management, Coast Guard, TSA, FEMA, U.S. Immigration and Customs
Enforcement (ICE), and the Management Directorate were unable to provide
sufficient evidence to support account balances presented in the financial
statements. In fiscal year 2006, DHS's financial statement auditors also
reported18 that DHS was not in compliance with the CFO Act as well as
other key financial management reform legislation, such as the Federal
Managers' Financial Integrity Act of 198219 and FFMIA. Resolving all
reported internal control weaknesses, addressing serious financial
management systems deficiencies, and complying with financial management
reform legislation are key to DHS's ability to produce relevant and
reliable financial information that will enable it to better manage the
department and provide accountability.

16The Department of Homeland Security Financial Accountability Act of
2004, Pub. L. No. 108-330 S 3, 118 Stat. 1275, 1276 (Oct. 16, 2004), added
DHS to the list of CFO Act agencies.

17Pub. L. No. 101-576, 104 Stat. 2838 (Nov. 15, 1990).

18Department of Homeland Security, Performance and Accountability Report
Fiscal Year 2006 (Washington, D.C.: November 2006).

19Pub. L. No. 97-255, 96 Stat. 814 (Sept. 8, 1982) (codified at  31 U.S.C.
S 3512 (c), (d)).

DHS Lacks a Fully Developed Financial Management Strategy and Plan

The eMerge2 program that was expected to integrate financial management
systems across the entire department and address financial management
weaknesses was a failure, and DHS wisely halted the project in December
2005. Since that time and 4 years after the creation of the agency, DHS is
still contemplating various financial management options. DHS has yet to
clearly define a financial management strategy and plan to move forward
with its financial management system modernization efforts. Such a plan is
needed to address the fundamental financial management problems that have
existed since the agency was created. In early March 2007, DHS officials
issued a high level plan, which DHS stated was intended to address
existing internal control weaknesses. While a first step, more detailed
implementation strategies and plans will be necessary to fully address the
financial management systems challenges.

DHS officials told us they have decided to consolidate the department's
financial management systems. DHS and Office of Management and Budget
(OMB) officials told us that OMB approved DHS's decision to rely on its
in-house core financial management operations. DHS officials within the
Office of the Chief Financial Officer stated that they were performing an
internal assessment of the financial management systems being used by the
components and revisiting current internal financial service providers,
such as the Coast Guard, to determine whether they can leverage those
resources. The systems used by TSA and CBP were some of the internal DHS
systems being considered. Recent plans call for the Coast Guard to move to
the TSA systems model. In accordance with this approach, DHS officials
told us that they have plans to develop three or four shared service
providers using the existing component financial management systems. Some
of the services may include information technology (IT) hosting,20
business process services,21 and application management services.22
However, DHS did not provide documentation or evidence of the internal
assessment that it was conducting or when it would be completed. In
commenting on a draft of this report, DHS indicated that it was focusing
on two current systems already in use at TSA and CBP and how to migrate
other DHS components to those systems.

20IT hosting involves providing secure facility space, networks, and
hardware to host software applications and providing the necessary
personnel to operate this secure environment.

21Business process services involve services ranging from transaction
processing to financial management services. The range of services may
include general ledger reconciliation, budget formulation, and audit
support.

22Application management services include services for running and
managing access to business software applications and the feeder systems
that provide data to the financial management software.

The components that DHS is considering as systems models have material
financial management weaknesses and consequently do not appear to be good
candidates to be the models used by an entity with an annual budget in
excess of $40 billion. While DHS has corrective action plans under way to
address identified weaknesses, most of the component core financial
management systems are unable to produce reliable, useful, and timely
financial information. The auditors have not been able to issue an opinion
on DHS's financial statements since the agency was created in 2003. For
example, in fiscal year 2006, TSA, one of the proposed internal systems
models, was unable to provide sufficient evidential matter or make
knowledgeable representation of facts and circumstances to the DHS
financial statement auditors to support transactions and account balances
of TSA for amounts reported on DHS's balance sheet. Specifically, TSA was
unable to support transactions related to property and equipment, accrued
unfunded employee leave, accounts payable, and components of net position.
In addition, the auditors reported that TSA did not have sufficient
processes and procedures to enable the successful completion of a
financial statement audit in fiscal years 2005 and 2006. In commenting on
a draft of this report, DHS officials stated that TSA's audit shortcomings
were centered on policies and procedures, not systems-oriented problems.
However, our analysis of the auditor's report indicated that the problems
were broad based. As DHS pointed out in its comments, success in financial
management rests upon a comprehensive framework of people, policy,
process, systems, and assurance. Accordingly, it is imperative that DHS
understand the policy and procedure weaknesses at TSA in order to prevent
such weaknesses from affecting subsequent users.

Further, the Coast Guard, TSA's current shared service provider, was
unable to provide sufficient evidential matter or make knowledgeable
representations of facts and circumstances to the DHS financial statement
auditors, to support transactions and account balances of the Coast Guard
for amounts reported on DHS's balance sheet. The Coast Guard was unable to
support transactions related to fund balance with treasury; accounts
receivable; actuarially-derived liabilities; environmental and legal
liabilities; operating materials and supplies; certain categories of
property, plant, and equipment; undelivered orders and changes in net
position; and adjustments, both manual and automated, made as part of the
Coast Guard's financial reporting process. The Coast Guard was also unable
to complete corrective actions, and make adjustments, as necessary, to
these and other balance sheet amounts, prior to the completion of the DHS
2006 Performance and Accountability Report. The total assets of the Coast
Guard, as reported on the DHS balance sheet as of September 30, 2006, were
$12.5 billion, or 16 percent of total DHS consolidated assets. In
addition, the auditors reported that the Coast Guard does not have an
organizational structure that fully supports the development and
implementation of effective policies, procedures, and internal controls.
Consequently, to the extent that the shared service approach is sustained,
it will be critical for DHS to avoid replicating these weaknesses and
ineffective policies and procedures at other components.

According to DHS officials, migration is only one component of an
improvement program and can be costly, risky, and very disruptive. We
agree that implementation of any financial management system brings a
degree of risk. This is magnified when an organization has a range of
serious problems as is the case with DHS. Our report23 summarizing
financial management systems implementation problems at other federal
agencies established that failure to effectively follow best practices was
a key shortcoming that lead to failure to meet cost, schedule, and
performance goals. Later in this report, we offer our perspective on how
DHS can embrace these best practices to minimize these risks as it moves
forward.

Managing the transformation of an organization of the size and complexity
of DHS requires comprehensive planning, integration of key management
functions across the department, and partnering with stakeholders across
the public and private sectors. On September 13, 2006, the department's
CFO testified before the Congress that DHS's goals for improving its
financial systems have not changed and a major effort remains to improve
all of its resource management systems. Rather than focus only on systems,
the CFO testified that the department was currently developing an
overarching strategy to address challenges in the areas of people,
process, policy, systems, and assurances to achieve the department's goals
of obtaining a clean audit opinion, establishing sound internal controls,
and improving the efficiency of financial operations. The CFO stated that
DHS understands that some systems are aging; that some fail to meet all
user requirements; and that some are not fully integrated with finance,
procurement, and asset management. To meet these needs, the DHS CFO
reported that DHS is building a financial management framework. The CFO
said that the centerpiece of the effort to improve agency financial
processes and address the existing financial management problems is DHS's
Internal Controls Over Financial Reporting (ICOFR) Playbook, released in
March 2007. DHS officials have reported that the ICOFR Playbook draws from
internal control best practices to establish a management control program
that measures performance and provides accountability for improvement. DHS
officials expect the ICOFR Playbook to guide DHS ahead for the next
several years through fundamental financial management improvement across
the spectrum of financial activities supporting the agency's mission.

23GAO, Financial Management Systems: Additional Efforts Needed to Address
Key Causes of Modernization Failures, [39]GAO-06-184 (Washington, D.C.:
Mar. 15, 2006).

We found that the ICOFR Playbook does not contain adequate detail to
clarify the approach that DHS plans to take to modernize its financial
management systems. For example, the ICOFR Playbook focuses on financial
statement preparation and only includes two tracks. The first track
focuses on corrective action strategies for material weaknesses, and the
second track focuses on building support for the Secretary's internal
control over financial reporting assurance statement. In its comments on a
draft of this report, DHS officials acknowledged that the ICOFR Playbook
is at the policy and process level and does not comprise a specific
strategy for financial systems modernization. Much more detail is needed
to provide a financial management strategy or plan for integrating and
modernizing DHS's financial management systems. While there continues to
be much focus on agency and governmentwide audit opinions, getting a clean
audit opinion, though important in itself, is not the end goal. The end
goal is the establishment of a fully functioning CFO operation that
includes (1) modern financial management systems that provide reliable,
timely, and useful information to support day-to-day decision-making and
oversight and for the systematic measurement of performance; (2) a cadre
of highly qualified senior level and supporting staff; and (3) sound
internal controls that safeguard assets and ensure proper accountability.

eMerge2 Costs Are Unknown and Work Products Have Limited Usefulness

Although DHS stated that it had spent about $52 million in agency costs
for the eMerge2 project, including approximately $18 million of contractor
costs, it did not provide adequate support for these amounts. Moreover,
DHS believes that the eMerge2 funds spent will benefit its future
financial management modernization efforts since a number of the work
products can still be used. However, our review of two key items--a
concept of operations document and system requirements--found that they
have significant deficiencies and will be of little use for future
efforts. Specifically, the concept of operations does not contain an
adequate description of the legacy systems and a clear articulation of the
vision that should guide the department's improvement efforts, while key
requirements developed for the project are unclear and incomplete. Based
on best practices that form the foundation for successful financial
management systems implementation, DHS will have little assurance that its
future efforts will meet their cost, schedule, and performance goals.
These issues are discussed in greater detail later in this report.

Actual Costs of eMerge2 Are Unknown

DHS officials told us they ended the eMerge2 program because (1) the
project fell behind schedule and (2) the contactor could not meet
established performance goals. We were unable to confirm the estimated $52
million in eMerge2 program costs because DHS officials did not provide
adequate supporting evidence to document this amount after repeated GAO
requests. eMerge2 was expected to establish the strategic direction for
migration, modernization, and integration of DHS financial, accounting,
procurement, personnel, asset management, and travel systems, processes,
and policies. DHS officials began working on the project in late fiscal
year 2003. DHS contracted with Bearing Point, Inc. (Bearing Point) to
develop the functional and technical eMerge2 requirements. These
requirements were approved by all DHS components in May 2004. Based on
these requirements, DHS developed a Request for Quotation for the
acquisition and implementation of eMerge2.

In September 2004, after a competitive acquisition process, Bearing Point
was awarded a blanket purchase agreement with a ceiling of about $229
million to acquire and implement the eMerge2 solution. The first task
order was issued under the agreement for solution development and
conference room pilot (CRP) testing.24 Bearing Point began the CRP
initiative in November 2004, and soon into work on this task order,
concerns began to arise regarding the extent to which there was a clear
understanding between DHS and Bearing Point on exactly what was to be
delivered. In December 2004, DHS officials formally communicated their
concerns to Bearing Point by requesting a performance improvement plan. In
January 2005, Bearing Point submitted a performance improvement plan.
According to DHS officials, Bearing Point missed deadlines, and some
products presented to the eMerge2 project team were deemed unacceptable.
In February 2005, the DHS CFO conducted a review of the eMerge2 effort.
DHS chose not to exercise the next contract option, and the Bearing Point
contract to acquire and implement eMerge2 expired in December 2005. See
figure 1 for a summary of the eMerge2 timeline. In March 2006, DHS's
Deputy CFO testified25 that eMerge2 was taking a new direction in that the
department was going to perform an internal assessment of existing
financial management systems at the component level to determine whether
resources could be leveraged. DHS officials also reported that they were
going to review the OMB Financial Management Line of Business initiative
to assess whether migration to a shared service provider was a feasible
option. Finally, in September 2006, the newly appointed CFO stated that
eMerge2 was officially "dead."

24CRP is a configured solution ready for the execution of scenarios. The
solution is measured against its capability to satisfy the eMerge2
requirements. The CRP was not executed.

25Department of Homeland Security - March 29, 2006, testimony before the
House Government Reform Subcommittee on Government Management, Finance,
and Accountability and the House Homeland Security Subcommittee on
Management, Integration, and Oversight.

Figure 1: eMerge2 Project Timeline

eMerge2 Work Products Have Limited Future Usefulness

According to DHS officials, several of the work products developed during
eMerge2 will benefit its future financial management modernization
efforts. These products included a concept of operations and over 7,000
requirements. A review of these two critical products found that they will
not provide much assistance to future efforts since they do not contain
the attributes normally associated with such documents. The concept of
operations document we reviewed did not include all the important elements
and the requirements did not flow from the concept of operations.
Moreover, key requirements (1) lacked the IEEE characteristics associated
with good requirements; (2) did not incorporate the functionality
associated with inventories, supplies, and materials; and (3) did not
consider appropriate internal control. Accordingly, these documents will
have to undergo significant rework before they can be used in future
efforts.

  The Concept of Operations Document Is Flawed

Our review of the DHS concept of operations found that it did not have the
types of information expected when compared to best practices. As we noted
in March 2006, a concept of operations defines how an organization's
day-to-day operations are (and will be) carried out to meet mission needs.
The concept of operations includes high-level descriptions of information
systems, their interrelationships, and information flows. It also
describes the operations that must be performed, who must perform them,
and where and how the operations will be carried out. Further, it provides
the foundation on which requirements definitions and the rest of the
systems planning process are built. Normally, a concept of operations
document is one of the first documents to be produced during a disciplined
development effort and flows from both the vision statement and the
enterprise architecture. According to IEEE standards,26 a concept of
operations is a user-oriented document that describes the characteristics
of a proposed system from the users' viewpoint. The key elements that
should be included in a concept of operations are major system components,
interfaces to external systems, and performance characteristics, such as
speed and volume.

Our review of the DHS concept of operations found that it lacked several
key attributes called for by best practices. For example, DHS officials
stated that the guiding principles of the functional vision for the
eMerge2 program focused on the "to-be" state and that they did not attempt
to document the "as-is" state. As noted in the IEEE standard, the "as-is"
environment is normally captured or depicted in the concept of operations
document. In the case of DHS this is especially important since when the
eMerge2 project began, DHS had identified over 500 financial management
and related systems in operation and much of its operational history was
contained in legacy systems data files. Figure 2 provides a summary of
DHS's systems inventory by resource functions.

26IEEE Std. 1362-1998.

Figure 2: DHS Systems Inventory

Due to the large number of systems, DHS needs to define in its concept of
operations (1) which legacy systems will be migrated to the new
environment and (2) conceptually how this transition is envisioned to
occur in order to achieve an integrated environment. As we noted in our
March 2006 testimony,27 the transition strategy outlined in the concept of
operations is useful for developing an understanding of how and when
changes will occur. Not only is this needed from an investment management
point of view, it is a key element in addressing human capital problems
relating to change management strategies. Simply saying "all systems will
be migrated to the new environment" does not provide an understanding of
how this transition will take place or provide the necessary specificity
to help the concept of operations serve as the foundation for the
requirements management process. For example, should DHS decide to develop
and implement a standard budget system that includes both formulation and
execution, it would need to ensure that the new budget system achieved the
functionality associated with over 60 existing budget legacy systems.

27 [40]GAO-06-553T .

  eMerge2 System Requirements Are Deficient

Although DHS officials told us that they expected the requirements
developed for eMerge2 to be salvageable and provide a foundation for its
future efforts, our review found that key requirements did not have
attributes associated with good requirements developed using best
practices. Requirements are specifications that system developers and
program managers use to design, develop, and acquire a system. They need
to be carefully defined, consistent with one another, verifiable, and
directly traceable to higher level business or functional requirements.
Most importantly, the eMerge2 requirements were not based on (1) a good
concept of operations, (2) reengineered business processes, and (3) an
appropriate internal control structure.

In our March 2006 report,28 we noted that business process models provide
a way of expressing the procedures, activities, and behaviors needed to
accomplish an organization's mission and are helpful tools to document and
understand complex systems. Business processes are the various steps that
must be followed to perform a certain activity. For example, the
procurement process would start when the agency defines its needs and
issues a solicitation for goods or services, and would continue through
contract award and receipt of goods and services, and would end when the
vendor properly receives payment. The identification of preferred business
processes is critical for the standardization of applications and training
and portability of staff.

DHS officials reportedly developed approximately 33 business processes
across five business domains29 using Business Process Modeling Notation
(BPMN)30 during the eMerge2 effort. While DHS officials stated that they
placed an emphasis on business processes when capturing requirements,
their business process emphasis focused on the "to-be" state versus the
"as-is" state. However, industry standards suggest that it is important to
model the processes currently in operation ("as-is") because it allows an
organization to discover the existing core business processes. An
organization needs to be fully aware of its existing core business
processes because reassessment of those processes is necessary to ensure
continued value and capability in a new system. In order to maximize the
success of a new system, redesigning the current business processes while
promoting consistency through the development of standard business
processes is essential for a large and complex agency like DHS.
Identifying or developing preferred business processes for standardization
of applications and training and portability of staff also helps when
selecting the appropriate software that best reflects the preferred
business processes.

28 [41]GAO-06-184 .

29The DHS five business domains are (1) accounting and reporting, (2)
acquisition and grants, (3) asset management, (4) budget, and (5) cost and
revenue performance management.

Since DHS has not defined its standard business processes, it is unclear
whether the requirements are valid because some of the requirements are
process specific and we were unable to test the linkage between
requirements and DHS business processes. DHS developed over 7,000 external
requirements and derived requirements. The external requirements were
compiled based upon externally mandated laws and regulations. The derived
requirements were compiled based upon business process modeling that
incorporated external requirements, business rules, leading practices,
known deficiencies, roles, data objects, and interface requirements. The
derived requirements were also organized by the five functional domains
noted above. However, even assuming that the requirements were "linked" to
the processes that DHS would like to employ, many of the key requirements
did not have the attributes associated with good requirements. The
following are examples of the requirements problems we noted.

30BPMN defines a Business Process Diagram, which is based on a
flowcharting technique tailored for creating graphical models of business
process operations. A business process model, then, is a network of
graphical objects, which are activities (i.e., work) and the flow controls
that define their order of performance.

           o One requirement stated that "the system must calculate gross
           pay, deductions, net pay, employee, and employer contributions for
           each employee on an effective pay period basis." The requirement
           is unnecessary because all of DHS's components have migrated
           payroll processing functions to the Department of Agriculture's
           National Finance Center. Moreover, the requirement does not
           address basic questions, such as (1) which payroll system will
           perform this function, (2) how is the gross pay amount defined,
           and (3) what deductions must be supported (taxes, retirement,
           employee allotments, etc.).
           o Another requirement stated that "the bottom line of this
           reconciliation would be the net cost of operations defined." It is
           unclear what reconciliation is being performed and how the net
           cost of operations is defined or which other requirement provided
           this formula.
           o We were unable to identify critical requirements relating to
           inventory. According to DHS's fiscal year 2006 statements, the
           department held about $677 million in inventory and supplies.
           Basic requirements, such as determining the inventory valuation
           method and ensuring that inventory items transferred between DHS
           locations retain their historical cost basis, were not included.
           These are critical items for maintaining visibility of assets and
           the financial presentation process.
           o All requirements were considered "equal." For example, some
           requirements were simply the language used in a given law or
           regulation while other requirements appeared to be intended to
           provide additional specificity to those requirements. However,
           these related requirements were not "linked" in such a manner that
           made these relationships clear. One approach that can be used is
           to provide a hierarchal structure. Under this concept, the general
           requirements are at one level while the more specific requirements
           are at a lower level and linked to the higher level requirements.
           This process maintains the necessary traceability (another best
           practice concept) between the requirements.31

           DHS officials have stated that the eMerge2 requirements did not
           consider the internal control structure. OMB's Core Financial
           System Requirements32 have several mandatory requirements that
           must be considered when migrating or implementing the system
           management function in federal financial management systems. Some
           of these requirements include accounting classification, document
           and transaction control, system generated transactions, and audit
           trails. OMB Circular No. A-123, Management's Responsibility for
           Internal Control, requires agencies to operate systems with
           appropriate internal controls to ensure accuracy of data,
           completeness and consistency of transaction processing, and
           adequate reporting. Automatic internal control capabilities needed
           to meet the provisions of Circular No. A-123 are expected to be
           integrated into financial management systems. For example,
           requirements that specify validations to be performed on invoice
           data before they can be certified as ready for payment and
           system-enforced separation of duties are some of the basic control
           activities that are expected to be integrated into a financial
           management system. As we have noted in numerous reports,
           requirements management problems are a leading cause of systems
           that do not meet their cost, schedule, and functionality
           objectives. (See our related GAO products section at the end of
           this report).
			  
31Requirements for projects can be expressed at various levels depending
on user needs. They range from agencywide business requirements to
increasingly detailed functional requirements that eventually permit the
software project managers and other technicians to design and build the
required functionality in the new system. Adequate traceability ensures
that a requirement in one document is consistent with and linked to
applicable requirements in another document.

32OMB's Office of Federal Financial Management, Core Financial System
Requirements, OFFM-NO-0106, January 2006.

           Four Key Building Blocks and Effective Human Capital Management
			  Must Drive DHS's Financial Management Transformation Efforts

           Based on industry best practices, we have identified four key
           building blocks that will be critical to DHS's ability to
           successfully complete its financial transformation. Our March 2006
           testimony33 pointed out that careful consideration of these four
           concepts, each one building upon the former, will be integral to
           the success of DHS's strategy. The four concepts are (1)
           developing a concept of operations, (2) defining standard business
           processes, (3) developing a migration and/or implementation
           strategy for DHS components, and (4) defining and effectively
           implementing disciplined processes necessary to properly manage
           the specific projects. Fully embracing these four building blocks
           and human capital best practices will be critical to the success
           of any future financial management plan or strategy that addresses
           implementing and/or migrating to an integrated departmentwide
           financial management system at DHS. DHS also has an opportunity to
           reap substantial benefits by reengineering business processes and
           standardizing those processes so that productivity gains and staff
           portability across the various components are realized. In
           addition, identifying staff with the requisite skills to implement
           such systems and identifying gaps in needed staff skills and
           filling them are necessary to successfully implement and operate a
           new financial management system. Any financial management plan or
           strategy implemented by DHS will be complex and challenging,
           making the adoption of best practices even more important for this
           undertaking. We will now highlight the key issues to be considered
           for each of the four areas and human capital. Moreover, detailed
           key questions for DHS to consider related to each concept can be
           found in appendix III.

33 [43]GAO-06-553T .

           Concept of Operations Provides Foundation

           As we discussed previously, a concept of operations defines how an
           organization's day-to-day operations are (or will be) carried out
           to meet mission needs. The concept of operations includes
           high-level descriptions of information systems, their
           interrelationships, and information flows. It also describes the
           operations that must be performed, who must perform them, and
           where and how the operations will be carried out. Further, it
           provides the foundation on which requirements definitions and the
           rest of the systems planning process are built. Normally, a
           concept of operations document is one of the first documents to be
           produced during a disciplined development effort and flows from
           both the vision statement and the enterprise architecture.
           According to the IEEE standards,34 a concept of operations is a
           user-oriented document that describes the characteristics of a
           proposed system from the users' viewpoint. The key elements that
           should be included in a concept of operations are major system
           components, interfaces to external systems, and performance
           characteristics, such as speed and volume.

           Another key element of a concept of operations is a transition
           strategy that is useful for developing an understanding of how and
           when changes will occur. Not only is this needed from an
           investment management point of view, it is a key element in the
           human capital problems discussed previously that revolved around
           change management strategies. Describing how to execute DHS's
           approach for implementing a new system or migrating to shared
           service providers, as well as the processes that will be used to
           deactivate legacy systems that will be replaced or interfaced with
           a new financial management system, are key aspects that need to be
           addressed in a transition strategy.
			  
34IEEE Std. 1362-1998.

           Standard Business Processes Promote Consistency

           Business process models provide a way of expressing the
           procedures, activities, and behaviors needed to accomplish an
           organization's mission and are helpful tools to document and
           understand complex systems. In our view, an agency's mission must
           drive the business processes and the resulting financial
           information is a derivative of these processes. Moreover, business
           processes are the various steps that must be followed to perform a
           certain activity. For example, the procurement process would start
           when the agency defines its needs and issues a solicitation for
           goods or services, and would continue through contract award and
           receipt of goods and services, and would end when the vendor
           properly receives payment. As we discussed earlier in this report,
           the identification of preferred business processes would be
           critical for standardization of applications and training and
           portability of staff.

           To maximize the success of a new system acquisition, organizations
           need to consider the redesign of current business processes. As we
           noted in our Executive Guide: Creating Value Through World-class
           Financial Management,35 leading finance organizations have found
           that productivity gains typically result from more efficient
           processes, not from simply automating old processes. Moreover, the
           Clinger-Cohen Act of 1996 requires agencies to analyze the
           missions of the agency and, based on the analysis, revise
           mission-related and administrative processes, as appropriate,
           before making significant investments in IT used to support those
           missions.36 Another benefit of what is often called business
           process modeling is that it generates better system requirements,
           since the business process models drive the creation of
           information systems that fit in the organization and will be used
           by end users. Other benefits include providing a foundation for
           agency efforts to describe the business processes needed for
           unique missions and developing subprocesses to support those at
           the departmentwide level.
			  
			  Strategy for Consolidating and Migrating Financial Management
			  Systems Will Be Key

           Although DHS officials have stated that they plan to consolidate
           their financial management systems, the department has not yet
           articulated a detailed plan for achieving this goal. In the
           context of consolidating financial management operations, which
           will include migrating to a selected systems model, critical
           activities include (1) developing specific criteria for requiring
           component agencies to migrate to one of the providers rather than
           attempting to develop and implement their own stove-piped business
           systems; (2) providing the necessary information for a component
           agency to select a DHS-approved financial management system; (3)
           defining and instilling new values, norms, and behaviors within
           component agencies that support new ways of doing work and
           overcoming resistance to change; (4) building consensus among
           customers and stakeholders on specific changes designed to better
           meet their needs; and (5) planning, testing, and implementing all
           aspects of the transition from one organizational structure and
           business process to another.
			  
35GAO, Executive Guide: Creating Value Through World-class Financial
Management, [44]GAO/AIMD-00-134 (Washington, D.C.: April 2000).

36See 40 U.S.C. S11303(b)(2)(C).

           Regardless of the strategy DHS takes, sustained leadership will be
           key to a successful migration strategy for moving DHS toward a
           consolidated financial management system. In our Executive Guide:
           Creating Value Through World-class Financial Management, we found
           that leading organizations made financial management improvement
           an entitywide priority by, among other things, providing clear,
           strong executive leadership. We also reported that making
           financial management a priority throughout the federal government
           involves changing the organizational culture of federal agencies.
           Although the views about how an organization can change its
           culture can vary considerably, leadership (executive support) is
           often viewed as the most important factor in successfully making
           cultural changes. Top management, such as the Secretary, must be
           totally committed in both words and actions to changing the
           culture, and this commitment must be sustained and demonstrated to
           staff. As pressure mounts to do more with less, to increase
           accountability, and to reduce fraud, waste, abuse, and
           mismanagement, and efforts to reduce federal spending intensify,
           sustained and committed leadership will be a key factor in the
           successful migration of DHS's financial management systems.
			  
			  Disciplined Processes Will Help Ensure Successful Implementation

           Once the concept of operations and standard business processes
           have been defined and a migration or implementation strategy is in
           place, the use of disciplined processes will be a critical factor
           in helping to ensure that the implementation is successful. The
           key to avoiding long-standing implementation problems is to
           provide specific guidance to component agencies for financial
           management system implementations, incorporating the best
           practices identified by the Software Engineering Institute, the
           IEEE, the Project Management Institute, and other experts that
           have been proven to reduce risk in implementing systems. Such
           guidance should include the various disciplined processes, such as
           requirements management, testing, data conversion and system
           interfaces, risk and project management, and related activities,
           which have been problematic in the financial systems
           implementation projects we and others have reviewed.

           Disciplined processes have been shown to reduce the risks
           associated with software development and acquisition efforts to
           acceptable levels and are fundamental to successful system
           implementations. The principles of disciplined IT systems
           development and acquisition apply to shared services
           implementation, such as that contemplated by DHS. A disciplined
           software implementation process can maximize the likelihood of
           achieving the intended results (performance) within established
           resources (costs) on schedule. For example, disciplined processes
           should be in place to address the areas of data conversion and
           interfaces, two of the many critical elements necessary to
           successfully implement a new system--the lack of which has
           contributed to the failure of previous agency efforts. Further
           details on disciplined processes can be found in appendix IV.
           Inadequate implementation of disciplined processes can manifest
           itself in many ways when implementing a financial management
           system. Full deployment has been delayed at some agencies and
           specific functionality has been delayed or flawed at other
           agencies.
			  
			  Strong Human Capital Management Needed at DHS

           Effective human capital management is critical to the success of
           systems implementations. As we reported in our Executive Guide:
           Creating Value Through World-class Financial Management,37 having
           staff with the appropriate skills is key to achieving financial
           management improvements, and managing an organization's employees
           is essential to achieving results. The independent public
           accountants that conducted DHS's fiscal year 2006 audit have
           stated that many of the department's difficulties in financial
           management and reporting can be attributed to the original
           stand-up of a large, new, and complex executive branch agency
           without adequate organizational expertise in financial management
           and accounting. Moreover, DHS's Resource Management Transformation
           Office (RMTO) officials have stated that outside contractors are
           currently performing some of the financial management activities
           or duties that internal DHS staff would normally perform because
           of staffing shortages. Having adequate and sufficient human
           resources with the requisite training and experience to
           successfully implement a financial management system is a critical
           success factor.
			  
37 [45]GAO/AIMD-00-134 .			  

           Our work38 has identified significant human capital issues,
           including the lack of IT expertise, that have affected financial
           systems implementation at other agencies. Some of the human
           capital problems we identified that have hampered the
           implementation of new financial management systems include
           incomplete strategic workforce planning and ongoing staff
           shortages as well as untrained staff. By not identifying staff
           with the requisite skills to implement such systems and by not
           identifying gaps in needed skills and filling them, agencies
           reduce their chances of successfully implementing and operating a
           new financial management system. Further, OMB guidance39 requires
           agencies to have qualified project managers for major IT
           investments.

           Strategic human capital management for financial management
           projects includes organizational planning, staff acquisition, and
           team development. Human capital planning is necessary for all
           stages of the system implementation. It is important that agencies
           incorporate strategic workforce planning by (1) aligning an
           organization's human capital programs with its current and
           emerging mission and programmatic goals and (2) developing
           long-term strategies for acquiring, developing, and retaining an
           organization's total workforce to meet the needs of the future. As
           we have recently testified,40 some of the most pressing human
           capital challenges at DHS include (1) successfully completing its
           ongoing transformation; (2) forging a unified results-oriented
           culture across the department; (3) obtaining, developing,
           providing incentives to, and retaining needed talent; and (4) most
           importantly, leadership at the top, to include a chief operating
           officer or chief management officer. The federal government has
           always faced the challenge of sustaining the momentum of
           transformation because of the limited tenure of key administration
           officials, and managing the transformation of an organization of
           the size and complexity of DHS requires comprehensive planning and
           integration of key management functions across the department.
			  
			  Conclusions

           GAO and others have found that the key to implementing systems
           that meet cost, schedule, and performance objectives is to have
           effectively implemented the disciplined processes necessary to
           reduce risks to acceptable levels. DHS has not yet taken the first
           step, which is to define a formal financial management strategy
           that addresses the fundamental financial management problems that
           have existed since the agency's creation. Ending eMerge2 was a
           judicious decision; however, we are concerned that DHS still lacks
           a clearly defined financial management strategy or financial
           management systems implementation effort to even begin to address
           DHS's integration and transformation issues as reported in our
           most recent high-risk report. Furthermore, because DHS is one of
           the largest and most complex executive branch agencies in the
           federal government, developing, operating, maintaining, and
           modernizing its financial management systems represent a
           monumental challenge. This challenge is compounded by DHS's
           newness and the poor condition of the range of legacy financial
           and related business systems it inherited. To that end, critical
           success factors include utilizing the four building blocks and
           human capital best practices to provide reasonable assurance that
           the risks associated with implementing a departmentwide integrated
           financial management system are minimized. Otherwise, DHS runs the
           risk of repeating the failure of eMerge2.
			  
38 [46]GAO-06-184 .

39See OMB, Information Technology Project Manager Qualification Guidance,
M-04-19 (Washington, D.C.: July 21, 2004), and OMB Circular No. A-11, S
300.

40GAO, Homeland Security: Management and Programmatic Challenges Facing
the Department of Homeland Security, [47]GAO-07-398T (Washington, D.C.:
Feb. 6, 2007).			

           Recommendations for Executive Action  

           To help reduce the risks associated with a departmentwide
           financial management system implementation effort, we recommend
           that the Secretary of DHS demonstrate commitment to integrating
           DHS's financial management systems and direct the Undersecretary
           for Management and Chief Financial Officer to take the following
           six actions. This would entail placing a high priority on fully
           integrating into its approach the following concepts and
           underlying key issues, which are related to the fundamental
           disciplined processes typically utilized in systems
           implementation.

           o Clearly define and document a departmentwide financial
           management strategy and plan to move forward with its financial
           management system integration efforts.
           o Fully embrace the four building blocks and best practices when
           developing and documenting the strategy and plan to foster the
           development of an integrated financial management system that
           meets expected performance and functionality targets. This would
           include the following:

                        o Developing a comprehensive concept of operations
                        document
                        o Reengineering business processes and standardizing
                        them across the department, including applicable
                        internal control
                        o Developing a detailed plan for consolidating and
                        migrating various DHS components to an internal
                        shared services approach if this approach is
                        sustained
                        o Utilizing and implementing the specific disciplined
                        processes below to minimize project risk

                                     1. Requirements management
                                     2. Testing
                                     3. Data conversion and system interfaces
                                     4. Risk management
                                     5. Configuration management
                                     6. Project management
                                     7. Quality assurance

           o Carefully consider key human capital practices as DHS moves
           forward with its financial management transformation efforts so
           that the right people with the right skills are in place at the
           right time.
			  
			  Agency Comments and Our Evaluation

           We received written comments on a draft of this report from DHS,
           which are reprinted in appendix V. DHS concurred with our
           recommendations and described the actions it has taken or plans to
           take to improve financial management systems and departmentwide
           financial accountability. As DHS moves forward to address the
           recommendations in our report, it is important that it prioritize
           its efforts and focus on the concepts and key issues we discussed,
           such as clearly documenting and defining a departmentwide
           financial management systems integration strategy and implementing
           disciplined processes. We are encouraged that DHS has recognized
           that attention is needed and is developing plans to address these
           financial management systems issues. It is critical that the
           departmentwide financial management strategy is documented and
           stresses the importance of a standard set of business processes.
           We continue to believe that careful consideration of all the
           building blocks and key issues we identified will be integral to
           the success of DHS's financial management systems integration
           efforts. DHS also provided technical comments, which we
           incorporated as appropriate.

           As arranged with your offices, unless you announce the contents of
           this report earlier, we will not distribute it until 30 days from
           its date. Then we will send copies of this report to interested
           congressional committees. We will also send copies to the
           Secretary of Homeland Security, the DHS Under Secretary for
           Management, and the DHS Chief Financial Officer. Copies will be
           made available to others upon request. In addition, this report
           will also be available at no charge on GAO's Web site at
           [42]http://www.gao.gov .

           If you or your staff have any questions about this report, please
           contact McCoy Williams, Director, Financial Management and
           Assurance, who may be reached at (202) 512-9095 or by e-mail at
           [email protected], or Keith A. Rhodes, Chief Technologist,
           Applied Research and Methods, who may be reached at (202) 512-6412
           or by e-mail at [email protected]. Contact points for our Offices of
           Congressional Relations and Public Affairs may be found on the
           last page of this report. GAO staff who made major contributions
           to this report are listed in appendix VI.

           McCoy Williams
			  Director
			  Financial Management and Assurance

           Keith A. Rhodes
			  Chief Technologist
			  Applied Research and Methods
             Center for Technology and Engineering
				 
           Appendix I: Scope and Methodology

           To determine whether the Department of Homeland Security (DHS) has
           developed plans for implementing and/or migrating to an integrated
           departmentwide financial management system, we interviewed key DHS
           officials, reviewed relevant DHS's Resource Management
           Transformation Office's (RMTO) policy and procedure documents, and
           analyzed the Electronically Managing Enterprise Resources for
           Government Effectiveness and Efficiency (eMerge2) work products
           related to the financial modernization effort. We reviewed DHS
           performance and accountability reports, particularly the
           Management Discussion and Analysis section, to determine whether
           there were any financial management system modernization
           initiatives under way. We also reviewed the Office of Management
           and Budget (OMB) Exhibit 300, and relevant contractor files and
           procurement data.

           To assess the potential usefulness of work products received for
           funds spent on eMerge2 efforts, we interviewed DHS officials and
           analyzed relevant DHS eMerge2planning documents, the eMerge2
           requirements database, and RMTO policy and procedure documents. We
           evaluated the key information in the requirements database by
           selecting requirements that focused on accounting and financial
           reporting issues. Based on our analysis, we concluded that the
           requirements we reviewed were unclear and incomplete. As a result,
           we determined it would not be useful for DHS's future efforts to
           integrate financial management systems. We also reviewed Bearing
           Point, Inc.'s contractor files to determine the nature and scope
           of contractual services provided by the systems integrator. We
           requested but did not receive invoices and other documents to
           support amounts spent on eMerge2. Accordingly we were unable to
           test amounts DHS officials told us were spent on the project. As a
           result we are unable to provide any assurance on the accuracy of
           these amounts.

           To provide our views on how DHS can incorporate key building
           blocks and human capital best practices into its plans for
           migrating to an integrated departmentwide financial management
           system going forward, we reviewed our prior reports and material
           from key industry groups and national experts to identify any
           potential solutions posed by those groups, lessons learned, and
           relevant best practices.

           We conducted our work in Washington, D.C., from September 2006
           through April 2007, in accordance with U.S. generally accepted
           government auditing standards. We did not evaluate the federal
           government's overall information technology strategy or whether
           DHS selected the most appropriate financial management systems
           approach. We are making recommendations to DHS in this report. We
           requested comments on a draft of this report from the Secretary of
           DHS or his designee. Written comments from the Department of
           Homeland Security are reprinted in appendix V and evaluated in the
           "Agency Comments and Our Evaluation" section.

           Source: GAO based on DHS performance and accountability report(s).

           Source: GAO.

Appendix II: Material Weaknesses/Reportable Conditions at DHS for Fiscal
Years 2003 through 2006

Number Material weakness/reportable conditions         2003 2004 2005 2006 
1      Financial management and oversight: DHS's       SQRT SQRT SQRT SQRT 
          Office of the Chief Financial Officer (OCFO)                        
          needs to establish financial reporting roles                        
          and responsibilities, assess critical needs,                        
          and establish standard operating procedures for                     
          the department. These conditions were not                           
          unexpected for a newly created organization,                        
          especially one as large and complex as DHS. The                     
          Coast Guard and the Strategic National                              
          Stockpile had weaknesses in financial oversight                     
          that have led to reporting problems.                                
2      Financial reporting: Key controls to ensure     SQRT SQRT SQRT SQRT 
          reporting integrity were not in place, and                          
          inefficiencies made the process more error                          
          prone. At the Coast Guard, the financial                            
          reporting process was complex and                                   
          labor-intensive. Several DHS bureaus lacked                         
          clearly documented procedures, making them                          
          vulnerable if key people leave the                                  
          organization.                                                       
3      Financial systems security: The auditors found  SQRT SQRT SQRT SQRT 
          weaknesses across DHS in its entitywide                             
          security program management and in controls                         
          over system access, application software                            
          development, system software, segregation of                        
          duties, and service continuity. Many bureau                         
          systems lacked certain functionality to support                     
          the financial reporting requirements.                               
4      Property, plant, and equipment (PP&E): The      SQRT SQRT SQRT SQRT 
          Coast Guard was unable to support the recorded                      
          value of $2.9 billion in PP&E due to                                
          insufficient documentation provided prior to                        
          the completion of audit procedures, including                       
          documentation to support its estimation                             
          methodology. The Transportation Security                            
          Administration lacked a comprehensive property                      
          management system and adequate policies and                         
          procedures to ensure the accuracy of its PP&E                       
          records.                                                            
5      Operating materials and supplies (OM&S) and     SQRT SQRT SQRT SQRT 
          seized property: Internal controls over                             
          physical counts of OM&S were not effective at                       
          the Coast Guard. As a result, the auditors were                     
          unable to verify the recorded value of $497                         
          million in OM&S. The Coast Guard also had not                       
          recently reviewed its OM&S capitalization                           
          policy, leading to a material adjustment to its                     
          records when an analysis was performed. The                         
          Coast Guard Inventory Control Point physical                        
          inventory procedures lacked key elements of an                      
          effective physical inventory.                                       
6      Actuarial liabilities: The Secret Service did   SQRT SQRT SQRT      
          not record the pension liability for certain                        
          employees and retirees, and when corrected, the                     
          auditors had insufficient time to audit the                         
          amount recorded. The Coast Guard does not have                      
          adequate policies, procedures, and controls to                      
          ensure the completeness and accuracy of the                         
          data necessary for the calculation of actuarial                     
          liabilities.                                                        
7      Transfers of funds, assets, and liabilities to  SQRT                
          DHS: DHS lacked controls to verify that monthly                     
          financial reports and transferred balances from                     
          legacy agencies were accurate and complete.                         
8      Fund Balance with Treasury: The Coast Guard has      SQRT SQRT SQRT 
          not designed and implemented policies,                              
          procedures, and internal controls, including                        
          effective reconciliations and the use of a                          
          financial system that complies with Federal                         
          Financial System Requirements, as defined in                        
          OMB No. Circular A-127 and the requirements                         
          published by the Joint Financial Management                         
          Improvement Program.                                                
9      Legal and other liabilities: The Office of                     SQRT 
          Financial Management (OFM), in association with                     
          the Office of the General Counsel (OGC), has                        
          not implemented adequate policies and                               
          procedures to ensure that OFM is provided with                      
          sufficient information to accurately and                            
          completely present legal liabilities and                            
          related disclosures in the financial statements                     
          throughout the year.                                                
10     Intragovernmental and intradepartmental              SQRT SQRT SQRT 
          balances: Immigration and Customs Enforcement                       
          (ICE), Emergency Preparedness and Response                          
          (EPR) and Coast Guard have not developed and                        
          adopted effective standard operating                                
          procedures, or established systems, to                              
          completely track, confirm, and reconcile                            
          intra-DHS balances and/or transactions with                         
          trading partners in a timely manner.                                
11     Undelivered orders (UDO), accounts and grants        SQRT SQRT      
          payable, and disbursements: ICE had difficulty                      
          maintaining accurate records relating to                            
          obligations and UDOs and did not establish                          
          sufficient controls to prevent duplicate                            
          payments.                                                           
12     Financial management structure: OCFO has not         SQRT           
          provided the DHS bureaus with sufficient                            
          management oversight and timely policy guidance                     
          to address accounting and reporting issues that                     
          cross multiple bureaus and affect the                               
          efficiency of bureau financial accounting and                       
          reporting operations.                                               
13     Budgetary accounting: DHS lacked effective           SQRT SQRT SQRT 
          internal controls for validation and                                
          verification of UDO balances to ensure that                         
          recorded obligations were valid, and recorded                       
          in a timely manner, and that proper approval                        
          and supporting documentation is maintained.                         

Appendix III: Key Questions for the Department of Homeland Security to
Consider Based on the Four Building Blocks

Building block            Key questions                                    
Concept of operations        o What is considered a financial management   
                                system? Are all the components using a        
                                standard definition?                          
                                o Who will be responsible for developing a    
                                DHS-wide financial management concept of      
                                operations, and what process will be used to  
                                ensure that the resulting document reflects   
                                the departmentwide solution rather than       
                                individual component agency stove-piped       
                                efforts?                                      
                                o How will DHS's concept of operations be     
                                linked to its enterprise architecture?        
                                o How can DHS obtain reliable information on  
                                the costs of its financial management systems 
                                investments?                                  
Standard business process    o Who will be responsible for developing      
                                DHS-wide standard business processes that     
                                meet the needs of its component agencies?     
                                o How will the component agencies be          
                                encouraged to adopt new processes, rather     
                                than selecting other methods that result in   
                                simply automating old ways of doing business? 
                                o How will the standard business processes be 
                                implemented by DHS components or the shared   
                                service providers to provide consistency      
                                across DHS?                                   
                                o What process will be used to determine and  
                                validate the processes needed for DHS         
                                components that have unique needs?            
Strategy for implementing    o What guidance will be provided to assist    
the shared service           DHS and its component agencies in adopting a  
approach                     change management strategy that reduces the   
                                risks of consolidating systems and migrating  
                                to a shared service provider that uses the    
                                selected financial management systems models? 
                                o What processes will be put in place to      
                                ensure that individual component agency       
                                financial management system investment        
                                decisions focus on the benefits of standard   
                                processes and shared service providers?       
                                o What process will be used to facilitate the 
                                decision-making by component agencies to a    
                                given systems model?                          
                                o How will component agencies incorporate     
                                strategic workforce planning in the migration 
                                approach and consolidation of financial       
                                management systems?                           
Disciplined Processes        o How can existing industry standards and     
                                best practices be incorporated into DHS-wide  
                                guidance related to financial management      
                                system implementation efforts, including      
                                migrating to shared service providers?        
                                o What actions will be taken to reduce the    
                                risks and costs associated with data          
                                conversion and interface efforts?             
                                o What oversight process will be used to      
                                ensure that modernization efforts effectively 
                                implement the prescribed policies and         
                                procedures?   
										  
Appendix IV: Disciplined Processes

Disciplined Processes Are Key to Successful Financial Management System
Implementation Efforts

Disciplined processes have been shown to reduce the risks
           associated with software development and acquisition efforts to
           acceptable levels and are fundamental to successful system
           implementations. A disciplined software implementation process can
           maximize the likelihood of achieving the intended results
           (performance) within established resources (costs) on schedule.
           Although a standard set of practices that will guarantee success
           does not exist, several organizations, such as the Software
           Engineering Institute (SEI) and the Institute of Electrical and
           Electronic Engineers (IEEE), and individual experts have
           identified and developed the types of policies, procedures, and
           practices that have been demonstrated to reduce development time
           and enhance effectiveness. The key to having a disciplined system
           development effort is to have disciplined processes in multiple
           areas, including requirements management, testing, data conversion
           and system interfaces, configuration management, risk management,
           project management, and quality assurance.
			  
           Requirements Management

           Requirements are the specifications that system developers and
           program managers use to design, develop, and acquire a system.
           They need to be carefully defined, consistent with one another,
           verifiable, and directly traceable to higher-level business or
           functional requirements. It is critical that they flow directly
           from the organization's concept of operations (how the
           organization's day-to-day operations are or will be carried out to
           meet mission needs).1

           According to the IEEE,2 a leader in defining the best practices
           for such efforts, good requirements have several characteristics,
           including the following:

           o The requirements fully describe the software functionality to be
           delivered. Functionality is a defined objective or characteristic
           action of a system or component. For example, for grants
           management, a key functionality includes knowing (1) the funds
           obligated to a grantee for a specific purpose, (2) the cost
           incurred by the grantee, and (3) the funds provided in accordance
           with federal accounting standards.
			  
1According to IEEE Std. 1362-1998, a concept of operations document is
normally one of the first documents produced during a disciplined
development effort since it describes system characteristics for a
proposed system from the user's viewpoint. This is important since a good
concept of operations document can be used to communicate overall
quantitative and qualitative system characteristics to the user,
developer, and other organizational elements. This allows the reader to
understand the user organizations, missions, and organizational objectives
from an integrated systems point of view.

2IEEE Std. 830-1998.
			  
           o The requirements are stated in clear terms that allow for
           quantitative evaluation. Specifically, all readers of a
           requirement should arrive at a single, consistent interpretation
           of it.
           o Traceability among various requirement documents is maintained.
           Requirements for projects can be expressed at various levels
           depending on user needs. They range from agencywide business
           requirements to increasingly detailed functional requirements that
           eventually permit the software project managers and other
           technicians to design and build the required functionality in the
           new system. Adequate traceability ensures that a requirement in
           one document is consistent with and linked to applicable
           requirements in another document.
           o The requirements document contains all of the requirements
           identified by the customer, as well as those needed for the
           definition of the system.

           Studies have shown that problems associated with requirements
           definition are key factors in software projects that do not meet
           their cost, schedule, and performance goals. Examples include the
           following:

           o A 1988 study found that getting a requirement right in the first
           place costs 50 to 200 times less than waiting until after the
           system is implemented to get it right.3 
           o A 1994 survey of more than 8,000 software projects found that
           the top three reasons that projects were delivered late, over
           budget, and with less functionality than desired all had to do
           with requirements management.4 
           o A 1994 study found that, on average, there is about a 25-percent
           increase in requirements over a project's lifetime, which
           translates into at least a 25-percent increase in the schedule.5 
           o A 1997 study noted that between 40 and 60 percent of all defects
           found in a software project could be traced back to errors made
           during the requirements development stage.6

3Barry W. Boehm and Philip N. Papaccio, "Understanding and Controlling
Software Costs," IEEE Transactions on Software Engineering, vol. 14, no.
10 (1988).

4The Standish Group, Charting the Seas of Information Technology (Dennis,
Mass.: The Standish Group, 1994).

5Caper Jones, Assessment and Control of Software Risks (Englewood Cliffs,
N.J.: Yourdon Press, 1994).

6Dean Leffingwell, "Calculating the Return on Investment from More
Effective Requirements Management," American Programmer (1997).	

           Testing		  

           Testing is the process of executing a program with the intent of
           finding errors.7 Because requirements provide the foundation for
           system testing, they must be complete, clear, and well documented
           to design and implement an effective testing program. Absent this,
           an organization is taking a significant risk that substantial
           defects will not be detected until after the system is
           implemented. As shown in figure 3, there is a direct relationship
           between requirements and testing.										                                  
7Glenford J. Myers, The Art of Software Testing (New York: John Wiley &
Sons, Inc., 1979).

Figure 3: Relationship between Requirements Development and Testing

Although the actual testing occurs late in the development cycle, test
planning can help disciplined activities reduce requirements-related
defects. For example, developing conceptual test cases based on the
requirements derived from the concept of operations and functional
requirements stages can identify errors, omissions, and ambiguities long
before any code is written or a system is configured. Disciplined
organizations also recognize that planning the testing activities in
coordination with the requirements development process has major benefits.

Although well-defined requirements are critical for implementing a
successful testing program, disciplined testing efforts for projects have
several characteristics,8 which include the following:

           o Testers who assume that the program has errors are likely to
           find a greater percentage of the defects present in the system.
           This is commonly called the testing mindset.
           o Test plans and scripts that clearly define what the expected
           results should be when the test case is properly executed and the
           program does not have a defect that would be detected by the test
           case. This helps to ensure that defects are not mistakenly
           accepted.
           o Processes that ensure test results are thoroughly inspected.
           o Test cases that include exposing the system to invalid and
           unexpected conditions as well as the valid and expected
           conditions. This is commonly referred to as boundary condition
           testing.
           o Testing processes that determine if a program has unwanted side
           effects. For example, a process should update the proper records
           correctly but should not delete other records.
           o Systematic gathering, tracking, and analyzing statistics on the
           defects identified during testing.

           Although these processes may appear obvious, they are often
           overlooked in testing activities.9
			  
			  Data Conversion and System Interfaces

           Data conversion is defined as the modification of existing data to
           enable them to operate with similar functional capability in a
           different environment.10 It is one of the many critical elements
           necessary to successfully implement a new system. Because of the
           difficulty and complexity associated with financial systems data
           conversion, highly skilled staff are needed. There are three
           primary phases in a data conversion:
			  
8Testing covers a variety of activities. The discussion of the testing
processes in this appendix has been tailored to selected aspects of system
implementation efforts and is not intended to provide a comprehensive
discussion of all the processes that are required or the techniques that
can be used to accomplish a disciplined testing process.

9Glendford J. Myers, The Art of Software Testing.

10Joint Financial Management Improvement Program, White Paper: Financial
Systems Data Conversion-Considerations (Washington, D.C.: Dec. 20, 2002).			  

           (1) Pre-conversion activities prior to and leading up to the
           conversion, such as determining the scope and approach or method,
           developing the conversion plan, performing data cleanup and
           validation, ensuring data integrity, and conducting necessary
           analysis and testing.

           (2) Cutover activities to convert the legacy data to the new
           system, such as testing system process and data edits, testing
           system interfaces (both incoming and outgoing), managing the
           critical path, supervising workload completion, and
           reconciliation.

           (3) Post-installation activities such as verifying data integrity,
           conducting final disposition of the legacy system data, and
           monitoring the first reporting cycle.

           There are also specific issues that apply uniquely to converting
           data as part of the replacement of a financial system, including

           o identifying specific open transactions and balances to be
           established,
           o analyzing and reconciling transactions for validation purposes,
           and
           o establishing transactions and balances in the new system through
           an automated or manual process.

           Further, consideration of various data conversion approaches and
           implications are important. Some considerations to be taken into
           account for the system conversion are the timing of the conversion
           (beginning-of-the-year, mid-year, or incremental) and other
           options such as direct or flash conversions, parallel operations,
           and pilot conversions. In addition, agencies should consider
           different data conversion options for different categories of data
           when determining the scope and timelines, such as

           o opting not to conduct a data conversion,
           o processing new transactions and activity only,
           o establishing transaction balances in the new system for
           reporting purposes,
           o converting open transactions from the legacy system, and
           o recording new activity on closed prior year transactions.

           Validation and adjustment of open transactions and data in the
           legacy system are essential prerequisites to the conversion
           process and have often been problematic. When data conversion is
           done right, the new system can flourish. However, converting data
           incorrectly has lengthy and long-term repercussions.

           System interfaces operate on an ongoing basis, linking various
           systems and providing data that are critical to day-to-day
           operations, such as obligations, disbursements, purchase orders,
           requisitions, and other procurement activities. Testing the system
           interfaces in an end-to-end manner is necessary so agencies can
           have reasonable assurance that the system will be capable of
           providing the intended functionality. Systems that lack
           appropriate system interfaces often rely on manual reentry of data
           into multiple systems, convoluted systems, or both. According to
           the SEI, a widely recognized model for evaluating the
           interoperability of systems is the Levels of Information System
           Interoperability. This model focuses on the increasing levels of
           sophistication of system interoperability. Efforts at the highest
           level of this model--enterprise-based interoperability--are
           systems that can provide multiple users access to complex data
           simultaneously, data and applications are fully shared and
           distributed, and data have a common interpretation regardless of
           format. This is in contrast to the traditional interface
           strategies that are more aligned with the lowest level of the SEI
           model. Data exchanged at this level rely on electronic links that
           result in a simple electronic exchange of data.
			  
			  Configuration Management

           According to the SEI, configuration management is defined as a
           discipline applying technical and administrative direction and
           surveillance to (1) identify and document the functional and
           physical characteristics of a configuration item, (2) control
           changes to those characteristics, (3) record and report change
           processing and implementation status, and (4) verify compliance
           with specified requirements.11 The purpose of configuration
           management is to establish and maintain the integrity of work
           products. Configuration management involves the processes of

           o identifying the configuration of selected work products that
           compose the baselines at given points in time,
           o controlling changes to configuration items,
           o building or providing specifications to build work products from
           the configuration management system,
           o maintaining the integrity of baselines, and
           o providing accurate status and current configuration data to
           developers, integrators, and end users.
			  
11IEEE Std. 610-1990.			  

           The work products placed under configuration management include
           the products that are delivered to the customer, designated
           internal work products, acquired products, tools, and other items
           that are used in creating and describing these work products.

           For commercial off-the-shelf (COTS) systems, configuration
           management focuses on ensuring that changes to the requirements or
           components of a system are strictly controlled to ensure the
           integrity and consistency of system requirements or components.
           Two of the key activities for configuration management include
           ensuring that (1) project plans explicitly provide for evaluation,
           acquisition, and implementation of new, often frequent, product
           releases12 and (2) modification or upgrades to deployed versions
           of system components are centrally controlled, and unilateral user
           release changes are precluded. Configuration management recognizes
           that when using COTS products, it is the vendor, not the
           acquisition or implementing organization, that controls the
           release of new versions and that new versions are frequently
           released.
			  
           Risk Management

           Risk and opportunity are inextricably related. Although developing
           software is a risky endeavor, risk management processes should be
           used to manage the project's risks to acceptable levels by taking
           the actions necessary to mitigate the adverse effects of
           significant risks before they threaten the project's success. If a
           project does not effectively manage its risks, then the risks will
           manage the project.

           Risk management is a set of activities for identifying, analyzing,
           planning, tracking, and controlling risks. Risk management starts
           with identifying the risks before they can become problems. If
           this step is not performed well, then the entire risk management
           process may become a useless exercise since one cannot manage
           something that one does not know anything about. As with the other
           disciplined processes, risk management is designed to eliminate
           the effects of undesirable events at the earliest possible stage
           to avoid the costly consequences of rework.
			  
12Donald J. Reifer, Victor R. Basili, Barry W. Boehm, and Betsy Clark,
"COTS-Based Systems--Twelve Lessons Learned about Maintenance."
(Presentation, 3rd International Conference on COTS-Based Software
Systems, Redondo Beach, Calif., Feb. 4, 2004.)

           After the risks are identified, they need to be analyzed so that
           they can be better understood and decisions can be made about what
           actions, if any, will be taken to address them. Basically, this
           step includes activities such as evaluating the impact on the
           project if the risk does occur, determining the probability of the
           event occurring, and prioritizing the risk against the other
           risks. Once the risks are analyzed, a risk management plan is
           developed that outlines the information known about the risks and
           the actions, if any, which will be taken to mitigate those risks.
           Risk monitoring is a continuous process because both the risks and
           actions planned to address identified risks need to be monitored
           to ensure that the risks are being properly controlled and that
           new risks are identified as early as possible. If the actions
           envisioned in the plan are not adequate, then additional controls
           are needed to correct the deficiencies identified.
			  
			  Project Management

           Effective project management is the process for planning and
           managing all project-related activities, such as defining how
           components are interrelated, defining tasks, estimating and
           obtaining resources, and scheduling activities. Project management
           allows the performance, cost, and schedule of the overall program
           to be continually measured, compared with planned objectives, and
           controlled. Project management activities include planning,
           monitoring, and controlling the project.

           Project planning is the process used to establish reasonable plans
           for carrying out and managing the software project. This includes
           (1) developing estimates of the resources needed for the work to
           be performed, (2) establishing the necessary commitments, and (3)
           defining the plan necessary to perform the work. Effective
           planning is needed to identify and resolve problems as soon as
           possible, when it is the cheapest to fix them. According to one
           author, the average project expends about 80 percent of the time
           on unplanned rework--fixing mistakes that were made earlier in the
           project. Recognizing that mistakes will be made in a project is an
           important part of planning. According to this author, successful
           system development activities are designed so that the project
           team makes a carefully planned series of small mistakes to avoid
           making large, unplanned mistakes. For example, spending the time
           to adequately analyze three design alternatives before selecting
           one results in time spent analyzing two alternatives that were not
           selected. However, discovering that a design is inadequate after
           development can result in code that must be rewritten, at a cost
           greater than analyzing the three alternatives in the first place.
           This same author notes that a good rule of thumb is that each hour
           a developer spends reviewing project requirements and architecture
           saves 3 to 10 hours later in the project.13

           Project monitoring and control help to understand the progress of
           the project and determine when corrective actions are needed based
           on the project's performance. Best business practices indicate
           that a key facet of project management and oversight is the
           ability to effectively monitor and evaluate a project's actual
           performance, cost, and schedule against what was planned.14 In
           order to perform this critical task, the accumulation of
           quantitative data or metrics is required and can be used to
           evaluate a project's performance. An effective project management
           and oversight process uses quantitative data or metrics to
           understand matters such as (1) whether the project plan needs to
           be adjusted and (2) oversight actions that may be needed to ensure
           that the project meets its stated goals and complies with agency
           guidance. For example, an earned value management system is one
           metric that can be employed to better manage and oversee a system
           project.15 An earned value management system attempts to compare
           the value of work accomplished during a given period with the work
           scheduled for that period. With ineffective project oversight,
           management can only respond to problems as they arise.

           Agency management can also perform oversight functions, such as
           project reviews and participation in key meetings, to help ensure
           that the project will meet the agency needs. Management can use
           independent verification and validation reviews to provide it with
           assessments of the project's software deliverables and processes.
           Although independent of the developer, verification and validation
           is an integral part of the overall development program and helps
           management mitigate risks. This core element involves having an
           independent third party--such as an internal audit function or a
           contractor that is not involved with any of the system
           implementation efforts--verify and validate that the systems were
           implemented in accordance with the established business processes
           and standards. Doing so provides agencies with needed assurance
           about the quality of the system, which is discussed in more detail
           in the following section.
			  
13Steve McConnell, Software Project Survival Guide (Redmond, Wash.:
Microsoft Press, 1998).

14GAO, Information Technology: DOD's Acquisition Policies and Guidance
Need to Incorporate Additional Best Practices and Controls, [75]GAO-04-722
(Washington, D.C.: July 30, 2004).

15According to Office of Management and Budget Circular No. A-11 S 300.4,
earned value management is a project (investment) management tool that
effectively integrates the investment scope of work with schedule and cost
elements for optimum investment planning and control. Agencies must
demonstrate use of an earned value management system that meets American
National Standards Institute/ Electronic Industries Alliance Standard 748,
for both government and contractor costs, for those parts of the total
investment that require development efforts (e.g., prototypes and testing
in the planning phase and development efforts in the acquisition phase)
and show how close the investment is to meeting the approved cost,
schedule, and performance goals. In addition, agencies must provide an
explanation for any cost or schedule variances that are more than plus or
minus 10 percent.		

           Quality Assurance	  

           Quality assurance is defined as a set of procedures designed to
           ensure that quality standards and processes are adhered to and
           that the final product meets or exceeds the required technical and
           performance requirements. Quality assurance is a widely used
           approach in the software industry to improve upon product delivery
           and the meeting of customer requirements and expectations. The SEI
           indicates that quality assurance should begin in the early phases
           of a project to establish plans, processes, standards, and
           procedures that will add value to the project and satisfy the
           requirements of the project and the organizational policies.
           Quality assurance provides independent assessments, typically
           performed by an independent verification and validation or
           internal audit team, of whether management process requirements
           are being followed and whether product standards and requirements
           are being satisfied. Some of the widely used quality assurance
           activities include defect tracking, technical reviews, and system
           testing.

           o Defect tracking---keeping a record of each defect found, its
           source, when it was detected, when it was resolved, how it was
           resolved (fixed or not), and so on.
           o Technical reviews---reviewing user interface prototypes,
           requirements specifications, architecture, designs, and all other
           technical work products.
           o System testing---executing software for the purpose of finding
           defects, typically performed by an independent test organization
           or quality assurance group.

           According to one author, quality assurance activities might seem
           to result in a lot of overhead, but in actuality, exactly the
           opposite is true.16 If defects can be prevented or removed early,
           a significant schedule benefit can be realized. For example,
           studies have shown that reworking defective requirements, design,
           and code typically consumes 40 to 50 percent of the total costs of
           software development projects.17 An effective quality assurance
           approach is to detect as many defects as possible as early as
           possible to keep the costs of corrections down. However, enormous
           amounts of time can be saved by detecting defects earlier than
           during system testing.
			  
16Steve McConnell, Software Project Survival Guide.

17Steve McConnell, Rapid Development: Taming Wild Software Schedules
(Redmond, Wash.: Microsoft Press, 1996).

           Appendix V: Comments from the Department of Homeland Security
			  
			  Appendix VI: GAO Contacts and Staff Acknowledgments
			  
			  GAO Contacts

           McCoy Williams (202) 512-9095 or [48][email protected] Keith A.
           Rhodes (202) 512-6412 or [49][email protected]
			  
           Acknowledgments

           In addition to the contacts named above, Kay Daly, Assistant
           Director; Chris Martin, Senior-Level Technologist; Chanetta Reed;
           Francine DelVecchio; and Felicia Brooks made key contributions to
           this report.
			  
			  Related GAO Products

           Federal Financial Management: Critical Accountability and Fiscal
           Stewardship Challenges Facing Our Nation. [50]GAO-07-542T .
           Washington, D.C.: March 1, 2007.

           Homeland Security: Applying Risk Management Principles to Guide
           Federal Investments. [51]GAO-07-386T . Washington, D.C.: February
           7, 2007.

           Homeland Security: Management and Programmatic Challenges Facing
           the Department of Homeland Security. [52]GAO-07-398T . Washington,
           D.C.: February 6, 2007.

           High-Risk Series: An Update. [53]GAO-07-310 . Washington, D.C.:
           January 2007.

           Financial Management: Improvements Underway But Serious Financial
           Systems Problems Persist. [54]GAO-06-970 . Washington, D.C.:
           September 26, 2006.

           Information Technology: Improvements Needed to More Accurately
           Identify and Better Oversee Risky Projects Totaling Billions of
           Dollars. [55]GAO-06-1099T . Washington, D.C.: September 7, 2006.

           Enterprise Architecture: Leadership Remains Key to Establishing
           and Leveraging Architectures for Organizational Transformation.
           [56]GAO-06-831 . Washington, D.C.: August 14, 2006.

           Homeland Security: Progress Continues, but Challenges Remain on
           Department's Management of Information Technology. [57]GAO-06-598T
           . Washington, D.C.: March 29, 2006.

           Financial Management Systems: DHS Has an Opportunity to
           Incorporate Best Practices in Modernization Efforts.
           [58]GAO-06-553T . Washington, D.C.: March 29, 2006.

           Financial Management Systems: Additional Efforts Needed to Address
           Key Causes of Modernization Failures. [59]GAO-06-184 . Washington,
           D.C.: March 15, 2006.

           CFO Act of 1990: Driving the Transformation of Federal Financial
           Management. [60]GAO-06-242T . Washington, D.C.: November 17, 2005.

           Information Technology: OMB Can Make More Effective Use of Its
           Investment Reviews. [61]GAO-05-276 . Washington, D.C.: April 15,
           2005.

           Financial Management: Effective Internal Control Is Key to
           Accountability. [62]GAO-05-321T . Washington, D.C.: February 16,
           2005.

           Financial Management: Improved Financial Systems Are Key to FFMIA
           Compliance. [63]GAO-05-20 . Washington, D.C.: October 1, 2004.

           Financial Management Systems: Lack of Disciplined Processes Puts
           Implementation of HHS' Financial Systems at Risk. [64]GAO-04-1008
           . Washington, D.C.: September 23, 2004.

           Financial Management: Department of Homeland Security Faces
           Significant Financial Management Challenges. [65]GAO-04-774 .
           Washington, D.C.: July 19, 2004.

           Information Technology: Homeland Security Should Better Balance
           Need for System Integration Strategy with Spending for New and
           Enhanced Systems. [66]GAO-04-509 . Washington, D.C.: May 21, 2004.

           Executive Guide: Creating Value Through World-class Financial
           Management. [67]GAO/AIMD-00-134 . Washington, D.C.: April 2000.

           Standards for Internal Control in the Federal Government.
           [68]GAO/AIMD-00-21.3.1 . Washington, D.C.: November 1999.
			  
			  GAO's Mission

           The Government Accountability Office, the audit, evaluation and
           investigative arm of Congress, exists to support Congress in
           meeting its constitutional responsibilities and to help improve
           the performance and accountability of the federal government for
           the American people. GAO examines the use of public funds;
           evaluates federal programs and policies; and provides analyses,
           recommendations, and other assistance to help Congress make
           informed oversight, policy, and funding decisions. GAO's
           commitment to good government is reflected in its core values of
           accountability, integrity, and reliability.
			  
           Obtaining Copies of GAO Reports and Testimony

           The fastest and easiest way to obtain copies of GAO documents at
           no cost is through GAO's Web site ( [69]www.gao.gov ). Each
           weekday, GAO posts newly released reports, testimony, and
           correspondence on its Web site. To have GAO e-mail you a list of
           newly posted products every afternoon, go to [70]www.gao.gov and
           select "Subscribe to Updates."
			  
			  Order by Mail or Phone

           The first copy of each printed report is free. Additional copies
           are $2 each. A check or money order should be made out to the
           Superintendent of Documents. GAO also accepts VISA and Mastercard.
           Orders for 100 or more copies mailed to a single address are
           discounted 25 percent. Orders should be sent to:

           U.S. Government Accountability Office 441 G Street NW, Room LM
           Washington, D.C. 20548

           To order by Phone: Voice: (202) 512-6000
			  TDD: (202) 512-2537
			  Fax: (202) 512-6061
			  
			  To Report Fraud, Waste, and Abuse in Federal Programs

           Contact:

           Web site: [71]www.gao.gov/fraudnet/fraudnet.htm
			  E-mail: [72][email protected]
			  Automated answering system: (800) 424-5454 or (202) 512-7470
			  
			  Congressional Relations

           Gloria Jarmon, Managing Director, [73][email protected] (202)
           512-4400 U.S. Government Accountability Office, 441 G Street NW,
           Room 7125 Washington, D.C. 20548
			  
			  Public Affairs

           Paul Anderson, Managing Director, [74][email protected] (202)
           512-4800 U.S. Government Accountability Office, 441 G Street NW,
           Room 7149 Washington, D.C. 20548

(195096)

[76]www.gao.gov/cgi-bin/getrpt?GAO-07-536 .

To view the full product, including the scope
and methodology, click on the link above.

For more information, contact McCoy Williams at (202) 512-9095 or Keith
Rhodes at (202) 512-6412.

Highlights of [77]GAO-07-536 , a report to the Subcommittee on Federal
Financial Management, Government Information, Federal Services, and
International Security, Committee on Homeland Security and Governmental
Affairs, U.S. Senate

June 2007

HOMELAND SECURITY

Departmentwide Integrated Financial Management Systems Remain a Challenge

Since the Department of Homeland Security (DHS) began operations in March
2003, it has faced the daunting task of bringing together 22 diverse
agencies and developing an integrated financial management system to
provide timely, reliable, and useful financial information. GAO was asked
to determine (1) whether DHS has fully developed plans for implementing
and/or migrating to an integrated departmentwide financial management
system, (2) the potential usefulness of the work products received for the
funds spent on the financial modernization effort, and (3) going forward,
how DHS can incorporate best practices into its plans for migrating to an
integrated departmentwide financial management system. GAO interviewed key
DHS officials, reviewed relevant DHS policy and procedure documents, and
analyzed work products related to the financial modernization effort.

[78]What GAO Recommends

To help reduce the risks associated with a departmentwide financial
management system implementation effort, GAO makes six recommendations
focused on the need for DHS to define a departmentwide financial
management strategy and embrace best practices to foster systems
development, including key human capital practices. DHS concurred with
GAO's recommendations.

DHS has not yet developed a financial management strategy and plan to move
forward with its financial management system integration efforts. In early
March 2007, DHS officials issued a plan to address existing internal
control weaknesses, but this plan is at a high level and more detailed
implementation strategies will be necessary to fully address the financial
management systems challenges. With Office of Management and Budget (OMB)
approval, DHS indicated that it has decided to migrate components to
internal service providers using selected financial management systems
models currently in place at two components. However, the components that
DHS is considering have material financial management weaknesses.

The Electronically Managing Enterprise Resources for Government
Effectiveness and Efficiency (eMerge2) program that was expected to
integrate financial management systems across the entire department and
address financial management weaknesses was halted in December 2005. DHS
has stated that it had spent about $52 million in total for the eMerge2
project, including approximately $18 million of contractor costs, but the
department did not provide support for these amounts. According to DHS
officials, several of the work products developed for eMerge2 will be
useful as they move forward with their financial management modernization
efforts, regardless of the strategic financial management direction
ultimately selected by DHS. GAO's review indicated that key work products
are of limited value. The concept of operations did not contain an
adequate description of the legacy systems and a clear articulation of the
vision that should guide the department's improvement efforts, and key
requirements developed for the project are unclear and incomplete.

Consolidation of an entity as large and diverse as DHS poses significant
management challenges, including integrating a myriad of redundant
financial management systems and addressing existing and newly identified
weaknesses in the inherited components. In order for DHS to avoid
long-standing problems that have plagued financial management system
improvement efforts at other agencies and not repeat the failure of
eMerge2, it must adopt solutions that reduce the risks associated with
these efforts to acceptable levels. Based on best practices, there are
four key building blocks that will be critical to DHS's ability to
successfully complete its financial transformation: (1) developing a
concept of operations, (2) defining standard business processes, (3)
developing a migration and/or implementation strategy for DHS components,
and (4) defining and effectively implementing disciplined processes
necessary to properly manage the specific projects. Moreover, effective
human capital management is critical to the success of systems
implementations. Having staff with the appropriate skills is key to
achieving financial management improvements, and managing an
organization's employees is essential to achieving results.

References

Visible links
  34. http://www.gao.gov/cgi-bin/getrpt?GAO-07-310
  35. http://www.gao.gov/cgi-bin/getrpt?GAO-06-553T
  36. http://www.gao.gov/cgi-bin/getrpt?GAO-04-774
  37. http://www.gao.gov/cgi-bin/getrpt?GAO-06-553T
  38. http://www.gao.gov/cgi-bin/getrpt?GAO-04-774
  39. http://www.gao.gov/cgi-bin/getrpt?GAO-06-184
  40. http://www.gao.gov/cgi-bin/getrpt?GAO-06-553T
  41. http://www.gao.gov/cgi-bin/getrpt?GAO-06-184
  42. http://www.gao.gov/
  43. http://www.gao.gov/cgi-bin/getrpt?GAO-06-553T
  44. http://www.gao.gov/cgi-bin/getrpt?GAO/AIMD-00-134
  45. http://www.gao.gov/cgi-bin/getrpt?GAO/AIMD-00-134
  46. http://www.gao.gov/cgi-bin/getrpt?GAO-06-184
  47. http://www.gao.gov/cgi-bin/getrpt?GAO-07-398T
  48. mailto:[email protected]
  49. mailto:[email protected]
  50. http://www.gao.gov/cgi-bin/getrpt?GAO-07-542T
  51. http://www.gao.gov/cgi-bin/getrpt?GAO-07-386T
  52. http://www.gao.gov/cgi-bin/getrpt?GAO-07-398T
  53. http://www.gao.gov/cgi-bin/getrpt?GAO-07-310
  54. http://www.gao.gov/cgi-bin/getrpt?GAO-06-970
  55. http://www.gao.gov/cgi-bin/getrpt?GAO-06-1099T
  56. http://www.gao.gov/cgi-bin/getrpt?GAO-06-831
  57. http://www.gao.gov/cgi-bin/getrpt?GAO-06-598T
  58. http://www.gao.gov/cgi-bin/getrpt?GAO-06-553T
  59. http://www.gao.gov/cgi-bin/getrpt?GAO-06-184
  60. http://www.gao.gov/cgi-bin/getrpt?GAO-06-242T
  61. http://www.gao.gov/cgi-bin/getrpt?GAO-05-276
  62. http://www.gao.gov/cgi-bin/getrpt?GAO-05-321T
  63. http://www.gao.gov/cgi-bin/getrpt?GAO-05-20
  64. http://www.gao.gov/cgi-bin/getrpt?GAO-04-1008
  65. http://www.gao.gov/cgi-bin/getrpt?GAO-04-774
  66. http://www.gao.gov/cgi-bin/getrpt?GAO-04-509
  67. http://www.gao.gov/cgi-bin/getrpt?GAO/AIMD-00-134
  68. http://www.gao.gov/cgi-bin/getrpt?GAO/AIMD-00-21.3.1
  69. http://www.gao.gov/
  70. http://www.gao.gov/
  71. http://www.gao.gov/fraudnet/fraudnet.htm
  72. mailto:[email protected]
  73. mailto:[email protected]
  74. mailto:[email protected]
  75. http://www.gao.gov/cgi-bin/getrpt?GAO-04-722
  76. http://www.gao.gov/cgi-bin/getrpt?GAO-07-536
  77. http://www.gao.gov/cgi-bin/getrpt?GAO-07-536
*** End of document. ***