HUD Information Systems: Improved Management Practices Needed to Control
Integration Cost and Schedule (Letter Report, 12/18/1998,
GAO/AIMD-99-25).
The Department of Housing and Urban Development (HUD) has spent hundreds
of millions of dollars during the past seven years to develop an
integrated financial management system. The system is intended to
provide timely and accurate information to management and enable HUD to
properly manage its financial resources. Although this effort has not
yet been completed, HUD has developed and deployed various modules and
systems for 12 of the 14 projects initiated under the 1993 and 1997
financial systems integration strategies. The agency, however, lacks the
rigorous processes needed to accurately determine how much more it will
cost or how much longer it will take to achieve its financial systems
integration objective, whether its efforts so far have achieved expected
results, or whether its latest strategy is cost beneficial. Cost
increases and schedule delays have been caused by (1) changes to the
financial systems integration strategy that were not supported by
thorough analyses and (2) inadequate project management and oversight.
The Year 2000 computing crisis has also affected the FHA Mortgage
Insurance System effort. Moreover, HUD's latest actions to establish new
financial systems integration management teams and to increase its
project management training program do not address and cannot correct
the root cause of the problems--the lack of a data-driven management
process to properly oversee and control information technology
investments, such as financial systems integration. HUD has yet to
implement a disciplined investment management process to select,
control, and evaluate financial systems integration projects in
accordance with industry best practices and as required by the
Clinger-Cohen Act and the Paperwork Reduction Act.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: AIMD-99-25
TITLE: HUD Information Systems: Improved Management Practices
Needed to Control Integration Cost and Schedule
DATE: 12/18/1998
SUBJECT: Financial management systems
Strategic information systems planning
Housing programs
Information resources management
Private sector practices
Federal agency accounting systems
Cost analysis
IDENTIFIER: HUD Central Accounting and Program System
HUD Integrated Business System
HUD Integrated Disbursement and Information System
Joint Financial Management Improvement Program
HUD Tenant Rental Assistance Certification System
FHA Financial Data Warehouse
HUD Grants Evaluation Management System
FHA Mortgage Insurance System
HUD Real Estate Management System
HUD Executive Information System
HUD Geographic Information System
HUD Financial Systems Integration Plan
HUD Year 2000 Program
Y2K
HUD 2020 Management Reform Plan
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
GAO/AIMD-99-25
Cover
================================================================ COVER
GAO/AIMD-99-25
Report to the Chairman, Subcommittee on Housing and Community
Opportunity, Committee on Banking and Financial Services, House of
Representatives
December 1998
HUD INFORMATION SYSTEMS - IMPROVED
MANAGEMENT PRACTICES NEEDED TO
CONTROL INTEGRATION COST AND
SCHEDULE
GAO/AIMD-99-25
HUD Information Systems
(511436)
Abbreviations
=============================================================== ABBREV
CFO - Chief Financial Officer
EIS - Executive Information System
FFS - Federal Financial System
FHA - Federal Housing Administration
FHAMIS - FHA Mortgage Insurance System
FHEO - Office of Fair Housing and Equal Opportunity
FMFIA - Federal Managers' Financial Integrity Act
FSI - Financial Systems Integration
GEMS - Grants Evaluation Management System
GIS - Geographic Information System
GMS - Grants Management System
HUD - Department of Housing and Urban Development
HUDCAPS - HUD Central Accounting and Program System
HPS - HUD Procurement System
IBS - Integrated Business System
IDIS - Integrated Disbursement and Information System
IRM - Information Resources Management
I-TIPS - Information Technology Investment Portfolio System
JFMIP - Joint Financial Management Improvement Program
OIG - Office of Inspector General
OMB - Office of Management and Budget
PIH - Office of Public and Indian Housing
REMS - Real Estate Management System
SAMS - Single Family Acquired Asset Management System
SEI - Software Engineering Institute
TRACS - Tenant Rental Assistance Certification System
Letter
=============================================================== LETTER
B-278710
December 18, 1998
The Honorable Rick Lazio
Chairman, Subcommittee on Housing
and Community Opportunity
Committee on Banking and Financial Services
House of Representatives
Dear Mr. Chairman:
In 1991, the Department of Housing and Urban Development (HUD)
initiated the Financial Systems Integration (FSI) effort to develop
and deploy an integrated financial and management information system
that would provide timely and accurate information to managers and
enable the department to properly manage its financial resources.
HUD has been working on this effort for the past 7 years, but it has
not yet been completed.
You asked us to identify (1) the initial objectives, development,
deployment and maintenance costs, and completion dates for HUD's FSI
effort and how they have changed, (2) the factors that have
contributed to FSI cost increases and schedule delays, and (3)
whether HUD is following industry best practices and has implemented
provisions of the Clinger-Cohen Act of 1996 and the Paperwork
Reduction Act of 1995 required to manage FSI projects as investments.
In addition, you asked that we identify whether HUD's Year 2000
program will impact its FSI activities.
We reviewed systems integration plans, budget requests, cost-benefit
analyses, current expenditure reports, and project documentation to
identify initial FSI objectives, development and deployment cost and
schedule estimates, and how they have changed. We met with project
managers and reviewed audit reports and project files to identify
factors that had contributed to FSI cost increases and schedule
delays and determine whether effective actions had been taken to
address these problems. To determine whether HUD is using best
practices in managing its information technology investments, we
compared the department's investment management practices to industry
best practices, provisions of the Clinger-Cohen Act and the Paperwork
Reduction Act, and the Office of Management and Budget's and our
information technology investment guides. Finally, we discussed
changes to FSI plans, strategy, cost and schedule estimates, and
HUD's investment management practices and Year 2000 effort with
officials of HUD's Office of the Chief Financial Officer, Office of
Information Technology, Office of the Inspector General, and program
offices.
As agreed with your office, we did not independently verify the
accuracy of FSI cost or schedule data provided by HUD. Also, the
scope of our review was not intended to, and does not, provide a
basis for concluding whether or not HUD's FSI efforts will achieve
their intended results. We performed our work at HUD headquarters in
Washington, D.C., from October 1997 through October 1998 in
accordance with generally accepted government auditing standards.
More details of our objectives, scope and methodology are included in
appendix I. We requested comments on a draft of this report from the
Secretary of Housing and Urban Development or his designee. The HUD
Chief Financial Officer provided us with written comments that are
discussed in the "Agency Comments and Our Evaluation" section of the
report and are reprinted in appendix II.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
While HUD's primary FSI objective of implementing an integrated
financial management system has remained the same, the underlying
strategy for achieving this objective--and consequently the estimated
development and deployment costs\1 and completion dates--have changed
significantly. In 1991 HUD approved a plan to replace about 100
financial and mixed systems--which support both management and
financial information needs--with nine new standard integrated
systems. At that time, HUD estimated that it would cost about $103
million to develop and deploy the nine systems by September 1998. In
1993, HUD abandoned its plan to develop nine new systems and
significantly revised its FSI strategy. The 1993 strategy required
HUD to develop a core financial system and the program offices to
develop new mixed systems to support their business and financial
needs and integrate them with the core system. HUD estimated that it
would cost about $209 million to develop and deploy the new mixed
systems and core financial system by December 1998. Under the 1993
strategy, HUD fully developed and deployed three FSI systems that
support the department's Section 8 financial and program management
functions, community planning and development grant programs, and
procurement process. However, much work remained to develop and
fully deploy the core financial system.
In 1997, HUD revised its FSI strategy again, extending the date for
fully deploying the core financial management system to October 1999
and incorporating the development and deployment of additional new
systems required to meet the department's latest management reforms
and organizational changes. However, the department did not
adequately assess the costs or benefits of the 1997 FSI strategy. As
a result, HUD has no assurance that it has selected the most
cost-beneficial solution to accomplish its FSI objectives. Also, HUD
has not yet finalized the detailed project plans or cost and schedule
estimates for this effort.
Without final plans and estimates to complete the systems integration
effort, FSI costs are uncertain. For example, HUD's cost estimates
through September 1999 have fluctuated considerably, with the latest
estimate being $239 million. However, the $239 million estimate does
not include at least $132 million associated with maintaining FSI
systems, and an earlier HUD estimate of FSI costs totaled $540
million. Until HUD finalizes its plans and cost and schedule
estimates to complete the 1997 strategy, the expected FSI cost will
remain uncertain.
Currently, the nine systems included in the 1997 FSI strategy are in
various stages of development and deployment. For instance, the
department has developed and deployed all the modules for the Office
of Public and Indian Housing's Integrated Business System and has
completed development of the second module for the Office of Fair
Housing and Equal Opportunity Grants Evaluation Management System.
HUD has also developed and deployed a consolidated departmental
general ledger and deployed a prototype of its executive information
system.
Revisions to the systems integration strategy and management and
oversight problems associated with individual projects are factors
that have contributed to FSI cost increases and schedule delays to
date. HUD's recent actions to address management and oversight
problems have included establishing various committees to manage and
oversee FSI projects and increasing project management training.
However, these actions may not be effective because HUD does not have
the essential processes to properly select, control, and evaluate
individual FSI projects. The problems underlying cost increases and
schedule delays for individual FSI projects could have been detected
earlier and minimized if HUD had implemented an effective process to
manage its information technology investments.
HUD has not yet fully implemented a complete, disciplined information
technology investment management process, which includes selecting,
controlling, and evaluating FSI projects and conforms with best
practices and related requirements in the Clinger-Cohen Act and the
Paperwork Reduction Act. The department's investment selection
decisions have not been based on current and complete project data,
such as actual versus estimated costs, benefits, schedules, and risk
assessments. In addition, HUD has not implemented (1) an adequate
process to control information technology projects once they have
been selected for implementation or (2) a process to evaluate
information technology projects and determine whether they have
achieved expected benefits. Without a complete and disciplined
information technology investment management process, HUD does not
have adequate assurance that it is selecting the right projects or
maximizing its return on investment.
HUD's Year 2000\2 program, a top priority effort that must be
completed on time, may further impact the FSI effort. For instance,
HUD recently suspended systems integration work on three
mission-critical FSI systems so the department could focus its
resources on completing Year 2000 software renovations.
--------------------
\1 HUD's 1991 and 1993 FSI cost estimates do not include systems
operations or maintenance costs and, therefore, do not represent
life-cycle cost estimates.
\2 For the past several decades, automated information systems have
typically represented the year using two digits rather than four in
order to conserve electronic data storage space and reduce operating
costs. In this format, however, 2000 is indistinguishable from 1900
because both are represented as "00." As a result, if computer
systems or applications that use dates or perform date or
time-sensitive calculations are not modified, they may generate
incorrect results beyond 1999.
BACKGROUND
------------------------------------------------------------ Letter :2
HUD is the principal federal agency responsible for programs dealing
with housing, community development, and fair housing opportunities.
Its mission includes making housing affordable through the Federal
Housing Administration's (FHA) mortgage insurance for multifamily
housing, providing rental assistance for about 4.5 million lower
income residents, helping to revitalize over 4,000 localities through
community development programs, and encouraging home ownership by
providing mortgage insurance. HUD is one of the nation's largest
financial institutions, responsible for managing more than a reported
$454 billion in mortgage insurance and, as of September 30, 1997, a
reported $531 billion in guarantees of mortgage-backed securities.
For fiscal year 1998, the agency's budget authority was about $24
billion, and its information technology budget was $222 million.
HUD's major program areas are managed by the Office of Housing, which
includes FHA's insurance and project-based rental assistance
programs; the Office of Community Planning and Development, which
includes programs for Community Development Block Grants, empowerment
zones/enterprise communities, and assistance for the homeless; the
Office of Public and Indian Housing, which provides funds to help
operate and modernize public and Indian housing and administers
tenant-based rental assistance programs; and the Office of Fair
Housing and Equal Opportunity, which is responsible for investigating
complaints and ensuring compliance with fair housing laws.
In 1984, we reported that HUD lacked adequate information and
financial management systems necessary to ensure accountability for,
and control over, departmental programs.\3 In 1989, HUD was involved
in highly publicized scandals that included instances in which
private real estate agents were able to steal millions of dollars by
retaining the proceeds from the sale of FHA-owned properties, rather
than transferring the funds to the Treasury. In 1992, we reported
that these scandals were attributed, in large part, to fundamental
deficiencies in the department's information and financial management
systems.\4
In particular, HUD's systems were inadequate, lacked credibility and
internal controls, and failed to meet program managers' needs or
provide adequate support for oversight of housing and community
development programs.
To address fundamental deficiencies in the department's information
and financial systems and meet the requirements of the Chief
Financial Officers Act of 1990, which called for financial management
reform across the federal government, the Secretary of Housing and
Urban Development initiated a number of actions. These actions
included the appointment of a Chief Financial Officer (CFO) to
oversee the department's financial operations and initiation of a
major Financial Systems Integration (FSI) effort to strengthen its
financial management systems. Although HUD proceeded with this high
priority effort, it continued to be affected by poorly integrated,
ineffective, and generally unreliable information systems that did
not satisfy management needs or provide adequate support to control
housing and community development programs.
In 1994, we designated the department a high-risk area, in part
because of its inadequate information and financial management
systems and slow progress in correcting fundamental management
weaknesses that had allowed the 1989 scandals to occur.\5 By 1997, we
reported that HUD had formulated approaches and initiated actions to
address departmentwide deficiencies, including information and
financial management systems problems, but many of these actions were
far from being completed.\6 In the meantime, HUD continues to rely on
unintegrated and inadequate program and financial management systems,
some of which are not yet Year 2000 compliant.
Recognizing the need to better manage information technology, recent
legislative reforms--the Clinger-Cohen Act of 1996, the Paperwork
Reduction Act of 1995, and the Federal Acquisition Streamlining Act
of 1994--provide guidance to federal agencies on how to plan, manage,
and acquire information technology as part of their overall
information resources management (IRM) responsibilities. These
legislative reforms highlight the need to ensure that IRM programs
and decisions are integrated with organizational planning, budgeting,
and financial management.
--------------------
\3 Increasing the Department of Housing and Urban Development's
Effectiveness Through Improved Management, Vol. I (GAO/RCED-84-9,
January 10, 1984).
\4 HUD Reforms: Progress Made Since the HUD Scandals but Much Work
Remains (GAO/RCED-92-46, January 31, 1992).
\5 Improving Government: Actions Needed to Sustain and Enhance
Management Reforms (GAO/T-OCG-94-1, January 27, 1994).
\6 High-Risk Series: Department of Housing and Urban Development
(GAO/HR-97-12, February 1997).
INTEGRATION COSTS HAVE
INCREASED AND EXPECTED
COMPLETION HAS BEEN DELAYED
------------------------------------------------------------ Letter :3
While HUD revised its FSI plan in 1993 and again in 1997, its primary
objective-- implementing an integrated financial management system to
meet the department's program and financial management
needs--remained unchanged. At the same time, HUD's implementation
strategy and the cost and schedule estimates to develop and deploy
FSI continue to change. HUD has not yet finalized the cost and
schedule estimates for its 1997 FSI strategy and has not performed
the detailed analyses needed to determine whether the strategy is
cost beneficial.
HUD'S 1991 FSI PLAN
---------------------------------------------------------- Letter :3.1
HUD adopted its first Financial Management Systems Strategic
Integration Plan in November 1991 to address and resolve material
weaknesses in its financial systems. In this plan, HUD acknowledged
that inadequate and unintegrated financial management systems
rendered it unable to properly manage its programs and financial
resources. The plan's primary objective was, therefore, to implement
an integrated financial management system that would meet the
department's program and financial management needs. The 1991 plan
contained specific objectives to establish sound financial management
controls, correct material weaknesses, improve financial management,
provide timely and accurate information to managers to enable them to
meet their organizational objectives, meet the goals of section 4 of
the Federal Managers' Financial Integrity Act (FMFIA) of 1982 and
comply with the Office of Management and Budget's (OMB) Circular
A-127.\7
HUD's strategy for achieving the FSI objectives was to replace about
100 separate financial and mixed systems with nine new fully
integrated systems. This strategy was based on an analysis which
concluded that HUD did not have the basic financial management
systems to serve as the foundation for an integrated systems
environment. The design of the 1991 FSI plan required that eight
financial systems be integrated with the new core accounting system
and this design was consistent with the Joint Financial Management
Improvement Program's (JFMIP) framework for financial management
systems.\8 Once the nine new standard systems were deployed, program
offices were to use them to support their business operations. In
addition, the plan noted that it would be necessary to make interim
improvements to existing systems, since the integration effort would
be a long-term project. The interim improvements would be needed to
manage programs, comply with legal mandates, and correct material
weaknesses until the nine new systems became available. The
department estimated that it would cost about $103 million to develop
and deploy the nine systems called for in the 1991 FSI plan by
September 1998. Table 1 shows the objectives, estimated development
and deployment costs, and scheduled deployment dates for the nine
planned integrated systems.
Table 1
Nine New Fully Integrated Systems in
1991 FSI Plan
Estimated
development
and deployment Scheduled
Planned system Objective cost\a deployment date
-------------- ------------------------------ -------------- ----------------
Core To serve as the central $8 million March 1996
Accounting accounting system, capturing,
System recording, controlling, and
summarizing the financial
results of operations for each
business area and the entire
department.
Mortgage To provide the capability to $19 million September 1996
Insurance plan, implement, administer,
System and evaluate all of HUD's
activities resulting from
mortgage insurance operations.
Administrative To provide administrative $3 million December 1996
Accounting accounting functionality for
System all business areas, and
support HUD's investment
management, budget
formulation, and planning
activities.
Mortgage To provide the capability to $2 million January 1997
Backed administer and evaluate all of
Securities HUD's activities involving
System mortgage backed securities.
Grants/ To provide the capability to $25 million September 1997
Subsidies/ budget, administer, and
Loan System evaluate HUD's activities
related to grants, subsidies,
and loans.
Debt To provide the ability to $3 million September 1998
Management/ perform debt collection
Collection activities for delinquent
System debts.
Note/Loan To provide the capability to $5 million September 1998
Servicing service all single-family and
System multifamily mortgage notes as
well as all other loans held
and administered by HUD.
Project/ To provide the capability to $6 million September 1998
Recipient plan, track, and report
Monitoring results of project/recipient
System monitoring activities for FHA-
insured and noninsured
multifamily projects, as well
as for all recipients of funds
from grants, subsidies, and
loans.
Management To serve as the department's $9 million September 1998
Information management reporting system by
System giving senior management
timely access to financial and
management information on all
programs and organizations.
--------------------------------------------------------------------------------
\a These estimates do not include $23 million for the development and
deployment of software applications intended to support functions
such as lender approval and property disposition management, which
are common to multiple program offices.
Source: HUD.
According to HUD's CFO, in fiscal years 1992 and 1993, HUD spent
about $58 million\9 on FSI. Specifically, $48 million was spent on
interim improvements to legacy systems, and $10 million was spent on
the Core Accounting System and Mortgage Insurance System projects.
During this period, HUD (1) procured Federal Financial System (FFS),
a commercial off-the-shelf software package to serve as the
department's Core Accounting System, (2) halted its efforts to
develop the Mortgage Insurance System as a result of poor planning,
and (3) terminated the Grants, Subsidies, and Loans project because
the department could not streamline its grants process. Work on the
remaining six projects was not scheduled to begin until 1994.
--------------------
\7 Section 4 of FMFIA requires agencies to report whether their
accounting systems conform to the accounting principles and standards
mandated by the Comptroller General of the United States; OMB's
Circular A-127 required agencies to develop and maintain a single,
integrated financial management system.
\8 Framework for Federal Financial Management Systems: Federal
Financial Management System Requirements (JFMIP, FFMSR-0, January
1995).
\9 According to the Office of the CFO, these expenditures do not
include staffing costs that may have been incurred by the program
offices.
1993 REVISION OF HUD'S 1991
FSI STRATEGY
---------------------------------------------------------- Letter :3.2
In September 1993, HUD fundamentally changed its FSI strategy. The
revised strategy was in response to (1) slow progress in implementing
systems integration, (2) the need to comply with revisions to OMB
Circular A-127, and (3) senior management's serious doubt about the
viability of creating nine new fully integrated systems and having
program offices adapt their business operations to meet the
requirements of these systems by 1998.
The 1993 FSI plan included the same primary objective of implementing
an integrated financial management system to meet the department's
program and financial management needs (which was delineated in the
1991 FSI plan), as well as objectives for improving HUD's financial
systems and bringing HUD into compliance with the provisions of FMFIA
and the revised OMB Circular A-127. In addition, the 1993 plan
included new objectives, such as eliminating the department's
financial systems from OMB's list of high-risk areas for management
improvement, being consistent with HUD's reform plan,\10 and meeting
the requirements of the Government Performance and Results Act of
1993.\11
The implementation strategy for the 1993 FSI plan was markedly
different from that of the 1991 plan. Under the 1993 FSI plan, the
CFO's office was required to complete the core financial system
project initiated under the 1991 strategy and program offices were
required to develop (1) new systems that would support program
management priorities, financial and management information needs,
and business needs and (2) integrate these systems with the core
financial system. Plans for the remaining eight standard systems
called for in the 1991 strategy were cancelled. As in 1991, the
conceptual design of the 1993 FSI strategy was consistent with the
Joint Financial Management Improvement Program's requirement that the
core financial system receive data from other financial and mixed
systems. By 1996, HUD had initiated 10 systems integration projects.
Despite differences in the strategies, HUD did not perform a
cost-benefit analysis on the 1993 strategy. Therefore, HUD had no
assurance that it had selected the most cost-beneficial solution for
FSI. In 1995, the department estimated that the development and
deployment cost of the 1993 strategy would be about $209 million.
Also, the department extended the deployment date to December 1998, 3
months after the initial scheduled completion date of September 1998.
Table 2 describes the 10 major projects under the 1993 FSI plan,
estimated development and deployment costs, and initial scheduled
deployment dates.
Table 2
10 Major Projects Initiated Under 1993
FSI Plan
Estimated
development
Major FSI and deployment Scheduled
projects Objective costs deployment date
-------------- ------------------------------ -------------- ----------------
Section 8 HUD To support the Office of $13 million March 1996
Central Public and Indian Housing
Accounting and Section 8 business processes,
Program System including budget formulation;
(HUDCAPS)\a applications and contract
processing; funds allocation
and control; payment
processing; housing authority
monitoring; information
management; and electronic
data interchange.
Integrated To develop an automated system $26 million February 1996
Disbursement for the Office of Community
and Planning and Development's
Information formula grant programs and
System (IDIS) provide grantees with reports
on projects and activities.
Tenant Rental To develop an integrated $21 million January 1997
Assistance financial management system to
Certification support Housing's management
System of its Section 8 programs,
(TRACS) resolve existing material
weaknesses, and automate (1)
contract processing, (2)
tenant eligibility
certification and
recertification, (3)
budgeting/funding control, and
(4) subsidy payment
verification.
Grants To automate a departmentwide $12 million September 1997
Management grants management process.
System (GMS)
HUD To replace four legacy $3 million October 1997
Procurement procurement systems, automate
System (HPS) the transmission of customer
procurement plans and
requests, track the status of
procurement activities for the
entire contract life-cycle,
and improve procurement
reporting.
Integrated To integrate most of the $25 million March 1998
Business Office of Public and Indian
System (IBS) Housing's 19 mixed systems to
better meet business needs and
correct existing problems.
Budget To provide the Office of $5 million July 1998
Formulation Budget the capability to meet
System (EZ the informational requirements
Budget System) of the department, the
Congress, and OMB in a timely
manner. This system should (1)
improve the accuracy of HUD's
budget and accounting data,
and (2) automate manually
intensive portions of the
budget process.
Grants To integrate the Office of $550,000 September 1998
Evaluation Fair Housing and Equal
Management Opportunity grants management
System (GEMS) process.
HUDCAPS\a To replace HUD's general $35 million September 1998
ledger systems (the Government
National Mortgage Association,
FHA, and the Program
Accounting System/Line of
Credit Control System) with a
single integrated core
financial system.
FHA Mortgage To (1) fully integrate the 35 $68 million December 1998
Insurance individual systems that
System support FHA's mortgage
(FHAMIS) insurance data and functions,
(2) support single family,
multifamily, and Title I
pro\gram areas, (3) improve
data integrity and
productivity and (4) enhance
internal controls.
--------------------------------------------------------------------------------
\a HUDCAPS has two subsystems (1) Section 8 Accounting, which
supports the Office of Public and Indian Housing's financial and
program management functions, and (2) Administrative Accounting,
which supports the department's core financial system requirements.
Source: HUD.
From fiscal years 1994 through 1997, the department spent about $181
million to develop, deploy, and maintain various functions of the 10
major FSI projects, according to the CFO. These expenditures are in
addition to the $58 million reportedly spent on FSI between fiscal
years 1992 and 1993 and do not include additional FSI costs that may
have been incurred by program offices. According to the Office of
the CFO and FSI project managers, the status of the 10 systems
integration projects under the 1993 FSI plan is as follows:
-- The Office of Public and Indian Housing deployed the HUD Central
Accounting and Program System (HUDCAPS) in fiscal year 1995 to
support its tenant-based Section 8 program.
-- The Office of Community Planning and Development's Integrated
Disbursement and Information System (IDIS) was deployed and was
being used to monitor an estimated 950 community development
grantees as of September 1998.\12
-- The Office of Housing deployed three of the four Tenants Rental
Assistance Certification System (TRACS) modules since fiscal
year 1997. These modules are being used for contract
processing, tenant voucher processing, and budget development
and analysis. The Office of Housing also developed and deployed
a computer income-matching module to verify the income of
tenants receiving rental subsidy from the department.
-- The departmental Grants Management System (GMS) project was
terminated in 1997.\13
-- In April 1997, the Office of Administration deployed the HUD
Procurement System (HPS), which is being used to track and
manage the department's procurement activities.
-- The Office of Public and Indian Housing (PIH) deployed five of
six modules of the Integrated Business System (IBS) as of
February 1998. This system is being used by PIH and the Office
of Native Americans to monitor their programs, including
information related to all housing authorities in the country.
-- The Office of Fair Housing and Equal Opportunity deployed one of
the three modules for its Grants Evaluation Management System
(GEMS) in fiscal year 1996.
-- The Office of Budget deployed the first module of its Budget
Formulation System in May 1997. This system is used by the CFO
to formulate, prepare and monitor the annual budget.
-- The CFO's Office deployed a portion of HUDCAPS to support the
department's administrative accounting functions in fiscal year
1994.
-- The Office of Housing's Federal Housing Administration Mortgage
Insurance System (FHAMIS) deployed a data warehouse\14 for
multifamily data in March 1997, developed information strategy
plans\15 and performed business process reviews to lay the
foundation for future FHAMIS systems development efforts.
--------------------
\10 In 1994, the Secretary announced the HUD Reinvention Blueprint
Plan, intended to reinvent and transition the department from a
"lumbering bureaucracy to a streamlined partner with state and local
governments."
\11 The Government Performance and Results Act of 1993 requires
agencies to set goals, measure performance, and report on their
accomplishments.
\12 We are currently reviewing internal controls related to IDIS as
part of another congressional request.
\13 The departmental GMS project was terminated after HUD had spent
over $500,000, because the Secretary's legislative proposal to
streamline the department's programs into fewer programs was not
approved by the Congress, and program offices could not agree upon a
standard grants management process necessary to define the functional
requirements of the system.
\14 A data warehouse is a central repository of program and financial
data.
\15 An information strategy plan documents the strategic
opportunities, goals, and critical success factors and information
needs of a specific business function, and describes how information
technology could be used to better meet goals and improve business
processes.
FSI REVISED AGAIN IN 1997
---------------------------------------------------------- Letter :3.3
In 1997, HUD again revised its FSI strategy after concluding that (1)
it could not fully deploy HUDCAPS--the core financial system--by
September 1998 and (2) the systems integration effort had to conform
to the HUD 2020 Management Reform Plan.\16 According to the CFO,
HUDCAPS could not be completed as scheduled. Specifically, the
system had been deployed on schedule to support the department's
administrative and Section 8 accounting functions, but it had not
been deployed to replace the three remaining general ledger systems
as planned. Also, program offices had not yet developed the
interfaces between the mixed systems and the core financial system.
The CFO also stated that the revised FSI strategy had to conform with
the HUD 2020 Management Reform Plan, which established an October
1999 deadline for fully deploying an integrated core financial system
and called for repairing or replacing the department's existing mixed
systems and developing new systems to support the reforms. The
primary objective of the 1997 FSI plan was consistent with the 1991
and 1993 FSI plans--to implement an integrated financial management
system, consisting of both financial and mixed systems, that would
provide the information necessary to carry out financial and
programmatic missions of the department. However, as in 1993, HUD
did not perform any cost-benefit analyses despite the additional
systems and schedule changes required by the plan. As a result, HUD
cannot assure that the 1997 strategy is the most cost-beneficial
alternative.
As of May 1998, the CFO had identified nine projects for the 1997 FSI
strategy, including five that were started under the 1993 FSI plan
(i.e., HUDCAPS, FHAMIS, GEMS, IDIS, and IBS), and four new projects
required to support HUD's 2020 Management Reform Plan. The objective
of the HUDCAPS project, however, was expanded to centralize the
development of interfaces between mixed systems and the core
financial management system.\17 New projects in the 1997 plan include
developing interfaces between HUD's geographic information system\18
and data warehouses, and deploying an executive information system,
Real Estate Management System, and FHA's Financial Data Warehouse.
The cost and schedule estimates to complete the 1997 FSI strategy
have not been finalized, and the FSI cost estimate through September
1999 has fluctuated considerably. For example, HUD's FSI estimate
has varied from $540 million in June 1998, to $255 million on
November 12, 1998, to $239 million a week later. However, we found
that the $255 million and the $239 million estimates do not include
at least $132 million associated with maintaining FSI systems. Until
HUD finalizes its plans and cost and schedule estimates to complete
the 1997 strategy, the expected FSI cost will remain uncertain. In
May 1998, the Office of the CFO said that the cost and schedule
estimates to complete the 1997 FSI strategy would be finalized by
September 30, 1998. However, as of October 19, 1998, these estimates
had not been approved. Table 3 displays the 9 projects included in
the 1997 FSI strategy as of May 1998, as well as their corresponding
initial development and deployment cost and schedule estimates.
Table 3
Major FSI Projects in 1997 Strategy
Initial
estimated Initial
development scheduled
and deployment deployment
Major projects Objective costs\a date\a
-------------- ------------------------------ -------------- ----------------
Integrated The same objective as in 1993, $26 million February 1996
Disbursement i.e., to develop an automated
and system for the Office of
Information Community Planning and
System (IDIS) Development's formula grant
programs and provide grantees
with reports on projects and
activities.
Integrated The same objective as in 1993, $25 million March 1998
Business i.e., to integrate most of the
System (IBS) Office of Public and Indian
Housing's 19 mixed systems to
better meet business needs and
correct existing problems.
FHA Financial To develop and deploy a data $688,365 July 1998
Data Warehouse warehouse required to transfer
FHA legacy system transactions
into HUDCAPS.
Grants The same objective as in 1993, $550,000 September 1998
Evaluation i.e., to integrate the Office
Management of Fair Housing and Equal
System (GEMS) Opportunity grants management
process.
FHA Mortgage The same objective as in 1993, $68 million December 1998
Insurance i.e., to (1) fully integrate
System the 35 individual systems that
(FHAMIS) support FHA's mortgage
insurance data and functions,
(2) support single-family,
multifamily, and Title I
program areas, (3) improve
data integrity and
productivity, and (4) enhance
internal controls.
Real Estate To deploy a fully integrated $4.2 million September 1999
Management database containing
System (REMS) multifamily housing data that
supports the Enforcement
Centers, the Real Estate
Assessment Center, and the
Section 8 Financial Management
Center.
HUD Central The same objective as in 1993, Not yet October 1999
Accounting and i.e., to replace HUD's general determined.
Program System ledger systems (the Government
(HUDCAPS) National Mortgage Association,
FHA, and the Program
Accounting System/Line of
Credit Control System) with a
single integrated core
financial system, and an
additional objective--to
develop the software
interfaces necessary to
integrate the existing and
planned mixed systems with the
core financial system.
Executive To provide necessary program $2 million October 1999
Information and financial management data
System (EIS) to HUD management.
Geographic To enable users to access $5 million October 1999
Information selected HUD program and
System (GIS) financial data by geographic
area, and display them either
in maps or tables.
--------------------------------------------------------------------------------
\a These are HUD's initial cost and schedule estimates which have not
been revised to reflect the 1997 FSI strategy.
Source: HUD.
HUD has been working to implement the 1997 FSI strategy. The CFO
stated that a large number of FSI systems or system modules have been
deployed and are being used to manage and monitor the department's
programs. According to the CFO, the status of the nine 1997 systems
integration projects as of October 1998 is as follows:
-- In addition to the work completed under the 1993 FSI plan, the
Office of Community Planning and Development has also deployed
the Integrated Disbursement and Information System (IDIS) in
nine states and the District of Columbia.\19
-- In addition to the work completed under the 1993 FSI plan, the
Office of Public and Indian Housing has developed and deployed
the sixth module to support the 1993 Integrated Business System
(IBS) requirements and implemented a new module to support the
business requirements of the Office of Native American programs.
-- The FHA Financial Data Warehouse is still being developed by the
Office of Housing. Therefore, the July 1998 estimated
deployment date was not met.
-- In addition to the work completed under the 1993 FSI plan, the
Office of Fair Housing and Equal Opportunity (FHEO) deployed an
enhanced version of the first module for the Grants Evaluation
Management System (GEMS) that supports the pre-award process and
completed the development of the second module to support
grantee tracking in fiscal year 1998. This system is being used
by FHEO to monitor its two major grant programs.
-- In addition to the work completed under the 1993 FSI plan and to
carry out Federal Housing Administration Mortgage Insurance
System (FHAMIS) goals in the information strategy plans and
resulting from business process reviews, the Office of Housing
deployed and is using the Single Family Premium Collection
Subsystem to collect and account for premiums. The Office of
Housing also deployed the Single Family Data Warehouse and a
multifamily data quality system to support its FHAMIS project.
-- The Office of Housing deployed the first phase of the Real
Estate Management System (REMS) in March 1998. The Office of
Housing is using REMS to collect and monitor data related to all
multifamily structures in the department.
-- In addition to the work completed under the 1993 FSI plan, the
Office of the CFO has developed and deployed a consolidated
HUD-wide general ledger for fiscal year 1999 that will include
summary transactions for the department, including FHA and the
Government National Mortgage Association; and developed an
interface to the Office of Public and Indian Housing's Section 8
HUDCAPS.
-- The Office of the CFO developed and deployed the first phase of
the department's Executive Information System (EIS). This
system was prototyped using selected data from HUD's program and
financial systems.
-- The Office of the CFO deployed HUD's Community 2020 geographic
information system (GIS) to provide program and management
information in a geographical referenced format to users of
HUD's programs.
During a November 9, 1998 meeting, the Deputy Secretary and CFO told
us that HUD is in the process of assessing the conceptual design for
a departmental grants management system. While the study has not yet
been completed, the officials stated that if a departmental grants
management system is deployed, some of the functions performed by
IDIS may be replaced by the new system.
--------------------
\16 The HUD 2020 Management Reform Plan, released by the Secretary in
June 1997, set forth a sweeping set of proposals intended to make
various organizational and operational changes and address management
weaknesses.
\17 Under the 1993 FSI strategy, individual program offices were
responsible for developing the interfaces between mixed systems and
the core financial system.
\18 A geographic information system is designed to assemble, store,
manipulate, and display geographically referenced data (i.e., data
that are associated with specific places, such as the location of a
community receiving funding from HUD).
\19 According to House Report 105-769 (October 5, 1998), HUD shall
not require additional states to implement the IDIS until problems
associated with it are corrected.
STATUS OF 1997 FSI STRATEGY
IMPLEMENTATION
---------------------------------------------------------- Letter :3.4
Project management plans help managers monitor projects and ensure
that activities are completed within specified costs and schedules.
HUD's system development methodology\20 specifically requires that
project plans be developed to document project activities and cost
and schedule estimates before a project is initiated. The
methodology also requires that project plans be updated if project
objectives change or significant budget or schedule variances occur.
Although HUD extended the HUDCAPS implementation date by 13 months
from September 1998 to October 1999, the department does not yet have
a final project plan that shows whether it can successfully deploy
the core financial system and integrate it with mixed systems by the
new target date. The 1998 HUDCAPS project plan included tasks,
costs, and schedules for fiscal year 1998 activities, but the 1999
plan does not include a schedule that shows key milestones, tasks,
task dependencies, and a critical path demonstrating how and when
fiscal year 1999 activities necessary to integrate HUDCAPS with the
mixed systems will be completed by October 1999. For example, the
HUDCAPS project plan does not show how or when the FHA Financial Data
Warehouse will be interfaced to the core financial system. In
addition, HUD has not yet finalized project plans that are necessary
to establish new milestones for FSI projects, such as (1) GEMS, which
missed its initial scheduled completion date, and (2) FHAMIS, which
will not meet its initial scheduled completion date.\21 These plans
should include tasks, task dependencies, and a critical path, as well
as development and deployment cost and schedule estimates for
individual FSI projects.
According to the Director of IRM Planning and Management, the
department required that detailed project plans be developed for each
FSI project by September 30, 1998. However, as of October 19, 1998,
these plans had not yet been finalized. In addition, the Director of
IRM Planning and Management expressed concern over the quality of the
project plans that had been submitted. This is an important matter
since the department has spent hundreds of millions of dollars on FSI
and expects to deploy an integrated core financial management system
that will rely extensively on data from the mixed systems by October
1999.
--------------------
\20 System Development Methodology, U.S. Department of Housing and
Urban Development (March 1997, Release 6.0).
\21 According to the project manager, software problems encountered
with the first GEMS module delayed the development and deployment of
the last module. According to the project manager, the FHAMIS
project will not meet the initial deployment date for several
reasons, including HUD's Year 2000 effort.
INEFFECTIVE MANAGEMENT AND
OVERSIGHT HAVE CONTRIBUTED TO
COST INCREASES AND SCHEDULE
DELAYS
------------------------------------------------------------ Letter :4
Ineffective project management and oversight have contributed to
numerous problems resulting in FSI cost increases and schedule
delays. In 1994, we reported that HUD did not adequately oversee the
planning and development of individual FSI projects.\22 As a result,
the first two FSI projects suffered delays and rising project costs.
To resolve these problems, we recommended that HUD strengthen the
management and oversight of individual FSI projects to ensure that
significant problems would be brought to the attention of senior
managers and corrected in a timely manner. We stated that these
measures must continue throughout the integration effort.
Between 1993 and 1997, HUD formed various committees to strengthen
project management and the oversight of projects initiated under the
1993 FSI plan. Nevertheless, we found that ineffective project
management and oversight continued to contribute to cost increases
and schedule delays on individual projects. For example, the Single
Family Acquired Asset Management System (SAMS) replacement system
which was developed and deployed as part of the FHAMIS 1993 FSI
project, was delivered late and over budget and did not meet critical
user needs because it was poorly managed. HUD estimated that the
SAMS replacement system would be developed and deployed for about
$3.2 million in 6 months. However, HUD awarded the contract to
develop SAMS before adequately defining the system's requirements.
As a result, the cost of SAMS grew tenfold to over $32 million, the
system was deployed 10 months late, and the system did not meet some
critical user needs. To meet these needs, HUD was forced to spend an
additional $8 million to enhance the system.
In April 1997, the HUD OIG also cited inadequate project management
and oversight as factors that contributed to the cost increases and
schedule delays for several projects initiated under the 1993 FSI
plan. The OIG recommended that the Deputy Secretary take over the
direction of FSI to provide the needed management oversight and
ensure that project managers receive adequate project management
training. In February 1998, HUD responded to these recommendations
by establishing new management teams to strengthen the oversight of
FSI projects and increasing its project management training program.
However, as discussed below, HUD lacks (1) the essential disciplined
processes required to effectively manage and oversee FSI projects and
other information technology investments and (2) objective data to
identify and resolve problems as they arise.
--------------------
\22 HUD Information Resources: Strategic Focus and Improved
Management Controls Needed (GAO/AIMD-94-34, April 14, 1994).
FSI PROJECTS ARE NOT BEING
MANAGED AS INVESTMENTS
------------------------------------------------------------ Letter :5
The department is not using recognized best practices for selecting,
controlling, and evaluating its investments as required by the
Clinger-Cohen Act of 1996 and the Paperwork Reduction Act of 1995.\23
The problems HUD has experienced in developing and deploying an
integrated financial management system are a direct result of not
managing information technology projects properly as investments.
HUD's investment selection process is not complete and has not
provided decisionmakers with key information necessary to make
investment decisions and monitor investments. For example,
decisionmakers have not had reliable, up-to-date information on
project costs, benefits, and risks to make well-informed decisions.
Further, HUD lacks an adequate process for monitoring and controlling
its FSI investments and does not have a process for evaluating FSI
information technology investments once they have been completed.
Therefore, the department cannot fully (1) determine whether its
investments have achieved expected benefits, (2) identify whether
major differences have occurred between actual and expected results
in terms of cost, schedule, and risks, or (3) revise its investment
management processes on the basis of lessons learned. As a result,
the department does not know whether it is making the right
investments, how to control these investments effectively, or whether
these investments have provided expected mission-related benefits
within estimated costs.
In reviewing HUD's investment management process, we also found that
the preparation of software cost estimates--key data required to make
good investment decisions--is not consistent with best practices.
Also, HUD does not follow best practices requiring that cost-benefit
analyses be updated to reflect the current status of investments.
--------------------
\23 Best practices refers to information technology practices of
leading private and public sector organizations which are highlighted
in our Executive Guide: Improving Mission Performance Through
Strategic Information Management and Technology--Learning From
Leading Organizations (GAO/AIMD-94-115, May 1994).
LEGISLATIVE REFORMS AND BEST
PRACTICES REQUIRE A
DISCIPLINED PROCESS TO
SELECT, CONTROL, AND
EVALUATE INVESTMENTS
---------------------------------------------------------- Letter :5.1
The Clinger-Cohen Act of 1996 and the Paperwork Reduction Act of 1995
require agency heads to implement an approach that maximizes the
value and assesses and manages the risks of information technology
investments. The acts stipulate that this approach be integrated
with the agency's budget, financial, and program management
processes.
An information technology investment process is an integrated
approach that provides for data-driven selection, control, and
evaluation of information technology investments.\24 The investment
process is comprised of three phases. The first phase involves
selecting investments using quantitative and qualitative criteria for
comparing and setting priorities for information technology projects.
The second phase includes monitoring and controlling selected
projects through progress reviews at key milestones to compare the
expected costs, risks, and benefits of earlier phases with the actual
costs incurred, risks encountered, and performance benefits realized
to date. These progress reviews are essential for senior managers to
decide whether to continue, accelerate, modify, or terminate a
selected project. The third phase involves a post-implementation
review or evaluation of fully implemented projects to compare actuals
against estimates, assess performance, and identify areas where
future decision-making can be improved. Overall, information from
one phase is used to support activities in other phases.
Reliable cost estimates are also needed to allow effective investment
decision-making. OMB's Circular A-130\25 requires agencies to
prepare cost-benefit analyses for system development projects and
update them as necessary throughout the life of the systems.
--------------------
\24 This process is documented in our Assessing Risks and Returns: A
Guide for Evaluating Federal Agencies' IT Investment Decision-making,
Version 1 (GAO/AIMD-10.1.13, February 1997) and OMB's Evaluating
Information Technology Investments: A Practical Guide, Office of
Management and Budget, Version 1.0, November 1995.
\25 OMB Circular A-130 states that agencies shall "conduct
cost-benefit analyses to support ongoing management oversight
processes that maximize return on investment and minimize financial
and operational risk for investments in major information systems on
an agency-wide basis."
HUD'S PROCESS FOR SELECTING
INVESTMENTS IS INADEQUATE
---------------------------------------------------------- Letter :5.2
As stated in our investment guide, proposed investments should be
screened to ensure that they meet minimum acceptance criteria, such
as a return-on-investment thresholds, linkage to an organization's
strategic objectives and compliance with an organization's
information technology architecture. Projects that pass the
screening process undergo an in-depth analysis. To help make good
decisions on information technology investments, best practices
require that the in-depth analysis be based on accurate, reliable,
up-to-date project information. This information includes
cost-benefit analyses, risk assessments, and implementation plans for
both new and ongoing projects. Once the information is analyzed,
projects are ranked based on their relative benefits, costs, and
risks. This ranking should determine which projects should be funded
and is the essence of information technology portfolio analysis.
After investment decisions have been made, schedules should be
established at key milestones to regularly monitor and track the
cost, schedule, benefits, and risks of selected projects.
In 1997, HUD implemented a new process to improve how the department
screens, ranks, and selects information technology investments.
First, proposed investments were screened to determine if they met
explicit criteria described in the department's strategic ranking
mechanism document. Although these criteria called for information
on the duration of the project, cost for fiscal year 1997 through
full deployment, technical risks, and impact on HUD's mission and
customer needs, the screening criteria did not include
return-on-investment thresholds or full life-cycle cost estimates as
required by OMB's guidance on evaluating information technology
investments.
Investment proposals were then analyzed, scored and ranked using the
same criteria and data that were used in the screening process.
However, the screening criteria did not require accurate and complete
data on life-cycle costs, benefits, risks, project schedules or the
corresponding analyses that were conducted to develop these
estimates. For example, after reviewing investment proposal data for
six FSI projects, we found that only two included life-cycle cost
estimates and none included cost-benefit analyses or risk mitigation
plans. Therefore, HUD made investment decisions without the
information needed for a thorough understanding of the projects to
make the necessary trade-offs among them. Finally, HUD's selection
process was also insufficient since it did not establish project
review schedules for selected projects as required by best practices.
In fiscal year 1998, HUD did not use its selection process because
the Secretary required that investment decisions be based on whether
the proposed projects supported the department management and
organizational changes called for in the HUD 2020 Management Reform
Plan. As in fiscal year 1997, decisions made in 1998 were not based
on reliable estimates of life-cycle costs, benefits, and return on
investment. HUD is using its fiscal year 1997 selection process to
make fiscal year 1999 investment decisions.\26 In addition, HUD has
deployed the Information Technology Investment Portfolio System
(I-TIPS)--a generic system developed by the Department of Energy--to
automate and support the management of information technology capital
planning and integrate this planning with the department's budget
process. HUD is using I-TIPS to support its processes to screen,
score, and select information technology proposals for fiscal year
1999 and plans to use I-TIPS when making investment decisions for
fiscal year 2000. However, because HUD has not corrected its
selection process and still does not require complete, accurate, and
current data to select information technology investments, there is
no assurance that HUD's 1999 investments decisions will be better
than they have been in the past.
--------------------
\26 As of October 8, 1998, HUD had not finalized its information
technology investment selections for fiscal year 1999.
HUD LACKS DISCIPLINED
PROCESSES TO CONTROL AND
EVALUATE SELECTED
INFORMATION TECHNOLOGY
INVESTMENTS
---------------------------------------------------------- Letter :5.3
Once information technology projects are selected, they should be
consistently monitored and controlled through progress reviews at key
milestone dates.\27
Progress reviews should assess several aspects of the project,
including deliverables, methodology, technical issues, schedule,
costs, benefits, and risks. Further, once a project has been fully
implemented, it should be evaluated through post-implementation
reviews. The post-implementation reviews should provide (1) a
project assessment, including an evaluation of customer/user
satisfaction and how well the project met its estimated cost and
schedule and provided mission-related benefits, and (2) lessons
learned so that the investment decision-making processes can be
improved.
HUD does not have an adequate process to control investments or a
process to evaluate investments. In 1997, the Technology Investment
Board working group was established to monitor approved projects and
advise the Technology Investment Board executive committee whether to
continue, modify, or terminate them.\28 However, the Director for IRM
Planning and Management stated that the working group mostly monitors
annual project expenditures and the rate of expenditure for any given
fiscal year. This degree of oversight is not adequate because it is
not based on the project-specific measures required to effectively
monitor and control information technology projects. These measures
include (1) an accumulation of actual cost data and comparisons to
estimated cost levels, (2) a comparison of the estimated and actual
schedule, (3) a comparison of expected and actual benefits realized,
and (4) an assessment of risks. The information should be regularly
collected, updated, and provided to decisionmakers to support
effective project monitoring.
The department also lacks a method for evaluating investments and
thus does not perform post-implementation reviews or use lessons
learned to improve the investment process. HUD's Director for IRM
Planning and Management acknowledged these weaknesses in both the
control and evaluation phases of the investment process and added
that HUD plans to define these processes by the spring of 1999,
before it deploys future releases of I-TIPS. Without processes to
control and evaluate investments, HUD cannot (1) determine if
projects should be modified, continued, accelerated, or terminated,
(2) determine whether a project has met its objectives, (3) compare
projected costs and schedules to actual costs incurred and
implementation dates, and (4) identify ways to modify or improve its
investment management process.
--------------------
\27 See footnote 24 and the Clinger-Cohen Act of 1996.
\28 The Technology Investment Board working group did not meet for a
period of 6 months--from August 1997 through January 1998.
HUD'S COST ESTIMATING
PROCESSES DO NOT MEET
ACCEPTED PRACTICES
---------------------------------------------------------- Letter :5.4
Reliable cost estimates are essential for making effective
information technology investment decisions. The reliability of cost
estimates is dependent on the thoroughness and discipline of an
organization's estimating processes. Consistently producing reliable
estimates requires defined institutional processes for deriving cost
estimates, archiving them, and measuring actual performance against
them.
Based on its research of leading government and private-sector
estimating practices, Carnegie Mellon University's Software
Engineering Institute\29 (SEI) identified six requisites for
developing cost estimates.\30 According to SEI, an organization must
have all six requisite processes to consistently produce reliable
cost estimates. These requisites are the following:
-- a corporate memory (or historical database), which includes cost
estimates, revisions, reasons for revisions, actuals, and
relevant contextual information;
-- structured processes for estimating software size and the amount
and complexity of existing software that can be reused;
-- cost models calibrated and tuned to reflect demonstrated
accomplishments on similar past projects;
-- audit trails that record and explain the values used as cost
model inputs;
-- processes for dealing with externally imposed cost or schedule
constraints in order to ensure the integrity of the estimating
process; and
-- data collection and feedback processes that foster capturing and
correctly interpreting data from work performed.
The Director of HUD's Systems Engineering Group stated that the
department's processes do not satisfy SEI's software cost estimating
criteria. As shown in table 4, HUD's cost estimating processes for
FSI projects partially meet one, but do not meet the remaining five
institutional process requisites that experts say are embedded in
leading information technology development and acquisition
organizations.
Table 4
HUD's Conformance With SEI's
Institutional Requisites
Conformance of HUD
institutional policies
SEI requisites and practices
---------------------------------------------- ----------------------
Corporate memory Partial
Sizing and reuse structure No
Extrapolation using actual performance No
Audit trails No
Integrity in dealing with dictated limits No
Data collection and feedback on actual No
performance
----------------------------------------------------------------------
According to the Director of the Systems Engineering Group, HUD uses
its experience in working with the program offices on software
development efforts, rather than cost models, to develop cost
estimates. The director acknowledged that HUD does not have an
automated historical database to use when developing estimates for
new FSI projects; instead, separate project files are kept with
historical data on individual projects. The director was unsure of
the usefulness of these files because they are not updated to
identify and correct inconsistencies. Finally, HUD does not update
or regularly review its initial cost estimates. As a result, HUD
does not have adequate assurance that FSI cost estimates are
consistently reliable. This increases the risk of poor FSI
investment decisions throughout the project's life cycle and the
likelihood of additional cost overruns.
--------------------
\29 SEI is a nationally recognized, federally funded research and
development center established to address software development
issues.
\30 Checklists and Criteria for Evaluating the Cost and Schedule
Estimating Capabilities of Software Organizations (CMU/SEI-95-SR-005,
January 1995).
HUD'S COST-BENEFIT ANALYSES
ARE NOT UPDATED
---------------------------------------------------------- Letter :5.5
OMB Circular A-130 requires agencies to prepare cost-benefit analyses
for systems development projects and update them as necessary
throughout the life of the systems. HUD's systems development
methodology has required the preparation and updating of cost-benefit
analyses since at least September 1992.
In reviewing three cost-benefit analyses for ongoing FSI projects, we
found that none had been updated as required in OMB Circular A-130.
According to several FSI project managers, cost-benefit analyses are
performed only once--to initiate a new information technology
project. The project managers stated that the analyses are not
updated, although they do prepare yearly project funding requests as
part of the budget process. These requests, however, do not reflect
any changes to the costs or benefits of a project. Therefore, HUD
cannot compare current cost estimates and actual expenditures to
determine whether unfavorable cost or benefit variances exist. As a
result, HUD may continue to invest in a system without knowing
whether costs or benefits have changed enough to warrant
discontinuing further investment.
The Office of the Inspector General found similar problems in 1996
and recommended that HUD's Office of Information Technology establish
guidance and define management responsibilities for updating the
cost-benefit analysis at appropriate intervals. HUD responded to
this recommendation by stating that its September 1995 Benefit/Cost
Analysis Methodology, Volume I and Benefit/Cost Analysis Workbook,
Volume II define and guide the development of the required components
of a cost-benefit analysis and management's responsibility for
periodically updating an analysis. The requirement to use and
document cost-benefit analyses in accordance with the methodology and
workbook was included in the March 1997 revision of HUD's system
development methodology.
According to HUD's IRM Director for Planning and Management, although
the system development methodology requires the use of both the
cost-benefit analysis methodology and the workbook, the department
has not officially mandated the use of either one. In addition,
several FSI project managers told us that these standards are
generally not followed. The director added that the quality, depth,
and documentation supporting cost-benefit analyses for FSI projects
have been inconsistent. For example, we found the IDIS cost-benefit
analysis was well documented and included a discussion of the
assumptions and constraints used in performing the analysis,
information on recurring and nonrecurring costs, and the estimated
life-cycle cost of the system. In contrast, we found that the
cost-benefit analysis for the Office of Fair Housing and Equal
Opportunity Grants Evaluation Management System was inadequate
because it did not quantify benefits.
YEAR 2000 COMPUTING CRISIS MAY
FURTHER IMPACT FSI
------------------------------------------------------------ Letter :6
FSI cost and schedule estimates may be impacted by HUD's Year 2000
program, a priority effort that must be completed on time. In March
1998, we reviewed the status of HUD's Year 2000 effort and reported
that 42 of 63 mission-critical systems were not yet Year 2000
compliant.\31 HUD has attempted to mitigate its Year 2000 risks, but
three mission-critical FHAMIS systems undergoing renovations,
testing, and certification are behind schedule. To better ensure
that these mission-critical systems are corrected on time, HUD
suspended systems integration work on these systems so that the
department could focus its resources on completing Year 2000 software
renovations. According to the project manager, this will cause a
major impact to the schedule for completing the FHAMIS systems
integration work.
In commenting on this report, HUD stated that it successfully
completed all of its Year 2000 renovations for both mission-critical
and nonmission-critical systems. HUD expects to complete the Year
2000 certification and validation process by January 31, 1999.
--------------------
\31 Year 2000 Computing Crisis: Strong Leadership Needed to Avoid
Disruption of Essential Services, (GAO/T-AIMD-98-117, March 24,
1998).
CONCLUSIONS
------------------------------------------------------------ Letter :7
HUD has spent hundreds of millions of dollars on its efforts to
develop and deploy an integrated financial management system over the
past 7 years. While this effort has not yet been completed, the
department has developed and deployed various modules and systems for
12 of the 14 different projects initiated under the 1993 and 1997 FSI
strategies. The department, however, does not have the rigorous
processes needed to accurately determine how much more it will cost
or how much longer it will take to achieve the FSI objective, whether
its efforts to date have achieved expected results, or whether its
latest strategy is cost beneficial. HUD has not yet finalized
project plans or cost and schedule estimates for completing all of
the components of the latest FSI plan. Without such plans, the
department is likely to continue to spend millions of dollars more,
miss milestones, and still not fully meet its objective of developing
and fully deploying an integrated financial management system.
Cost increases and schedule delays have been caused by (1) changes to
the FSI strategy that were not supported by thorough analyses and (2)
inadequate project management and oversight. In addition, the Year
2000 computing crisis has impacted the schedule for the FHAMIS
effort. Further, HUD's latest actions to establish new FSI
management teams and increase its project management training program
do not address and cannot correct the root cause of the problems--the
lack of a data-driven management process to properly oversee and
control information technology investments such as FSI.
HUD has not yet implemented a disciplined investment management
process to select, control, and evaluate FSI projects in accordance
with industry best practices and as required by the Clinger-Cohen Act
and the Paperwork Reduction Act. In the absence of such a process,
HUD decisionmakers (1) continue to make FSI investment decisions
without reliable, complete, and up-to-date data on expected and
actual costs, benefits, and risks, (2) cannot adequately monitor and
control investments and detect and correct problems early, and (3)
cannot evaluate completed projects to determine whether they have
achieved expected benefits and improve the investment management
process based on lessons learned. Also, HUD does not have
well-defined, structured cost estimating processes that are in
accordance with industry best practices for developing reliable
software cost estimates. Finally, HUD does not follow best practices
since it does not require that cost estimates or cost-benefit
analyses be updated periodically for decision-making purposes.
RECOMMENDATIONS
------------------------------------------------------------ Letter :8
In order to strengthen FSI management and oversight and HUD's
information technology investment management decisions, we recommend
that the Secretary of Housing and Urban Development ensure that the
department takes the following actions:
-- Prepare complete and reliable estimates of the life-cycle costs
and benefits of the overall 1997 FSI strategy and individual FSI
projects. In addition, HUD should
finalize the detailed project plan for the core financial management
system (HUDCAPS) to establish the milestones, tasks, task
dependencies, a critical path, and staffing requirements and
demonstrate that it is cost-effective to meet the October 1999
scheduled implementation date called for in HUD's 2020 Management
Reform Plan and
finalize detailed project plans for individual FSI projects (mixed
systems) that establish the milestones, tasks, task dependencies and
critical paths, and staffing requirements to complete the 1997 FSI
strategy.
-- Fully implement and institutionalize a disciplined and
documented process consistent with provisions of the
Clinger-Cohen Act and the Paperwork Reduction Act, as well as
our and OMB's guidance for selecting, controlling, and
evaluating information technology investments. This process
should, at a minimum, include steps to
select information technology investments based on complete,
accurate, reliable, and up-to-date project-level information,
including estimated life-cycle costs, expected benefits, projected
schedule, and risks;
conduct formal in-process reviews at key milestones in a project's
life cycle--including comparing actual and estimated project costs,
benefits, schedule, and risks--and provide these results to
decisionmakers, who will determine whether to continue, accelerate,
modify, or terminate FSI projects; and
initiate post-implementation reviews within 12 months of deployment
to compare completed project cost, schedule, and benefits with
original estimates and provide the results of these reviews to
decisionmakers so that improvements can be made to HUD's information
technology investment and management processes.
-- Develop and use defined processes for estimating FSI costs. At
a minimum, these processes should include the following SEI
requisites:
a corporate memory (or historical database), which includes cost and
schedule estimates, revisions, reasons for revisions, actuals, and
relevant contextual information;
structured processes for estimating software size and the amount and
complexity of existing software that can be reused;
cost models calibrated to reflect demonstrated accomplishments on
similar past projects;
audit trails that record and explain the values used as cost model
inputs;
processes for dealing with externally imposed cost or schedule
constraints in order to ensure the integrity of the estimating
process; and
data collection and feedback processes that foster capturing and
correctly interpreting data from work performed.
AGENCY COMMENTS AND OUR
EVALUATION
------------------------------------------------------------ Letter :9
In commenting on a draft of this report, HUD agreed that the
management and oversight of FSI could be improved by fully
implementing and institutionalizing the provisions of the
Clinger-Cohen Act and the Paperwork Reduction Act. In this regard,
HUD agreed with our recommendations to implement defined processes
for selecting, controlling, and evaluating its information technology
investments and for estimating costs. The department also said it
agreed that it needs to prepare complete life-cycle costs and
benefits estimates for its systems strategy, but it did not
specifically address our recommendation to finalize the detailed
project plans for HUDCAPS and other individual FSI projects included
in the 1997 strategy.
HUD expressed concern that the $540 million FSI estimate through
fiscal year 1999 mentioned in our draft report included non-FSI costs
and that a more accurate FSI estimate would be approximately $255
million. As noted in our report, HUD has not yet finalized the
plans, cost, and schedule to complete its current FSI strategy and,
therefore, FSI costs continue to be uncertain. Accordingly, HUD's
estimates through September 1999 have fluctuated considerably, as
reflected in various documents received from the CFO and his staff.
For example, cost estimates have changed from $540 million reported
by HUD in June 1998, to $255 million cited in the department's
November 12, 1998 comments to our draft report, to $239 million that
HUD reported a week later. However, we found that the $255 million
and the $239 million estimates do not include at least $132 million
associated with maintaining FSI systems. HUD's continuing
uncertainty as to what is the FSI cost estimate through September
1999 further demonstrates the department's need to develop and use
well-defined cost estimating processes to prepare reliable cost
estimates.
HUD said our report does not properly compare like systems when
making year-to-year comparisons. The question we were asked to
address was to identify the initial objectives, development,
deployment and maintenance costs, and completion dates for HUD's FSI
effort and how they have changed. In order to respond to that
question, we describe the systems and the estimated systems costs
that were included as part of the three plans and strategies for
achieving integrated financial management systems and carefully
explain that HUD's underlying strategy to implement an integrated
financial management system has changed three times. In addition, to
avoid any misunderstandings, we added language to clarify what the
estimates for the FSI strategies and the expected FSI costs include
through fiscal year 1999.
Finally, HUD described its FSI accomplishments and stated that our
conclusions do not summarize or emphasize the importance of actions
taken to improve its mission-critical financial management systems.
To address this issue, we noted the actions taken by HUD to date and
added information to our discussion of various FSI systems throughout
the report.
---------------------------------------------------------- Letter :9.1
We are sending copies of this report to the Vice Chair and the
Ranking Minority Member of the Subcommittee on Housing and Community
Opportunity, House Committee on Banking and Financial Services, and
the Chairman and Ranking Minority Member of the Subcommittee on Human
Resources, House Committee on Government Reform and Oversight. We
are also providing copies to the Secretary of Housing and Urban
Development and the Director of the Office of Management and Budget.
We will make copies available to others upon request.
Please contact me at (202) 512-6253 or by e-mail at
[email protected] if you have any questions concerning this
report. Major contributors to this report are listed in appendix
III.
Sincerely yours,
Joel C. Willemssen
Director, Civil Agencies Information Systems
OBJECTIVES, SCOPE, AND METHODOLOGY
=========================================================== Appendix I
Our objectives were to identify (1) the initial objectives,
development, deployment and maintenance costs, and completion dates
for HUD's FSI effort and how they have changed, (2) the factors that
have contributed to FSI cost increases and schedule delays, and (3)
whether HUD is following industry best practices and has implemented
provisions of the Clinger-Cohen Act of 1996 and the Paperwork
Reduction Act of 1995 required to manage FSI projects as investments.
We were also asked to identify whether HUD's Year 2000 program would
impact its FSI activities.
To identify the objectives, development and deployment costs, and
completion dates for HUD's initial FSI effort and how they have
changed, we reviewed the 1991, 1993, and 1997 FSI plans. To identify
initial and revised cost and schedule estimates for major FSI
projects, we reviewed initial cost-benefit analyses, project plans,
budget documents provided to the OMB and FSI cost estimates for
fiscal years 1998 and 1999 provided by HUD's CFO. To identify cost
and schedule estimates to complete HUD's 1997 FSI strategy, we met
with program managers for each of the major systems integration
projects and representatives from the Office of the CFO. We also
reviewed the fiscal years 1998 and 1999 project management plans to
deploy an integrated core financial management system (HUDCAPS).
To determine whether FSI costs had increased, we reviewed the (1)
initial FSI development and deployment cost estimates reported in
both the 1991 and 1993 FSI plans, (2) OMB budget submissions, and (3)
CFO's reports on actual systems integration expenditures between
fiscal years 1992 and 1997 and development, deployment and
maintenance cost estimates for fiscal years 1998 and 1999. To
determine whether schedule delays had occurred, we identified the
initial scheduled deployment dates for major FSI projects--including
HUDCAPS, FHAMIS, GEMS, IBS, IDIS, and TRACS--and met with their
respective project managers to determine whether those dates had been
or would be met. We met with project managers for FHAMIS, GEMS,
HUDCAPS, IBS, IDIS, and TRACS and officials from the Office of the
CFO and reviewed audit reports to determine what factors had
contributed to FSI cost increases and schedule delays. We reviewed
HUD's responses to audit recommendations to determine whether HUD had
taken any actions to address management problems. Further, we
discussed these actions with FSI project managers and with OIG
officials to determine whether or not they had been effectively
implemented. To determine whether addressing Year 2000 requirements
would impact FSI cost and schedule estimates, we met with project
managers for individual FSI projects. We also reviewed reports on
the status of HUD's Year 2000 effort to determine whether this effort
would affect the development and deployment schedule of any FSI
project.
To determine whether HUD was following best practices in managing FSI
projects as investments, we compared HUD's information technology
investment procedures and information resources management policies
with criteria in our guidance Assessing Risks and Returns: A Guide
for Evaluating Federal Agencies' Information Technology Investment
Decision-making (GAO/AIMD-10.1.13, February 1997), OMB's guidance
Evaluating Information Technology Investments: A Practical Guide
(November 1995), and OMB's Capital Programming Guide (July 1997), as
well as provisions of the Clinger-Cohen Act of 1996 and the Paperwork
Reduction Act of 1995. We determined whether HUD was following best
practices for selecting investments by reviewing (1) criteria used to
make information technology investment decisions during fiscal years
1997 through 1999, (2) documents for individual FSI projects that
were used to make investment decisions for fiscal years 1997 and
1998, (3) HUD's information technology investment portfolio for
fiscal years 1997 and 1998, and (4) minutes from the Technology
Investment Board working group, which document meetings on FSI
investments. Further, we met with key officials of HUD's Office of
the CFO and Office of Information Technology to obtain additional
details on the investment management process and the department's
plans to implement the Information Technology Investment Portfolio
System.
In addition, we compared the processes and practices HUD used to
develop FSI project cost estimates with the key components of cost
estimating practices publicized by Carnegie Mellon University's SEI.
We also reviewed cost-benefit analyses for several major FSI projects
and met with FSI project managers to determine whether these analyses
had been updated, as required by OMB Circular A-130.
We did not independently verify the accuracy of FSI cost or schedule
data provided by HUD. Also, the scope of our review was not intended
to, and does not, provide a basis for concluding whether or not HUD's
FSI efforts will achieve their intended results.
(See figure in printed edition.)Appendix II
COMMENTS FROM THE DEPARTMENT OF
HOUSING AND URBAN DEVELOPMENT
=========================================================== Appendix I
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
The following are GAO's comments on the Department of Housing and
Urban Development's letter dated November 12, 1998.
GAO COMMENTS
1. As discussed in the "Agency Comments and Our Evaluation" section,
HUD agreed with most of our recommendations. HUD also stated that
the department has had a structured process in place since 1990 for
selecting information and technology investments and monitoring the
major system development through the Technology Investment Board
Executive Committee, which is chaired by the Secretary. We reviewed
HUD's recent selection and control processes beginning with fiscal
year 1997 and found that both processes are incomplete and inadequate
to make sound investment decisions and properly manage selected
investments. The major deficiencies we found with HUD's processes
were that (1) investment decisions were made without reliable,
complete, up-to-date project level information and (2) project
oversight was not based on project-specific measures required to
effectively monitor and control information technology projects.
2. HUD provided us with a copy of the HUDCAPS project plan for
fiscal year 1998 activities and a plan for fiscal year 1999
activities, but the second year project plan was not presented to us
as a draft. Furthermore, as we discuss in the report, the fiscal
year 1999 HUDCAPS plan was not complete because it did not include a
schedule that showed key milestones, tasks, task dependencies, and a
critical path demonstrating how HUDCAPS would be completed and
interfaced with the mixed systems by October 1999.
3. We added a sentence to the conclusions that summarizes the status
of HUD's FSI effort to date, and we expanded the report's discussion
of individual FSI projects to reflect the new information provided by
HUD.
4. We revised the report to indicate that HUD reported that it
completed Year 2000 renovation work for all of its mission-critical
and nonmission-critical systems.
5. We incorporated additional language in our report to avoid any
misunderstanding between what is included in (1) estimates for the
FSI plans and (2) expected FSI costs through fiscal year 1999.
6. Discussed in the "Agency Comments and Our Evaluation" section.
As noted in our report, HUD has not yet finalized the plans, cost,
and schedule to complete its current FSI strategy and, therefore, FSI
costs continue to be uncertain. In addition, HUD's FSI cost estimate
through September 1999 has varied considerably, as reflected in
various letters received from the CFO. For example, FSI cost
estimates have changed from $540 million reported by HUD in June
1998, to $255 million reported on November 12, 1998, to $239 million
reported a week later. However, the $255 million and $239 million
estimates do not include at least $132 million in maintenance costs.
HUD's continuing uncertainty regarding the FSI cost estimate through
September 1999 further demonstrates the department's need to develop
and use well-defined cost estimating processes for preparing reliable
FSI cost estimates. Finally, as we note in appendix I, we did not
independently verify the accuracy of FSI cost data provided by HUD.
HUD's statement that the $103 million for the 1991 FSI strategy
includes development costs only is inconsistent with its 1991 FSI
plan, which states that the $103 million included both development
and deployment costs.
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================= Appendix III
ACCOUNTING AND INFORMATION
MANAGEMENT DIVISION, WASHINGTON,
D.C.
David G. Gill, Assistant Director
Yvette R. Banks, ADP Telecommunications Analyst-in-Charge
Madhav S. Panwar, Senior Technical Advisor
Teresa L. Jones, Issue Area Assistant
*** End of document. ***