Land Management Systems: Major Software Development Does Not Meet BLM's
Business Needs (Testimony, 03/04/99, GAO/T-AIMD-99-102).
The Bureau of Land Management (BLM) spent more than 15 years and
invested about $411 million to plan and develop the Automated Land and
Mineral Record System Project, only to have the major software component
(ALMRS Initial Operating Capability) fail. As a result, the Bureau has
decided not to deploy the software component at this time. GAO has
reported earlier on the significant problems and risks that BLM has
encountered. GAO has made many recommendations to reduce those risks,
but BLM has been slow to implement some recommendations and has not yet
fully implemented others. BLM now needs to determine whether it can
salvage any of the more than $67-million it reportedly invested in the
ALMRS Initial Operating Capability software by analyzing the software to
determine if it can be cost-beneficially modified to meet BLM's needs.
In addition, to reduce the risk that future efforts will result in
failure, BLM should assess its information technology investment
practices and systems acquisition capabilities.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: T-AIMD-99-102
TITLE: Land Management Systems: Major Software Development Does
Not Meet BLM's Business Needs
DATE: 03/04/99
SUBJECT: Land management
Management information systems
Computer software
Strategic information systems planning
Computer software verification and validation
ADP procurement
Information resources management
Systems design
Systems conversions
IDENTIFIER: BLM Automated Land and Minerals Record System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. This text was extracted from a PDF file. **
** Delineations within the text indicating chapter titles, **
** headings, and bullets have not been preserved, and in some **
** cases heading text has been incorrectly merged into **
** body text in the adjacent column. Graphic images have **
** not been reproduced, but figure captions are included. **
** Tables are included, but column deliniations have not been **
** preserved. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
AI99102t.book GAO United States General Accounting Office
Testimony Before the Subcommittee on Interior and Related
Agencies, Committee on Appropriations, House of Representatives
For Release on Delivery Expected at 9 a. m. Thursday, March 4,
1999
LAND MANAGEMENT SYSTEMS Major Software Development Does Not Meet
BLM's Business Needs
Statement of Joel C. Willemssen Director, Civil Agencies
Information Systems Accounting and Information Management Division
GAO/T-AIMD-99-102
GAO/T-AIMD-99-102
Page 1 GAO/T-AIMD-99-102
Mr. Chairman and Members of the Subcommittee: Thank you for
inviting us to participate in today's hearing on the Bureau of
Land Management's (BLM) Automated Land and Mineral Record System
project, also known as the ALMRS/ Modernization. The ALMRS/
Modernization was intended to provide modern computer and
telecommunications equipment and office automation for over 200
offices nationwide as well as software to more efficiently record,
maintain, and retrieve land description, ownership, and use
information to support BLM, other federal programs, and interested
parties. As you requested, I will discuss (1) the history of the
project, (2) the results of our reviews, including the key reasons
for problems, and (3) where we believe BLM should go from here.
Mr. Chairman, BLM spent over 15 years and estimates that it
invested about $411 million planning and developing the ALMRS/
Modernization, only to have the major software component known as
the ALMRS Initial
Operating Capability (IOC) fail. As a result of that failure, the
bureau decided not to deploy ALMRS IOC at this time. We have
previously reported on the significant problems and risks that BLM
has encountered. We have made many recommendations to reduce those
risks; however, BLM has been slow to implement some
recommendations and has not yet fully implemented others. The
bureau now needs to determine whether it can salvage any of the
more than $67- million reported investment in ALMRS IOC software,
by analyzing the software to determine if it can be cost-
beneficially modified to meet BLM's needs. In addition, to reduce
the risk that future efforts will result in similar failures, BLM
should assess its information technology investment practices and
systems acquisition capabilities.
We performed our work from July 1998 through February 1999, in
accordance with generally accepted government auditing standards.
The ALMRS/ Modernization: A Brief History
During the energy boom of the early 1980s, BLM found that it could
not handle the case processing workload associated with a growing
number of applications for oil and gas leases. The bureau
recognized that to keep up with increased demand, it needed to
automate its manual records and case processing activities.
Therefore, in the mid- 1980s, it began planning to acquire an
automated land and mineral case processing system. At that
Page 2 GAO/T-AIMD-99-102
time, BLM estimated that the life- cycle cost of such a system
would be about $240 million. In 1988 BLM expanded the scope of the
system to include a land information system (LIS). The expanded
system was to provide automated information systems and geographic
information systems technology 1 capabilities to support other
land management functions, such as land use
and resource planning. BLM combined the LIS with a project to
modernize the bureau's computer and telecommunications equipment,
and estimated the total life- cycle cost of this combined project
to be $880 million.
The project was reduced in scope in 1989 to respond to concern
about the high cost and named the ALMRS/ Modernization. The
project consisted of three major components the ALMRS IOC, a
geographic coordinate database, 2 and the modernization of BLM's
computer and telecommunications infrastructure and rehost of
selected management and administrative systems. Estimated life-
cycle costs were $575 million (later
reduced to $403 million), and BLM planned to complete the entire
project by the end of fiscal year 1996. The ALMRS IOC was to be
the flagship of the ALMRS/ Modernization, and was to replace
various manual and ad hoc automated systems. The bureau designated
the ALMRS IOC a critical system for (1) automating land and
mineral records, (2) supporting case processing activities,
including leasing oil and gas reserves and recording valid mining
claims, and (3) providing information for land and resource
management activities, including timber sales and grazing leases.
The system was expected to more efficiently record, maintain, and
retrieve land description, ownership, and use information to
support BLM, other federal programs, and interested parties. It
was to do this by using the new computer and telecommunications
equipment that was deployed throughout the bureau, integrating
multiple databases into a single geographically referenced
database, shortening the time to complete case processing
activities, and automating costly manual records.
1 A geographic information system is computer technology designed
to assemble, store, manipulate, and display geographically
referenced data, such as the location of a lake or oil well. 2 We
previously reported significant cost overruns and milestone
slippages on an earlier project to develop the database. See Land
Management Systems: Extensive Cost Increases and Delays in BLM's
Major Data Base Project (GAO/IMTEC-91-55, August 5, 1991).
Page 3 GAO/T-AIMD-99-102
Despite the promise of ALMRS IOC to significantly improve business
operations, repeated problems with its development have prevented
deployment. For example, during a user evaluation test in May
1996, problems were reported involving unacceptably slow system
performance. Subsequent testing in 1996 uncovered 204 high-
priority software problems, which delayed project completion by
about a year. In testing conducted in November 1997, BLM
encountered workstation failures and slowdowns caused by
insufficient workstation memory and by problems discovered in two
BLM- developed software applications. Some of these problems had
been identified in earlier tests but had not been corrected.
Additional testing uncovered software errors that resulted in
missing, incorrect, and incomplete data, and error files that
contained accurate data. As a result of these problems, BLM
postponed the Operational Assessment Test and Evaluation (OAT& E)
that had been scheduled for December 1997. The OAT& E was to
determine whether ALMRS IOC was ready to be deployed to the first
state office. In October 1998, the OAT& E was conducted and showed
that ALMRS IOC was not ready to be deployed because it did not
meet requirements. During the test, users reported several
problems, including that ALMRS IOC (1) did not support BLM's
business activities, (2) was too complex, and
(3) significantly impeded worker productivity. For example, one
tester reported that entering data for a $10 sale of a commodity,
such as gravel, required an hour of data entry using ALMRS IOC,
whereas with the existing system, the same transaction would have
taken about 10 minutes. Users also reported that system response
time problems were severe or catastrophic at all test sites. One
user said It is ridiculous to spend 2 or 3 hours to enter
information in this system, when it takes 30 minutes to an hour to
process the information into the legacy system. Finally, users
reported data converted from legacy databases were not accurate,
and that validation of the converted legacy data required
inordinate effort and time. Because these problems are
significant, senior BLM officials have decided that ALMRS IOC is
not currently deployable. According to BLM, it obligated about
$411 million on the ALMRS/ Modernization project between fiscal
years 1983 and 1998, of which more than $67 million was spent to
develop ALMRS IOC software. The $67 million does not include ALMRS
IOC costs that are part of other cost categories, such as costs
for work performed from fiscal years 1983 through 1988, project
management, computer and telecommunications hardware and software,
data management, and systems operation and maintenance. The
reported
Page 4 GAO/T-AIMD-99-102
obligations associated with the major cost categories of the
ALMRS/ Modernization are summarized in table 1.
Table 1: Reported ALMRS/ Modernization Obligations, Fiscal Years
1983 Through 1998
Source: BLM; amounts include funding from all sources. We did not
independently verify these data.
Senior BLM officials told us that although ALMRS IOC is not
currently deployable, BLM has benefited from the ALMRS/
Modernization work. BLM has deployed about 6,000 workstations
throughout the bureau, provided office automation capabilities,
and implemented a national telecommunications network with
electronic mail and internet access, which has enhanced
communications and enabled BLM to communicate
with other federal agencies. BLM's view of the benefits received,
however, does not reflect the fact that it has not realized the
significant businessrelated benefits and improvements ALMRS IOC
was to provide. Cost category Explanation Obligations
FY 1983- 1988 obligations Data collection Concept development
Requirements definition Contract preparation $32,925, 000
Project management BLM and contractors' costs for project
management, oversight, and administration BLM salaries, benefits,
and travel 74,690, 940
ALMRS IOC software development ALMRS Release 1 and 2 67,547, 220
Administrative systems modernization Rehost of selected
management and administrative systems from outdated
mainframe computers to BLM's modern, networked environment 8, 198,
466 Computer and telecommunications hardware and software
Acquiring and installing new hardware and software to support
office automation and administrative functions
121,348, 325 Data management BLM and contractor costs to collect,
edit, and convert BLM land and mineral program related data
Ongoing work to establish the geographic coordinate database
Conversion of BLM data from legacy systems into the ALMRS database
85,476, 518
Operations and maintenance Configure, operate, and maintain
administrative, database, and e- mail software, telecommunications
and computer hardware, and operating systems software Contract
maintenance fees BLM and contractor labor costs 20,957, 296
Total ALMRS modernization $411,143, 765
Page 5 GAO/T-AIMD-99-102
Our Reviews Have Shown Long- Standing Project Weaknesses
Mr. Chairman, since May 1995 we have reported many problems and
risks that threatened the successful development and deployment of
the ALMRS/ Modernization. Our reports have discussed these issues,
their causes, and our recommended corrective actions. 3 BLM has
been slow to implement some of our recommendations and has not yet
fully implemented others. Following is a summary of the problems,
causes, and associated recommendations we have reported.
BLM did not develop a system architecture or formulate a concept
of operations before designing and developing the ALMRS/
Modernization. A system architecture describes the components of a
system, their interrelationships, and principles and guidelines
governing their design and evolution. A concept of operations
describes how an organization would use planned information
technology to perform its business operations and accomplish its
missions. Designing and developing the
project without a system architecture and concept of operations
unnecessarily increased the risk that the ALMRS/ Modernization
would not meet the business and information needs of the bureau.
BLM has never had a credible project schedule, reliable
milestones, or a critical path to manage the development and
deployment of the ALMRS/ Modernization. As a result, BLM has not
known with any certainty how long it would take and, therefore,
how much it would cost to complete
the ALMRS/ Modernization. Because BLM has not implemented our
recommendation to establish a credible project schedule, the
ALMRS/ Modernization has been driven by self- imposed deadlines.
In trying to meet those deadlines, BLM has deferred some tasks
until after completion of the project, and has not corrected all
problems when it found them because doing so would cause it to
miss the self- imposed project deadlines. BLM faced serious risks
because it had not established a robust configuration management
program for the ALMRS/ Modernization. Configuration management is
essential to controlling the composition of and changes to
computer and network systems components and documentation. The
lack of configuration management increased the risks that system
modifications could lead to undesirable consequences, 3 Land
Management Systems: Progress and Risks in Developing BLM's Land
and Mineral Record System (GAO/AIMD-95-180, August 31, 1995), Land
Management Systems: BLM Faces Risks in Completing the Automated
Land and Mineral Record System (GAO/AIMD-97-42, March 19, 1997),
Land Management Systems: Information on BLM's Automated Land and
Mineral Record System Release 2 Project (GAO/
AIMD- 97- 109R, June 6, 1997), and Land Management Systems:
Actions Needed in Completing the Automated Land and Mineral Record
System Development (GAO/AIMD-98-107, May 15, 1998).
Page 6 GAO/T-AIMD-99-102
such as causing system failures, endangering system integrity,
increasing security risks, and degrading system performance. In
response to our recommendation, BLM later developed a
configuration management plan and related policies and procedures
for the ALMRS/ Modernization. We planned to review field office
implementation of the configuration management program after
completion of the ALMRS IOC; however, we have not done so because
the system was not
deployed. BLM incurred serious risks because it had not
established a security plan or security architecture for the
ALMRS/ Modernization. The lack of such security controls increased
risks to the confidentiality, integrity, and availability of
stored and processed data. BLM recently completed work in response
to our recommendation. It performed a risk analysis, developed a
system security plan and architecture, identified management and
operational controls, and developed disaster and recovery plan
procedures. As with configuration management, we planned to review
field office implementation of the security program
after completion of the ALMRS IOC, but have not done so because
the system was not deployed. BLM invited serious risks because it
had not established transition plans
to guide the incorporation of ALMRS IOC into its daily operations.
Deploying a major information system that people will use to do
their jobs requires careful planning to avoid business and
operational problems. Without transition plans, BLM increased the
risk that using ALMRS IOC would disrupt, rather than facilitate,
its work processes and ability to conduct land and mineral
management business. In response to our recommendation, BLM
developed transition plans; however, the
plans were not adequate. They did not outline needed changes in
organizational roles, responsibilities, and interrelationships, or
address issues such as how state and subordinate offices would
deal with oil and gas, mining, and solid mineral business process
changes that would result from implementing ALMRS IOC. BLM faced
serious risks because it had not established operations and
maintenance plans. The lack of plans increased the risk that the
bureau would not meet its automation objectives or the daily needs
of its offices. BLM developed operations and maintenance plans in
response
to our recommendation. We expected to review field office
implementation of the operations and maintenance plans after
completion of the ALMRS IOC; however, we have not done so because
the system was not deployed.
BLM invited serious risks because it planned to stress test only
the ALMRS IOC component state and district offices, ALMRS IOC
servers,
Page 7 GAO/T-AIMD-99-102
terminals, and workstations. This increased the risk that BLM
would deploy the ALMRS IOC nationwide without knowing whether the
ALMRS/ Modernization ALMRS IOC, office automation, e- mail,
administrative systems, and various departmental, state, and
district software applications in a networked environment would
perform as intended during peak workloads. BLM agreed to fully
stress test the entire ALMRS/ Modernization before deploying the
ALMRS IOC component throughout the bureau. BLM did not develop a
Year 2000 contingency plan to ensure that critical legacy systems
could operate after January 1, 2000, if the ALMRS IOC could not be
delivered in 1999. We recommended that BLM develop a
Year 2000 contingency plan to ensure continued use of those
critical legacy systems ALMRS IOC was to replace. BLM implemented
this recommendation and began executing the plan in 1998, when it
became clear that ALMRS IOC would not be fully implemented by the
end of 1999. Where BLM Should Go From Here
At this point, BLM has made an enormous investment in software
that does not meet its business needs. At the same time, it has
not adopted information technology management practices required
by recent legislation or suggested by industry best practices.
Because of its large investment, BLM should analyze ALMRS IOC to
determine whether the software can be cost- beneficially modified
to meet the bureau's needs. In addition, to reduce the risk that
future information technology efforts will result in a similar
outcome, BLM should assess its investment management practices and
its systems acquisition capabilities. Until these assessments and
subsequent improvement actions are taken, BLM will not be
adequately prepared to undertake any sizable system acquisition.
Analysis of ALMRS IOC Software Is Needed
We believe that since BLM has invested over $67 million to develop
the ALMRS IOC software, the bureau should thoroughly analyze the
software to determine whether it can be modified to meet users'
needs and at what cost. This analysis should be part of an overall
effort to identify and assess all viable alternatives, including
(1) using or modifying ALMRS IOC software, (2) modifying or
evolving existing land and recordation systems, (3) acquiring
commercial, off- the- shelf software, or (4) developing new
systems. The alternative analysis should clearly identify the
risks, costs, and benefits of each alternative, and should be
performed only after BLM is assured that it has fully verified its
current business requirements. In this regard, senior BLM
officials said they are performing an analysis to
Page 8 GAO/T-AIMD-99-102
determine where ALMRS IOC failed to meet users' expectations and
critical business requirements.
Assessment of BLM's Information Technology Investment Management
Practices Is Needed
According to the acting land and resources information systems
program manager, BLM is beginning to develop plans for future
information technology modernization. These plans are to identify
alternatives to deploying ALMRS IOC, and evaluate those
alternatives based on cost, functionality, and return on
investment. BLM also plans to document its current and planned
business processes and systems architectures as part of this
effort.
While such planning is necessary, BLM also needs to assess its
investment management practices to help avoid future problems. The
Clinger- Cohen Act of 1996 seeks to maximize the return on
investments in information systems by requiring agencies to
institute sound capital investment decision- making. Under the
act, agencies must design and implement a process for maximizing
the value and assessing and managing the risks of information
technology acquisitions. An information technology investment
process is an integrated approach that provides for data- driven
selection, control, and evaluation of information technology
investments. 4 The investment process is comprised of three
phases. The first phase involves selecting investments using
quantitative and qualitative criteria for comparing and setting
priorities for information technology projects. The second phase
includes monitoring and controlling selected projects through
progress reviews at key milestones to compare the expected costs,
risks encountered, and performance benefits realized to date.
These progress reviews are
essential for senior managers to decide whether to continue,
accelerate, modify, or terminate a selected project. The third
phase involves a postimplementation review or evaluation of fully
implemented projects to compare actuals against estimates, assess
performance, and identify areas where future decision- making can
be improved.
According to senior BLM officials, the bureau has established an
Information Technology Investment Board to provide support for its
capital 4 This process is documented in our Assessing Risks and
Returns: A Guide for Evaluating Federal Agencies' IT Investment
Decision- Making, Version 1 (GAO/ AIMD- 10. 1. 13, February 1997)
and OMB's Evaluating Information Technology Investments: A
Practical Guide, Office of Management and Budget, Version 1.0,
November 1995.
Page 9 GAO/T-AIMD-99-102
planning processes. It intends to apply more rigorous, structured
processes to analyze its information technology investments and
select, control, and evaluate information technology investment
alternatives. Until such processes are fully in place, the bureau
cannot be assured that future investments will be properly
selected, managed, and evaluated using sound investment criteria
to provide effective support for the bureau's mission and goals.
Further, to ensure that information technology investment
processes are carried out adequately, the Clinger- Cohen Act also
requires agencies to assess the knowledge and skills of its
executive and management staff to meet agencies' information
resources management requirements, and to take steps to rectify
any deficiencies. The Software Engineering Institute 5 (SEI) has
identified the need for organizations to focus on information
resources management capabilities. 6 Organizations should improve
their capabilities using a process to characterize the maturity of
their workforce
practices, guide a program of workforce development, set
priorities for immediate actions, and establish a culture of
software engineering excellence. According to senior BLM
officials, the bureau examined the kind of skills that its field
office computer specialists had, and identified the skills they
would need. However, the officials recognize that this was not the
same as the more comprehensive assessment suggested by SEI. Such
assessments are needed to better identify and manage information
technology investments. Consequently, the bureau should evaluate
and, where needed, enhance the knowledge and skills of its staff
to help ensure that the investment management processes it puts in
place can be effectively carried out by its information resources
management organization. 5 SEI, located at Pittsburgh's Carnegie
Mellon University, is a nationally recognized, federally funded
research and development center established to address software
development issues.
6 Software Engineering Institute, People Capability Maturity
Model, CMU/ SEI- 95- MM- 02, September 1995.
Page 10 GAO/T-AIMD-99-102
Finally, the Clinger- Cohen Act requires agencies to develop,
maintain, and facilitate the implementation of a sound and
integrated information technology architecture. 7 An information
technology architecture provides a comprehensive blueprint that
systematically details the breadth and depth of an organization's
mission- based mode of operation. An architecture provides details
first in logical terms, such as defining business functions,
providing high- level descriptions of information systems and
their interrelationships, and specifying information flows; and
second in technical terms, such as specifying hardware, software,
data, communications, security, and performance characteristics.
By enforcing an information technology architecture to guide and
constrain a modernization program, an agency can preclude
inconsistent systems design and development decisions, and the
resulting suboptimal performance and excess cost. As I discussed
earlier, BLM did not develop a system architecture before
designing and developing the ALMRS/ Modernization. This is a key
reason why ALMRS IOC did not meet the bureau's business needs. BLM
still has not developed an architecture that documents its
business processes and the technology and systems that support
them. BLM needs to develop an information technology architecture
to guide its future investment plans. BLM Needs to Assess Its
Systems Acquisition Capabilities
Research by SEI has shown that defined and repeatable processes
for managing software acquisition are critical to an
organization's ability to consistently deliver high- quality
information systems on time and within budget. These critical
management processes include project planning, requirements
management, software project tracking and oversight, software
quality assurance, software configuration management, and change
control management. 8 To assist organizations in evaluating and
enhancing systems acquisition capabilities and processes, SEI has
developed models for conducting software process assessments and
7 The Clinger- Cohen Act of 1996 gives the Chief Information
Officer of an executive agency the responsibility for developing,
maintaining, and facilitating the implementation of a sound and
integrated information technology architecture. An information
technology architecture is sometimes
referred to as a system architecture. 8 Definitions of these
processes were obtained from the Software Engineering Institute's
Capability Maturity Model for Software, Version 1.1, 1993
Carnegie Mellon University.
Page 11 GAO/T-AIMD-99-102
software capability evaluations to determine the state of their
capabilities and identify areas requiring improvement. 9 BLM also
needs an independent assessment of its systems acquisition
capabilities, and must ensure that it uses sound systems
acquisition processes. As I discussed earlier, BLM did not develop
several key management controls for the ALMRS/ Modernization. BLM
did not develop a credible project schedule or develop adequate
transition plans. In addition, the lack of a configuration
management program, security plan and architecture, and operations
and maintenance plans further increased BLM's risks. These
problems indicate the need for BLM to ensure that the deficiencies
in its systems acquisition capabilities and processes are
acknowledged and corrected. Until such assessments are completed
and corrective action taken, BLM should not undertake any sizable
systems acquisition or development efforts. Mr. Chairman, that
concludes my statement. I would be happy to respond to any
questions that you or other members of the Subcommittee may have
at this time.
9 SEI has developed two models to assist organizations in
assessing the maturity of their software development processes.
These models are the Capability Maturity Model for Software and
the Software Acquisition Capability Maturity Model. Capability
Maturity Model SM is the service trademark of Carnegie Mellon
University, and CMM is registered in the U. S. Patent and
Trademark Office.
(511470) Let t er
Page 12 GAO/T-AIMD-99-102
Ordering Information The first copy of each GAO report and
testimony is free. Additional copies are $2 each. Orders should be
sent to the following address, accompanied by a check or money
order made out to the Superintendent of Documents, when necessary,
VISA and
MasterCard credit cards are accepted, also. Orders for 100 or more
copies to be mailed to a single address are discounted 25 percent.
Orders by mail: U. S. General Accounting Office P. O. Box 37050
Washington, DC 20013
or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW)
U. S. General Accounting Office Washington, DC
Orders may also be placed by calling (202) 512- 6000 or by using
fax number (202) 512- 6061, or TDD (202) 512- 2537.
Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512- 6000 using a
touchtone phone. A recorded menu will provide information on how
to obtain these lists. For information on how to access GAO
reports on the INTERNET, send an e- mail message with info in the
body to:
info@ www. gao. gov or visit GAO's World Wide Web Home Page at:
http:// www. gao. gov
United States General Accounting Office Washington, D. C. 20548-
0001
Official Business Penalty for Private Use $300
Address Correction Requested Bulk Rate
Postage & Fees Paid GAO Permit No. GI00
*** End of document. ***