Coast Guard: Update on Marine Information for Safety and Law
Enforcement System (17-OCT-01, GAO-02-11).
The Marine Information for Safety and Law Enforcement (MISLE)
system is an information system to track marine safety and
law-enforcement activities involving commercial and recreational
vessels. In 1999, the United States Coast Guard(USCG) terminated
a contract to acquire MISLE, after spending about four years and
$26 million, and is instead developing the system at its
Operation Systems Center. USCG has made progress in developing
MISLE and was poised to deploy a minimum level of functionality
in November 2001. In its efforts to develop and deploy a complete
MISLE system, USCG faces significant challenges and risks in
several areas, including managing system requirements and user
expectations, testing the system, transitioning to an operational
system, and managing program risks.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-02-11
ACCNO: A02298
TITLE: Coast Guard: Update on Marine Information for Safety and
Law Enforcement System
DATE: 10/17/2001
SUBJECT: Contract termination
Information systems
Law enforcement information systems
Marine safety
Systems design
DOT Marine Information System for Safety
and Law Enforcement
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-02-11
A
Report to the Subcommittee on Coast Guard and Maritime Transportation,
Committee on Transportation and Infrastructure, House of Representatives
October 2001 COAST GUARD Update on Marine Information for Safety and Law
Enforcement System
GAO- 02- 11
October 17, 2001 The Honorable Frank LoBiondo Chairman The Honorable Corrine
Brown Ranking Minority Member Subcommittee on Coast Guard and Maritime
Transportation Committee on Transportation and Infrastructure House of
Representatives
Over the last decade, the United States Coast Guard (USCG) has experienced
difficulties in acquiring the Marine Information for Safety and Law
Enforcement (MISLE) system- an information system to track marine safety and
law- enforcement activities involving commercial and recreational vessels.
In 1999, the Coast Guard terminated a contract to acquire MISLE, after
spending about 4 years and $26 million, and is instead developing the system
at its Operations Systems Center. As you requested, our objective was to
provide an update on MISLE?s status, plans, and
technical and programmatic risks. To fulfill this objective, we evaluated
MISLE project plans, costs, and schedules by comparing original and current
job management documents. We also assessed technical and programmatic risks
facing the MISLE acquisition and USCG?s plans for addressing those risks. We
performed our work from May through September 2001, in accordance with
generally accepted government auditing standards. USCG officials provided us
with comments on a draft of this report; they are discussed in the ?Agency
Comments? section. On September 18, 2001, we provided a detailed briefing to
your office on the results of this work. The briefing slides are included in
appendix I. The purpose of this letter is to provide the published briefing
slides to you and to officially transmit our recommendations to the
Secretary of Transportation.
In brief, we reported that USCG has made progress in developing MISLE and
was poised to deploy a minimum level of functionality in November 2001.
However, the system was several months behind schedule, and USCG had already
spent most of the $61 million acquisition cost estimate- most of it on the
original contract. With only about $11 million remaining, much remains to be
done to deliver the complete system. In its efforts to develop and deploy a
complete MISLE system, USCG faces significant
challenges and risks in several areas, including managing system
requirements and user expectations, testing the system, transitioning to an
operational system, and managing program risks. We made specific
recommendations to address these risks. Recommendations To mitigate USCG?s
MISLE risks, we recommend that the Secretary of Transportation direct the
USCG Commandant to ensure that the
appropriate officials complete the following actions. In the system
requirements area, define and prioritize, in conjunction with system
users, all needed
system functions, corrections, and enhancements that must occur to meet
valid user needs; and develop cost and schedule estimates for providing
these functions,
corrections, and enhancements. In the software testing area, close all
critical problems before initiating the next state of testing. In the area
of transition planning, finalize and implement Vessel Documentation System
(VDS) transition plans, and develop and implement VDS training materials.
In the risk mitigation area,
develop a single list of system risks, evaluate system risks to
determine their severity and prioritize these
risks, develop and implement comprehensive mitigation strategies for each
of
the risks, and regularly oversee the status of risks and risk mitigation
efforts to determine whether additional mitigation activities are warranted.
Agency Comments We provided drafts of our briefing and this report to
Department of Transportation and USCG officials, including representatives
of the Office of the Secretary of Transportation and the USCG Acquisition
Management Office. We met with USCG officials to obtain their comments on
our drafts. These officials generally agreed with our recommendations, but
characterized the changing requirements and testing issues we raised as
?tradeoff decisions,? necessary to deliver an initial version of MISLE. USCG
officials also noted that, given the recent terrorist attacks, MISLE
deployment will likely be delayed until at least December 2001. We are
sending copies of this report to the Secretary of Transportation, the USCG
Commandant, the Director of the Office of Management and Budget, and other
interested parties. Copies will also be made available to others upon
request.
Should you or your staffs have any questions concerning this report, please
contact me at (202) 512- 6240 or by e- mail at koontzl@ gao. gov. Nabajyoti
Barkakati, Barbara Collier, Michael Fruitman, Colleen Phillips, Margaret
Sullivan, and Glenda Wright were major contributors to this report.
Linda D. Koontz Director, Information Management Issues
Appendi x I
GAO?s September 18, 2001, Briefing United States Coast Guard s ( USCG)
Marine Information for Safety and Law Enforcement ( MISLE) System
An Update
Briefing for Staff Subcommittee on Coast Guard and Maritime Transportation
Committee on Transportation and Infrastructure House of Representatives
September 18, 2001 1
Briefing Outline
Objective, Scope, and Methodology MISLE Background MISLE Status and Plans
Key Risk Areas
System requirements Software testing Transition planning Risk management
Conclusions Recommendations Agency Comments and Our Evaluation 2
Objective, Scope, and Methodology Objective
To provide an update on MISLE s status, plans, and technical and
programmatic risks
Scope and Methodology
We evaluated MISLE project plans, costs, and schedules by comparing original
and current job management documents. We also assessed technical and
programmatic risks facing the MISLE acquisition, as well as USCG s plans for
addressing those risks, by evaluating USCG program documents, comparing them
to accepted system engineering principles, and interviewing project
officials, developers, and system users. Further, we reviewed plans for
future MISLE enhancements and discussed these enhancements with system users
and developers.
We conducted our review at USCG headquarters in Washington, D. C. , and at
USCG s Operations Systems Center and National Vessel Documentation Center in
the Martinsburg, WV, area. We conducted our work from May through September
2001, in accordance with generally accepted government auditing standards.
3
MISLE Background Overview
MISLE is intended to be a Web- based information system for entering and
obtaining data on Coast Guard activities concerning commercial and federally
documented recreational vessels in support of USCG s marine safety and law
enforcement missions.
A software- intensive system, the MISLE project also includes the purchase
of hardware, including network servers and data storage devices.
MISLE is to support over 5000 system users at about 550 sites using the
existing Coast Guard intranet.
4
MISLE Background Overview ( cont d)
In the mid- 1990s, MISLE w s envisioned to develop nd integrate two major
systems the Marine Safety Network and the Vessel Identification and
Documentation System and to integrate them with a third, already developed
system, the Law Enforcement Information System II.
Since then, however, this concept has evolved. A description of the
subsystems and recent changes to these subsystems are discussed below:
Marine Safety Network ( MSN) a system to allow Coast Guard personnel to
input and obtain information on USCG marine safety activities, such as
vessel inspections and boardings; it relies on vessel data from the Vessel
Documentation System
5
MISLE Background Overview ( cont d)
MISLE components ( cont d) Vessel Identification and Documentation System
a system planned to integrate two subsystems in response to Public Law 100-
710 ( commonly called the Ship Mortgage Act of 1988) : Vessel Identification
System planned to automate nd integrate
vessel registration information from participating states; due to
complications in implementing this system, USCG no longer plans to provide
VIS functionality as part of MISLE ( see Key Risk Area System Requirements)
Vessel Documentation System ( VDS) a system to support the
National Vessel Documentation Center s processes for documenting vessels
Law Enforcement Information System II ( LEIS- II) a system for tracking USCG
law enforcement activities, such as drug interdiction activities; due to the
age of the LEIS- II platform, USCG now plans to replace LEIS- II rather than
integrate it with MISLE
6
MISLE Background Overview ( cont d)
As of early September 2001, USCG s estimates for MISLE costs nd deployment
schedules included the following:
Acquisition cost: $ 61 million through 2003 Life cycle cost: $ 94 million
through 2008 Initial capability: November 2001 Final capability: September
2003
7
MISLE Background History
In 1986, USCG began developing requirements for replacing its legacy Marine
Safety Information System ( MSIS) by 1995.
In the early 1990s, USCG delayed plans to replace MSIS in order to integrate
requirements for multiple systems into one system development effort called
MISLE.
In 1995, USCG awarded a contract to Computer Sciences Corporation ( CSC) to
develop and deliver a complete MISLE system by 2002. At that time, USCG
officials estimated that this contract could cost up to $ 35 million.
A March 1999 change required the developer to replace MSIS functions by June
2001.
8
MISLE Background History ( cont d)
The MISLE development contract encountered escalating costs and schedule
delays; it was partially terminated for government convenience in October
1999.
CSC delivered two components under this contract: Vessel Identification
System a system to automate and
integrate vessel registration information from participating states. This
system was never implemented due to problems in integrating state data. USCG
no longer plans to provide VIS functionality as part of MISLE. Mission
Analysis and Planning a decision support tool for
obtaining statistical information from the legacy MSIS system Total spent on
the CSC contract: $ 26 million In October 1999, USCG transferred
responsibility for MISLE development from CSC to its Operations Systems
Center ( OSC) a government- owned, contractor- operated facility. USCG s
acquisition office manages the project.
9
MISLE Background Acquisition Strategy
OSC s strategy is to develop and deploy a minimal level of MISLE
functionality as soon as possible to replace MSIS.
MSIS once scheduled for termination in 1995 is still serving as the primary
information system for marine safety programs.
MSIS h s failed repeatedly over the past several years, causing days and
weeks of dat processing backlogs.
USCG officials stated that MSIS hardware is no longer supportable; the
original vendor is no longer in business, and the agency is currently
reusing parts to keep the system running.
10
MISLE Background Acquisition Strategy ( cont d)
OSC s system development approach: Rapid Application Development/ Joint
Application Design
a system development methodology that focuses on involving users early and
often in the design and development of a system in order to develop it
quickly.
Timebox Methodology
a system development methodology, often used in conjunction with the Rapid
Application Development methodology, in which a time limit is established
for developing a system ( this forces users to focus on a minimal set of
requirements and allows them to drop some functions if necessary to meet the
time limit) .
11
MISLE Background Planned Delivery Schedule
OSC s three- phased delivery approach:
MISLE Phase 1
Oct. 1999 to Nov. 2001
Deliverables: initial MSN capability needed to replace MSIS initial VDS
capability
MISLE Phase 2
Feb. 2001 to March 2002
Deliverables: replacement of LEIS- II extended MSN and VDS capabilities
initial data warehouse
MISLE Phase 3
Feb. 2002 to Sept. 2003
Deliverables: completion of rem ining MSN and VDS capabilities expanded data
warehouse
12
MISLE Status and Plans Progress to Date MISLE Phase 1
40 high level requirements have been translated to over 13,000 detailed
software requirements supporting two key components: the Marine Safety
Network ( MSN) and the Vessel Documentation System ( VDS)
MSN Status: Completed system testing 1 User input obtained throughout
VDS Status: Major components postponed to later releases Completed system
testing
MISLE Phase 2
Development began February 2001 1 USCG tests increasingly l rger portions of
MISLE in stages: ( 1) unit testing- - informal tests of software modules; (
2) functional testing- - formal tests of
13 softw re modules; ( 3) integration testing- - testing the integration of
all modules in a core area; and ( 4) system testing- - testing all of the
core areas in deliverable. MISLE then undergoes operational test and
evaluation ( OT& E) - - testing of operational effectiveness and suitability
by system users.
MISLE Status and Plans Costs
USCG s $ 61 million total estimated MISLE acquisition cost includes: bout $
7 million spent on requirements analysis, design, and
validation in the early 1990s $ 26 million spent on the 1995 MISLE contract
that w s terminated bout $ 28 million for current MISLE development efforts,
including
lmost $ 17 million spent through fiscal ye r 2001 to develop most of MISLE
Phase 1 part of MISLE Phase 2
bout $ 11 million in fiscal year 2002 and beyond to complete residual tasks
supporting MISLE Phases 1 and 2 to develop nd implement MISLE Phase 3
. 14
MISLE Status and Plans Schedule
MISLE Phase 1 has fallen behind schedule. MISLE Phase 1
Dec. 1999 June and
May 2001 Aug. 1, 2001
Sept. 5, 2001 milestones baseline
Dec. 2000 estimate estimate estimate change
estimates b System testing complete N/ I c N/ I 7/ 20/ 01 MSN 8/ 15/ 01
MSN 8/ 27/ 01 VDS 9/ 15/ 01 VDS 8/ 27/ 01
Operational Test and 6/ 01 2/ 01 8/ 15/ 01 MSN 8/ 30/ 01
MSN 9/ 21/ 01 Evaluation complete VDS 9/ 30/ 01 VDS 9/ 21/ 01
MISLE month complete N/ I N/ I 9/ 15/ 01 9/ 30/ 01 10/ 19/ 01 Data migration
complete N/ I N/ I 9/ 30/ 01 10/ 10/ 01 11/ 2/ 01 MISLE Phase 1 deployed N/
I 4/ 01 10/ 01 10/ 15/ 01 11/ 5/ 01 In commenting on a draft of this
briefing, the MISLE project manager noted this baseline document also
includes an OT& E completion range from January to September 2001, and that
OT& E would be completed within this range. However, USCG internal m
nagement reports reflect the June 2001 completion d te. In fact, these
reports show MISLE in breach of its schedule. b These estim tes are
identified in the June 2000 MISLE Implementation Pl n, as well as the
December 2000
MISLE Acquisition and Project M nagement Plans. In commenting on a draft of
this briefing, USCG officials characterized these documents as internal,
optimistic project plans- - not firm schedule estim tes. c Not identified.
15
MISLE Status and Plans Schedule ( cont d)
USCG officials identified the following reasons for schedule delays:
Problems in integrating VDS and MSN components caused a
major VDS redesign ( October 2000) USCG needed to retrofit MSN to work with
the current version of
Internet Explorer ( January 2001) VDS functional testing was delayed because
of system instability
( January 2001) MSN integration testing identified a missing function ( June
2001) :
bility to communicate with USCG s financial center VDS integration testing
encountered problems ( July 2001) :
test scripts would not run at users facility shortcoming in ability to edit
MSIS data migrated to MISLE
( estimated 4 week delay) 16
Key Risk Areas
The MISLE acquisition is facing considerable challenges in the following key
risk areas:
System Requirements Software Testing Transition Planning Risk Management
17
Key Risk Areas System Requirements
System requirements define the minimum functions and performance levels
needed to satisfy user needs. USCG identified system requirements for MISLE
in 1994. Also, when the agency restructured the MISLE program in 1999, it
identified specific functions to be delivered in each of the three MISLE
phases.
However, MISLE requirements and the functions to be provided in each of the
three deliverables are now changing. USCG plans to drop several of MISLE s
original system requirements and has recently postponed key functions due to
problems encountered during testing and the agency s need to field a system
quickly. Also, in response to system users needs,
, USCG plans to add and accelerate key functions. Further, while testing and
training on the system, users identified new and missing functions which
USCG plans to add to MISLE later.
USCG s plans for dropping, delaying, accelerating, and adding functions are
discussed below.
18
Key Risk Areas System Requirements ( cont d)
USCG is planning to drop significant MISLE requirements:
Vessel Identification System This system, planned to automate states vessel
registration data and integrate it with data on federally documented vessels
in response to the Ship Mortgage Act of 1988, was developed as part of MISLE
but never successfully implemented. USCG is considering developing a new
system outside MISLE.
Portable/ Remote Access This function was planned to provide the ability to
access MISLE information from remote locations, such as Coast Guard cutters.
USCG officials stated that MISLE will allow for remote ccess via laptop and
dial- up modem, but that providing cutter connectivity will require
decisions on the USCG infrastructure that are beyond MISLE s scope. USCG is
considering leasing satellite time to provide cutter connectivity.
19
Key Risk Areas System Requirements ( cont d) Dropped requirements ( cont d)
External Access This function was planned to provide non- USCG personnel
with access to MISLE data. USCG officials stated that MISLE has this
capability but will not provide external access because of security
concerns. USCG plans to reassess the need for external access on a case- by-
case basis.
Merchant Mariner Licensing and Documentation This function was to provide
licensing and documentation capabilities within MISLE. However, instead of
waiting for MISLE, USCG decided to develop a stand- alone system to provide
this capability. MISLE Phase 1 is planned to link to the Merchant Mariner
Licensing and Documentation stand- alone system.
20
Key Risk Areas System Requirements ( cont d) Dropped requirements ( cont d)
Pollution Funds Management This function was to provide the ability to
manage pollution funds within MISLE. Instead, USCG developed an alternative
system outside MISLE to provide this functionality. MISLE will provide a
link to that system.
Personnel Resource Management This function was to provide the ability to
manage personnel within MISLE. Instead, USCG developed an alternative system
outside MISLE to provide this functionality. USCG officials expect that
MISLE will be able to access data in that system.
21
Key Risk Areas System Requirements ( cont d)
USCG has delayed some MISLE functions:
Significant Phase 1 deliverables are now planned for later releases.
Examples include VDS ability to
provide Abstracts of Titles provide Certificates of Ownership utomate NVDC s
paper- intensive work processes
Key Phase 2 deliverables are now planned to be Phase 3 deliverables. Two
high priority examples include
VDS ability to handle requests for nonvessel information ( this involves
non- U. S. corporations and oil spill response organizations) MSN s ability
to capture information on the approval and oversight
of the construction of portable quarters on vessels 22
Key Risk Areas System Requirements ( cont d) Delayed functions ( cont d)
Other Phase 2 deliverables are being reconsidered that is, USCG reported
that they may not be doable in Phase 3. . Examples include MSN s ability to
submit Web forms, such as vessel arrival, inspection request, and marine
event application forms capture and track information on reviews of manuals
( such as
Operations Manuals, Emergency Evacuation Plans, and Passenger Terminal
Security Plans)
Other examples include VDS ability to provide electronic access to data in
response to FOIA requests llow e- commerce, such as credit card transactions
and
submission of Web forms 23
Key Risk Areas System Requirements ( cont d)
USCG has added and accelerated MISLE functions:
In response to user needs, USCG h s decided to replace LEIS- II in Phase 2,
rather than integrate it with MISLE ccelerate a critical function, incident
response planning, from
Phase 3 to Phase 2 In commenting on a draft of this briefing, USCG officials
noted that the agency has also moved multiple functions from Phase 2 to
later releases of Phase 1 in response to user requests to get these
functions sooner. However, in prior discussions, USCG officials acknowledged
that they have not yet scheduled these later releases and, in fact, some of
the later Phase 1 releases may not be delivered until after Phase 2 is
delivered.
24
Key Risk Areas System Requirements ( cont d) USCG plans to add functions to
MISLE
While testing and training on MISLE, system testers and users are
identifying both needed system corrections and new user needs that MISLE
does not currently provide. USCG plans to add these functions to MISLE as
future enhancements.
As of August, USCG identified 72 future enhancements. Examples include
making vessel details on maps readable incorporating data from additional
information systems outside
MISLE making a print icon work
In commenting on a draft of this briefing, MISLE project officials stated
that these future enhancements are low- level requirements and user
preferences, and that they do not jeopardize the project s ability to
deliver on its agreed- upon scope.
25
Key Risk Areas System Requirements ( cont d)
USCG has developed a schedule for the functions that it deems are the
highest priority MSN release 1. 1 and 1.2, and MISLE Phase 2. However, USCG
has not yet prioritized all of the additional and delayed MISLE functions,
nor h s it developed schedules for providing them. For example, USCG has not
yet established a schedule for delivering the significant VDS functions that
were deferred from Phase 1 to the VDS release 1. 1, or for the many other
MSN nd VDS functions anticipated in later releases of Phase 1.
By dropping and/ or delaying key MISLE functions, USCG runs the risk that
the system will not function as intended or expected, and that the deployed
system will fall short of user needs. Further, without a schedule or cost
estimate for the delayed functions and enhancements, it is not clear when
these functions will be implemented nor how much it will cost to implement
them.
26
Key Risk Areas Software Testing
According to leading information technology organizations, to be effective,
software testing practices should be planned and conducted in a structured
and disciplined fashion. Typically, this involves testing increasingly
larger increments of a system until the complete system is tested and
accepted, and resolving critical problems before moving to the next phase of
testing.
USCG policies call for testing MISLE software in successively larger
increments 2
unit testing functional testing integration testing system testing
operational test and evaluation ( OT& E) managing problems reported during
testing:
problem reports are to be prioritized in levels 1 to 5 ( levels 1 and 2 are
most critical) , fixed, tested, and closed ll critical problem reports (
levels 1 and 2) are to be closed before the
next stage of testing begins 2 We define simil r concepts in our guide for
man ging testing ctivities,
27
Year 2000 Computing Crisis: A Testing Guide ( GAO/ AIMD- 10.1.21, November
1998) .
Key Risk Areas Software Testing ( cont d)
USCG s current testing schedule for MISLE components: MSN Integration
( 4/ 1/ 01) ( 6/ 26/ 01) Testing MSN System
( 6/ 30/ 01) ( 8/ 27/ 01) Testing
MSN OT& E ( 9/ 10/ 01) ( 9/ 21/ 01)
VDS Integration ( 5/ 21/ 01) ( 8/ 10/ 01)
Testing VDS System
( 8/ 16/ 01) ( 8/ 27/ 01) Testing
VDS OT& E ( 9/ 10/ 01) ( 9/ 21/ 01)
April May June July Aug Sept 2001
28
Key Risk Areas Software Testing ( cont d)
USCG has undertaken risky testing practices: Key VDS functions were not
tested during functional testing because
they were under development at the time nd were deferred to integration
testing. For example:
system security processing and printing Abstracts of Titles printing
Certificates of Documentation
29
Key Risk Areas Software Testing ( cont d)
Risky testing practices ( cont d) : Significant problems were not closed
before the next stage of testing
began. For example: 9 critical problems identified during MSN integration
testing were
not closed before system testing began USCG s testing practices increase the
risk that MISLE will not perform as expected, or may take longer to develop
than expected.
30
Key Risk Areas Transition Planning
According to the CIO Council s guidance on enterprise architectures,
agencies should carefully plan the transition from legacy systems to new
systems. To ease the transition from MSIS to MISLE, USCG developed a
transition plan that calls for developing data migration plans and training
system users.
USCG has acknowledged that transitioning from MSIS to MISLE will be
difficult. It will involve moving data from one system to the next, ensuring
the accuracy of the transported data, and training users on new business
processes. USCG has undertaken several transition efforts:
MSN dat migration plan has been developed Hundreds of MSN users have
undergone training A 4- week period, known as MISLE month, is planned in
which MSIS
and MISLE will be run concurrently to allow MSN users to become acquainted
with the new system and new work processes
31
Key Risk Areas Transition Planning ( cont d)
Despite its transition guidance, USCG has not yet completed critical VDS
transition planning because the initial system deliverables changed
dramatically in July 2001. Specifically,
VDS transition plan has not been finalized VDS training plan has not been
developed VDS users have not been trained
VDS users stated that training and transition planning are not critical for
the first release of VDS, because the functions being delivered will not
dramatically affect their operations. However, they acknowledge that
transition and training are critical to the next software release, 3 which
is planned to reengineer and automate their work processes.
Incomplete plans for transitioning to a new system increase the likelihood
that the system will not be deployed on schedule. Insufficient training
plans also increase the likelihood that users will not readily adapt to new
work processes and systems.
3 USCG has not yet scheduled the next software release for VDS. 32
Key Risk Areas Risk Management
Risk management is a key component of a sound systems development approach.
An effective risk management approach typically includes identifying,
prioritizing, resolving, and monitoring project risks. In support of this
approach, USCG s risk management plan calls for
ssigning a severity rating ( high, medium, or low) to risks that bear
particular attention and placing these risks on a risk watch list,
prioritizing these risks, planning a response or strategy for each risk on
the risk w tch list, and
drafting a detailed response plan, and reviewing and evaluating all risks on
the the risk watch list during
monthly management meetings. 33
Key Risk Areas Risk Management ( cont d)
USCG h s not effectively implemented its risk management plan. USCG
developed three different risk lists, and none of the risks were
assigned a severity rating. Risks on two of the risk lists have not been
prioritized. USCG developed detailed mitigation strategies for some, but not
all risks.
For example, det iled plans exist for d ta migration and transition risks
such plans do not exist for managing user expectations and VDS
instability risks USCG officials addressed some, but not all, active risks
at monthly status
briefings. Additionally, USCG officials stated that they discuss all active
risks during monthly management meetings; however, we found no evidence that
all risks were addressed at these informal meetings.
Inadequate risk management practices increase the likelihood that MISLE will
not be delivered on schedule, within budget, and able to perform as
expected. USCG officials acknowledged areas where they could improve their
risk management and have undertaken improvement efforts.
34
Conclusions
After a long history of problems in developing MISLE, USCG ended the
original MISLE contract in October 1999. Since that time, USCG has made
progress in developing MISLE and is now poised to deploy a minimum level of
functionality in November 2001. However, MISLE is several months behind
schedule and USCG has already spent most of its $ 61 million acquisition
cost estimate- - most of it on the original contract. With only about $ 11
million remaining, much remains to be done.
In its efforts to develop and deploy a complete MISLE system, USCG faces
significant challenges in several areas:
managing system requirements and user expectations testing the system
transitioning to an operational system, and managing program risks
Unless USCG can effectively address these challenges, MISLE deployment
schedules are likely to slip further and costs are likely to increase.
Further, MISLE may fall short of user expectations and its promised
functionality.
35
Recommendations
To mitigate MISLE risks, we are making several categories of recommendations
to USCG s MISLE program managers:
System Requirements
Define and prioritize, in conjunction with system users, all needed system
functions, corrections, and enhancements that must occur to meet valid user
needs. Develop cost and schedule estimates for providing these functions,
corrections, and enhancements.
Software Testing
Close all critical problems before initiating the next stage of testing. 36
Recommendations ( cont d) Transition Planning
Finalize and implement VDS transition plans. Develop and implement VDS
training materials.
Risk Mitigation
Develop a single list of system risks. Evaluate system risks to determine
their severity and prioritize these
risks. Develop and implement comprehensive mitigation strategies for each
of the risks. Regularly oversee the status of risks and risk mitigation
efforts to
determine whether additional mitigation activities are warranted. 37
Agency Comments and our Evaluation
In providing oral comments on a draft of this briefing, USCG officials
generally agreed with our recommendations, but disagreed with our conclusion
that unless the agency can address key risk areas, MISLE deployment
schedules are likely to slip further, costs are likely to increase, and
MISLE may fall short of user expectations and its promised functionality.
USCG officials stated that many of the changing requirements and testing
issues we raised were tradeoff decisions made to meet the primary goal of
delivering an initial version of MISLE to replace MSIS. Additionally, USCG
officials stated that some MISLE functions were deferred to later releases
in order to accelerate other user- requested functions. USCG officials
expressed confidence that they will deliver MISLE within budget and on
schedule.
38
Agency Comments and our Evaluation Our evaluation: We acknowledge that USCG
expects to meet its primary goal of replacing MSIS in the next few months.
However, MISLE will now provide fewer capabilities than originally planned,
and significant functions have been delayed because of problems in testing.
By dropping and delaying key MISLE functions, USCG runs the risk that the
system will fall short of user needs. Further, because there are no schedule
or cost estimates for most of the delayed functions, it is not clear when
they will be delivered or what they will cost.
( 310331) 39
(310331) Lett er
GAO?s Mission The General Accounting Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance to
help Congress make informed oversight, policy, and funding decisions. GAO?s
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of The fastest and easiest way to obtain copies of GAO
documents is through the Internet. GAO?s Web site (www. gao. gov) contains
abstracts and full- text files of GAO Reports and current reports and
testimony and an expanding archive of older products. The Testimony Web site
features a search engine to help you locate documents using key words and
phrases. You can print these documents in their entirety, including charts
and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as ?Today?s Reports,? on its Web
site daily. The list contains links to the full- text document files. To
have GAO E- mail this list to you every afternoon, go to our home page and
complete the easy- to- use electronic order form found under ?To Order GAO
Products.?
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out to
the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U. S. General Accounting Office P. O. Box 37050 Washington, D. C. 20013
To order by Phone: Voice: (202) 512- 6000 TDD: (301) 413- 0006 Fax: (202)
258- 4066
Visit GAO?s Document GAO Building Distribution Center Room 1100, 700 4th
Street, NW (corner of 4th and G Streets, NW)
Washington, D. C. 20013 To Report Fraud, Contact: Waste, and Abuse in
Web site: www. gao. gov/ fraudnet/ fraudnet. htm, E- mail: fraudnet@ gao.
gov, or
Federal Programs 1- 800- 424- 5454 (automated answering system).
Public Affairs Jeff Nelligan, Managing Director, NelliganJ@ gao. gov (202)
512- 4800 U. S. General Accounting Office, 441 G. Street NW, Room 7149,
Washington, D. C. 20548
What GAO Found
What GAO Found
The Coast Guard has made progress since taking over its Marine Information
for Safety and Law Enforcement (MISLE) system development; it is now poised
to deploy the system with limited capabilities. But much remains to be
accomplished in order to deliver the complete system, with only about $11
million of its $61 million estimate remaining.
MISLE is facing risks in four key areas:
Changing system requirements. MISLE requirements have repeatedly changed;
several have been dropped, and others postponed to later development phases
due to problems found during testing and the Coast Guard?s emphasis on
replacing its current system as soon as possible. Other requirements are
being added or accelerated. Such a continually changing scenario increases
the risk that MISLE will fall short of user needs.
Software testing. The Coast Guard undertook risky software testing practices
in that it deferred testing some functions and did not resolve all critical
problems uncovered before moving on to the next testing stage. This approach
increases the likelihood that the system will not perform as expected and/
or may take longer to develop than anticipated.
Transition planning. Deployment of MISLE involves planning for accurately
moving data from the older system and training system users. However,
critical transition plans are not yet complete. Beyond adding to possible
delays, the absence of transition plans and insufficient training increase
the chance of user discomfort with the new system.
Risk management. Finally, the Coast Guard?s risk management approach has
been ineffective: risks were not assigned severity ratings, and not all have
been prioritized. Further, the Coast Guard has not developed detailed
mitigation plans for all significant risks.
Unless these challenges are successfully addressed, performance
shortcomings, cost escalation, and schedule delay are likely.
a
GAO United States General Accounting Office
Why GAO Did This Study
The Coast Guard is developing a Web- based information system to replace an
aging computer system that it uses to track safety and law- enforcement
actions- such as inspections, drug interdiction, and oil spill assistance-
involving commercial and recreational vessels. In 1995, the Coast Guard
awarded a contract to develop the new system, called MISLE. After the
project encountered cost and schedule problems, development responsibility
was transferred to the Coast Guard?s systems development center in 1999, and
the contract terminated after about $26 million had been spent.
The Coast Guard?s history of systems development problems and information
technology weaknesses prompted the Subcommittee to ask GAO to review MISLE?s
current status and risks.
October 2001 COAST GUARD Update on Marine Information for Safety and Law
Enforcement System
This is a test for developing Highlights for a GAO report. For additional
information about the report, call Linda Koontz at (202) 512- 6240. To
provide comments on this test Highlights page, call Keith Fultz at (202)
512- 3200 or send an email to HighlightsTest@ gao. gov.
Highlights of GAO- 02- 11, a report to the Subcommittee on Coast Guard and
Maritime Transportation, House Committee on Transportation and
Infrastructure
What GAO Recommends
GAO is making several recommendations in the four key areas of system
requirements, software testing, transition planning, and risk mitigation.
In comments on a draft of our report, Coast Guard officials generally agreed
with GAO?s recommendations, but noted that some of the risks were ?tradeoff
decisions,? necessary to deliver an initial version of MISLE.
United States General Accounting Office
G A O Accountability Integrity Reliability
Highlights
Page 1 GAO- 02- 11 Coast Guard United States General Accounting Office
Washington, D. C. 20548 Page 1 GAO- 02- 11 Coast Guard
A
Page 2 GAO- 02- 11 Coast Guard
Page 3 GAO- 02- 11 Coast Guard
Page 4 GAO- 02- 11 Coast Guard
Appendix I
Appendix I GAO?s September 18, 2001, Briefing
Page 5 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 6 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 7 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 8 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 9 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 10 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 11 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 12 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 13 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 14 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 15 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 16 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 17 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 18 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 19 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 20 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 21 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 22 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 23 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 24 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 25 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 26 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 27 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 28 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 29 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 30 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 31 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 32 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 33 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 34 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 35 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 36 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 37 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 38 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 39 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 40 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 41 GAO- 02- 11 Coast Guard
Appendix I GAO?s September 18, 2001, Briefing
Page 42 GAO- 02- 11 Coast Guard
United States General Accounting Office Washington, D. C. 20548- 0001
Official Business Penalty for Private Use $300
Address Correction Requested Presorted Standard
Postage & Fees Paid GAO Permit No. GI00
*** End of document. ***