Information Technology: Executive office for U.S. Attorneys Needs
to Institutionalize Key IT Management Disciplines (25-JUL-03,
GAO-03-751).
The Executive Office for United States Attorneys (EOUSA) of the
Department of Justice is responsible for managing information
technology (IT) resources for the United States Attorneys'
Offices. GAO was asked to determine the extent to which EOUSA has
institutionalized key IT management capabilities that are
critical to achieving Justice's strategic goal of improving the
integrity, security, and efficiency of its IT systems.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-03-751
ACCNO: A07719
TITLE: Information Technology: Executive office for U.S.
Attorneys Needs to Institutionalize Key IT Management Disciplines
DATE: 07/25/2003
SUBJECT: Information technology
Internal controls
Legal information systems
Information resources management
Computer security
Strategic planning
Best practices
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-03-751
A
Report to Congressional Requesters
July 2003 INFORMATION TECHNOLOGY Executive Office for U. S. Attorneys
Needs to Institutionalize Key IT Management Disciplines
GAO- 03- 751
Contents Letter 1
Results in Brief 2 Background 3 EOUSA Has Yet to Institutionalize Key IT
Management Disciplines 6
Conclusions 23 Recommendations 23 Agency Comments and Our Evaluation 25
Appendixes
Appendix I: Objective, Scope, and Methodology 29
Appendix II: Assessment of ECMS Acquisition Practices against Level 2 of
SEI*s Software Acquisition Capability Maturity Model 32
Appendix III: Comments from the Department of Justice 44 GAO Comments 50
Appendix IV: GAO Contact and Staff Acknowledgments 54 GAO Contact 54
Acknowledgments 54
Tables Table 1: Summary of Version 1.0 of GAO*s EA Management Maturity
Framework Stages 9
Table 2: Assessment of EOUSA*s EA Efforts against GAO*s EA Maturity
Framework 10 Table 3: The Five Stages of GAO*s ITIM Maturity Framework 12
Table 4: Assessment of EOUSA*s ITIM Efforts against Stage 2 of
GAO*s ITIM Maturity Framework 13 Table 5: SA- CMM Levels and Descriptions
21 Table 6: Assessment of ECMS Acquisition against SEI*s SA- CMM
Level 2 Key Process Area 22 Table 7: Software Acquisition Planning 32
Table 8: Solicitation 34 Table 9: Requirements Development and Management
36 Table 10: Project Management 38 Table 11: Contract Tracking and
Oversight 40 Table 12: Evaluation 42
Figures Figure 1: Simplified Diagram of EOUSA*s Network Connections 4
Figure 2: Estimated IT Expenditures for Fiscal Year 2003 6
Abbreviations
CIO chief information officer EA enterprise architecture ECMS Enterprise
Case Management System EOUSA Executive Office for United States Attorneys
GWAC Governmentwide Acquisition Contracts IRB Investment Review Board IT
information technology ITIM IT investment management JCN Justice
Consolidated Network LIONS Legal Information Office Network System SA- CMM
Software Acquisition Capability Maturity Model SEI Software Engineering
Institute USAO United States Attorneys* Office VANITS Value Added Niche
Information Technology Service VPN virtual private network WAN wide- area
network
This is a work of the U. S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
Letter
July 25, 2003 The Honorable F. James Sensenbrenner, Jr. Chairman The
Honorable John Conyers, Jr. Ranking Minority Member Committee on the
Judiciary House of Representatives The Honorable Christopher Cannon
Chairman The Honorable Melvin L. Watt Ranking Minority Member Subcommittee
on Commercial and Administrative Law House of Representatives This report
is one of a series in response to your request that we evaluate the
management activities of the Executive Office for United States Attorneys
(EOUSA) and the U. S. Attorneys* Offices (USAO). As part of this
request, you asked us to determine the extent to which EOUSA* the
organization responsible for managing information technology (IT)
resources for the USAOs* has institutionalized key IT management
capabilities critical to achieving the Department of Justice*s strategic
goal of improving the integrity, security, and efficiency of its IT
systems. As agreed, to meet this objective, we focused on whether EOUSA
had institutionalized four important IT management disciplines: enterprise
architecture management, investment management, system acquisition
management, and security management. Research shows that these disciplines
are institutionally employed by leading public and private sector
organizations. They are also provided for in legislation and federal
guidance. 1 Details on our objective, scope, and methodology are in
appendix I.
1 See, for example, Clinger- Cohen Act of 1996, Public Law 104- 106;
Office of Management and Budget, Management of Federal Information
Resources, Circular No. A- 130 (February 1996); Office of Management and
Budget, Funding Information Systems Investments,
Memorandum M- 97- 02; and National Institute of Standards and Technology,
Generally Accepted Principles and Practices for Securing Information
Technology Systems, NIST SP 800- 14 (September 1996).
Results in Brief To varying degrees, EOUSA has partially defined and
implemented each of four key IT management disciplines that are critical
to successfully
achieving the Department of Justice*s strategic goal of improving the
integrity, security, and efficiency of its IT systems. However, it has yet
to institutionalize any of these disciplines* meaning that it has not
defined its policies and procedures in accordance with relevant guidance*
and it has yet to fully implement what it has defined. In particular,
while EOUSA has an enterprise architecture* a blueprint for guiding
operational and technological change* it has not developed the
architecture in accordance
with certain best practices. In addition, while the office has implemented
certain process controls for selecting, controlling, and evaluating its IT
investments, it has not yet implemented others that are necessary in order
to develop an effective foundation for investment management. Further, it
has not implemented important management practices that are associated
with an effective security program, such as implementing and monitoring
security policies and controls and promoting security awareness. To its
credit, the office has defined* and is implementing on a major system*
most, though not all, of the management practices associated with
effective systems acquisition.
The institutionalization of these IT management disciplines has not been a
sufficiently high priority for EOUSA, as evidenced by the absence of plans
for fully implementing best practices for each discipline and, in some
cases, an absence of requisite resources. Until each discipline is given
the priority it deserves, EOUSA will not have the IT management
capabilities that are critical to effectively achieving the Justice
Department*s strategic goal of improving the integrity, security, and
efficiency of its IT systems. We are making recommendations to the
Attorney General to strengthen management of each of these disciplines.
In its written comments on a draft of this report, EOUSA agreed with our
specific findings for three of the four IT management disciplines, and
stated that it would address the weaknesses that we identified in each.
However, it stated that its security program is strong, citing a number of
security initiatives, including ones recently planned or started that are
consistent with our recommendations for addressing security weaknesses. We
do not question the initiatives that EOUSA cited. However, our analysis of
its security program, including these initiatives, identified serious
security weaknesses, and as a result, we do not agree that EOUSA*s
security program is strong. Further, it stated that institutionalization
of each of the IT disciplines is currently an office priority and that the
state of its IT
management capabilities is not an impairment to achieving departmental
strategic goals. However, it did not dispute either of our two reasons for
concluding otherwise; namely, that it did not have a plan for fully
implementing best practices for each discipline, and it had not allocated
adequate resources to support such a plan. EOUSA*s comments are summarized
and evaluated in the Agency Comments and Our Evaluation section of this
report.
Background U. S. Attorneys prosecute criminal cases brought forward by the
federal government, prosecute and defend civil cases in which the United
States is
a party, and collect debts owed to the federal government that are
administratively uncollectible. EOUSA was established in 1953 as a
component of the Department of Justice to, among other things, provide
general executive assistance and administrative and operational support to
the 93 USAOs located throughout the 50 states, the District of Columbia,
Guam, the Marianas Islands, Puerto Rico, and the U. S. Virgin Islands 2
and to coordinate with other Department of Justice organizational units
and other federal agencies on behalf of the U. S. Attorneys. One of
EOUSA*s key
responsibilities is managing the USAOs* IT resources, including preparing
their annual IT budget submissions and supporting their acquisition and
maintenance of IT assets. IT plays an important role in helping the USAOs
meet their mission
objectives and, according to EOUSA planning documents, the USAOs* reliance
on IT is to increase in response to expected growth in the number and
complexity of their cases. Currently, EOUSA manages an IT environment
consisting of central and distributed computing and communication
resources in Washington, D. C., and 93 USAOs, respectively.
Connectivity among these offices, Justice headquarters, and Justice*s Data
Center in Rockville, MD, 3 is through a virtual private network (VPN) 4
connection on the Justice Consolidated Network (JCN), with such security
2 U. S. Attorneys* Offices and their branches comprise over 240 sites. 3
Access to the Internet and to research services is provided through
Justice*s Rockville, MD, Data Center. 4 A virtual private network uses a
public or shared telecommunication infrastructure to provide remote users
with secure access to an organization's network.
safeguards as firewalls 5 between USAO local area networks and JCN. The
VPN/ firewall combination, which provides the foundation for secure
communications between EOUSA and the sites mentioned above, is currently
being replaced. Figure 1 generally depicts EOUSA*s network topology. The
USAOs* support is also provided by such application systems as the Legal
Information Office Network System, which is a case management system that
compiles, maintains, and tracks information about defendants, crimes,
criminal charges, court events, and witnesses, and the Victim Notification
System, which notifies crime victims of the status of their cases and
assists with checking compliance with regulations and policies concerning
victim notification.
Figure 1: Simplified Diagram of EOUSA*s Network Connections
5 Network firewalls are devices or systems that control the flow of
traffic between networks with different security requirements.
Organizations employ firewalls to prevent unauthorized access to their
respective systems and resources.
Recognizing the importance of IT to achieving the USAOs* mission, EOUSA
appointed a Chief Information Officer (CIO) in May 2001 and assigned the
CIO accountability and responsibility for managing central and distributed
IT resources and services, including
managing the IT budget for the office and all of the USAOs; developing
and acquiring new systems, including case management systems, and
providing support for existing systems;
managing network, telephone, and video communications; and securing IT
assets (data, applications, and supporting networks). In fiscal year 2003,
EOUSA reports that it plans to spend approximately $125 million on about
20 initiatives. Roughly $110 million of this amount is to be spent on IT
infrastructure and office automation projects (e. g., telecommunications
programs). The remainder is to be spent on acquiring
mission support systems (e. g., Enterprise Case Management System (ECMS),
6 Victim Notification System) and maintaining existing ones. Figure 2
shows the breakdown of estimated expenditures for fiscal year 2003.
6 ECMS is intended to replace the Legal Information Office Network System
(LIONS) and a system for reporting data stored in LIONS and to be the
enterprise solution for managing and tracking case workload within the
USAOs.
Figure 2: Estimated IT Expenditures for Fiscal Year 2003
EOUSA Has Yet to Research into the IT management practices that are
employed by leading
Institutionalize Key IT public- and private- sector organizations has
identified key institutional IT
management disciplines that are interrelated and critical to ensuring,
Management
among other things, the integrity, security, and efficiency of IT systems.
Disciplines
These disciplines are also addressed in legislation and federal guidance.
7 7 See, for example, Clinger- Cohen Act of 1996, Public Law 104- 106;
Office of Management and Budget, Management of Federal Information
Resources, Circular No. A- 130 (February 1996); Office of Management and
Budget, Funding Information Systems Investments,
Memorandum M- 97- 02; and National Institute of Standards and Technology,
Generally Accepted Principles and Practices for Securing Information
Technology Systems, NIST SP 800- 14 (September 1996).
They include 1. enterprise architecture management, which involves
defining,
maintaining, and implementing an institutional blueprint that defines both
the business and the supporting technology of the organization*s current
and target operating environments and a roadmap to achieve the target
environment;
2. IT investment management, which involves selecting, controlling, and
evaluating a portfolio of investments within the context of an enterprise
architecture;
3. IT security management, which involves protecting the integrity,
confidentiality, and availability of an organization*s IT assets (e. g.,
data, application systems, and networks) and reducing the risks of
tampering, unauthorized intrusions and disclosures, and disruption of
operations. 4. system acquisition management, which involves managing
selected
investments (system projects) in a manner that increases the probability
of promised system capabilities being delivered on time and within budget.
As we have previously reported, 8 to successfully institutionalize these
disciplines, organizations should develop integrated plans to guide their
efforts that (1) specify measurable goals, objectives, and milestones; (2)
specify needed resources; and (3) assign clear responsibility and
accountability for accomplishing well- defined tasks. In addition, these
plans should be approved by senior management. In implementing these
plans, it is important that organizations allocate adequate resources and
measure and report progress against planned commitments and that
appropriate corrective actions be taken to address deviations. EOUSA has
defined and implemented each of the four IT management disciplines
mentioned above to some degree. However, none has been institutionalized,
meaning that they are not fully defined in accordance
8 U. S. General Accounting Office, Information Technology: INS Needs to
Better Manage the Development of Its Enterprise Architecture, GAO/ AIMD-
00- 212 (Washington, D. C.: August 2000); Information Technology: DLA
Needs to Strengthen Its Investment Management Capability, GAO- 02- 314
(Washington, D. C.: March 2002).
with best practices and what has been defined has not been fully
implemented. While these disciplines have been given attention since the
recent appointment of the CIO, they have not been treated as priorities,
in that action plans needed for successful institutionalization have not
been developed or resourced. As a result, EOUSA is currently limited in
its ability to meet Justice*s strategic goal of improving its IT systems,
and the USAOs will be challenged in their ability to effectively and
efficiently meet their mission goals and priorities.
EOUSA Is Not Performing An enterprise architecture (EA) is an investment
blueprint that defines,
Important Practices both in logical terms (including business functions
and applications, work
Associated with Effective locations, information needs and users, and the
interrelationships among
Enterprise Architecture these variables) and in technical terms (including
hardware, software, data
communications, and security) how an organization operates today (* as
Management
is*), how it intends to operate tomorrow (* to be*), and a roadmap for
transitioning from today to tomorrow. The development, maintenance, and
implementation of architectures are recognized hallmarks of successful
public and private organizations. According to a guide published by the
federal CIO Council, 9 effective architecture management consists of a
number of core elements.
In February 2002, we published version 1.0 of our EA management maturity
framework, which arranges the core elements of the CIO Council*s guide
into five hierarchical stages. 10 The framework provides an explicit
benchmark for gauging the effectiveness of architecture management and
provides a roadmap for making improvements. Table 1 summarizes the
framework*s five stages of maturity.
9 Chief Information Officers Council, A Practical Guide to Federal
Enterprise Architecture, Version 1. 0 (February 2001). 10 U. S. General
Accounting Office, Information Technology: Enterprise Architecture Use
across the Federal Government Can Be Improved, GAO- 02- 6 (Washington, D.
C.: February 2002). We issued version 1.1 of the framework in April 2003*
A Framework for Assessing and Improving Enterprise Architecture
Management, GAO- 03- 584G* but did not use it for our evaluation because
we had already completed our audit work evaluating EOUSA against the
initial framework.
Tabl e 1: Summary of Version 1.0 of GAO*s EA Management Maturity Framework
Stages Stage Description
Stage 5: Leveraging the EA to This stage entails, among other things,
incorporating the manage change EA into corporate decision making to (1)
avoid (includes all elements in unwarranted overlap across investments,
(2) enable stage 4) maximum systems interoperability, and (3) ensure
selection and funding of IT investments with manageable risks and returns.
Stage 4: Completing the EA This stage is characterized by having EA
products that products have been approved by the EA steering committee
(includes all elements in
(established in stage 2) or an investment review board, stage 3)
and by the CIO. Stage 3: Developing the EA
This stage focuses on developing EA products according products to the
selected framework, methodology, tool, and (includes all elements in
established management plans. stage 2)
Stage 2: Building the EA This stage focuses on assigning EA management
roles management foundation and responsibilities, establishing plans for
developing EA
products, and committing the resources necessary for developing these
products.
Stage 1: Creating EA This stage is characterized either by no plans to
develop awareness and use an EA or by plans that do not demonstrate an
awareness of the value of having and using an EA.
Source: GAO.
EOUSA has satisfied many of the framework*s core elements. Specifically,
it has satisfied about 80 percent of the elements associated with building
the EA management foundation* stage 2 of our EA management maturity
framework* and half of the 12 core elements associated with higher
maturity stages. At stage 2, it has established a chief architect and has
selected a framework (the Federal Enterprise Architecture Framework) and,
according to officials, selected a tool (the Enterprise Architecture
Management System) to serve as a repository for its EA artifacts. At the
higher stages of our framework, the CIO, for example, approved a version
of an EA in May 2002 that describes the *as is* and *to be* environments
for its core business functions.
However, the office has yet to satisfy several of the core elements that
are critical to effective EA management. For example, a committee or group
representing the enterprise has not yet been established to guide and
oversee the development of future versions of the architecture. Instead,
the current version of its architecture has been primarily guided and
directed
by the CIO*s office. Until a committee or group representing the
enterprise
is established, there is increased risk that the architecture will not
represent a corporate decision- making tool and will not be viewed and
endorsed officewide as such a tool. Another example is the absence of a
written or approved policy for
maintaining the EA. Without a documented, approved policy for EA
maintenance that, for example, assigns responsibility and accountability
for configuration management and version control, EOUSA risks allowing its
architecture to become outdated and irrelevant, thus limiting its
effectiveness in selecting and guiding IT investments. EOUSA does not have
a written plan of action for strengthening EA
management and evolving the current version of its EA, because, according
to the CIO, developing such a plan is not a priority. Table 2 shows
EOUSA*s performance in addressing the core elements of our maturity
framework.
Tabl e 2: Assessment of EOUSA*s EA Efforts against GAO*s EA Maturity
Framework Stage Core element Satisfied? Comments
Stage 5: Leveraging Written/ approved policy exists for EA No According to
agency officials, there is no the EA for managing maintenance. written/
approved policy for EA change
maintenance. (includes all elements
EA steering committee, investment No The EA has not been reviewed by any
from stage 4)
review board, or agency head has steering committee or investment review
approved EA. board or by the agency head.
Metrics exist for measuring EA benefits. No According to agency officials,
metrics for measuring EA benefits have not been developed.
Stage 4: Completing Written/ approved policy exists for IT No While there
are criteria for ranking IT architecture products
investment compliance with EA. investments that call for determining
(includes all elements compliance with the EA, no from stage 3)
written/ approved policy addresses this. EA products describe the
enterprise*s Ye s EA products describe EOUSA*s core business* and the
data, applications,
business functions and the data,
and technology that support it. applications, and technology that support
them.
EA products describe the *as is* Ye s EA products describe the *as is*
environment, the *to be* environment,
environment, the *to be* environment, and a and a sequencing plan.
sequencing plan. Agency CIO has approved EA. Ye s The CIO approved the EA
in May 2002.
(Continued From Previous Page)
Stage Core element Satisfied? Comments
Stage 3: Developing Written/ approved policy exists for EA
No According to agency officials, there is no architecture products
development. approved policy for EA development. (includes all elements
EA products are under configuration No Although agency officials reported
that EA from stage 2)
management. products are under configuration management, they could not
provide documentation to support this statement. EA products describe or
will describe the
Yes EA products describe core business enterprise*s business* and the
data, functions and the data, applications, and applications, and
technology that support technology that support them.
it. EA products describe or will describe the
Yes EA products describe the *as is* *as is* environment, the *to be*
environment, the *to be* environment, and a environment, and a sequencing
plan.
sequencing plan. EA scope is enterprise- focused. Yes EA products describe
the *as is* and *to be*
environments for the enterprise*s core business functions. Stage 2:
Building the
Committee or group representing the No There is no committee or group EA
management
enterprise is responsible for directing, representing the enterprise that
is foundation
overseeing, or approving EA. responsible for directing, overseeing, or
approving the EA. The CIO is currently responsible for direction,
oversight, and approval of the architecture. Program office responsible
for EA Yes Roles and responsibilities for developing the development
exists. EA were assigned to a group of individuals.
Chief architect exists. Yes The CIO has been designated as the chief
architect.
EA is being developed using a framework Yes The EA was developed using the
Federal and an automated tool. Enterprise Architecture Framework. In
addition, agency officials reported that they are using the Enterprise
Architecture Management System as their EA tool. EA plans call for
describing the
Yes EA products describe core business enterprise in terms of business,
data, functions and the data, applications, and applications, or
technology. technology that support them.
EA plans call for describing *as is* Yes EA products describe the *as is*
environment, *to be* environment, or environment, the *to be* environment,
and a sequencing plan.
sequencing plan. Stage 1: EA
Agency is aware of EA. Yes In August 2002, the CIO issued a memo to
awareness agency staff notifying them of the need to
use the EA to inform investment management decisions. Source: GAO.
EOUSA Has Not Established Effective IT investment management provides for
evaluating each
Key Capabilities proposed and ongoing investment, based on EA alignment
and measurable Needed to Effectively risks and returns and for selecting
and controlling these investments as a
Manage IT Investments portfolio of competing investment options. We have
developed a
framework that defines and measures an organization*s maturity in IT
investment management (ITIM) and provides a basis for improving investment
management. 11 This framework, which is based on the IT investment
management practices of leading private- and public- sector
organizations, is structured to permit progression through five maturity
stages (shown in table 3). Each maturity stage consists of critical
processes and key practices that should be implemented for an organization
to become more effective in managing its IT investments.
Table 3: The Five Stages of GAO*s ITIM Maturity Framework Stage
Description
Stage 5: Investment benchmarking and IT- enabled change management
Leveraging IT for
techniques are deployed to strategically shape business outcomes.
strategic outcomes Stage 4:
Process evaluation techniques focus on improving the performance Improving
the
and management of the organization*s IT investment portfolio. investment
process
Stage 3: Comprehensive techniques are in place for selection and control
of Developing a the IT investment portfolio that incorporate benefit and
risk criteria complete investment linked to mission goals and strategies.
portfolio
Stage 2: Repeatable investment control techniques are in place and the key
Building the foundation capabilities have been implemented. investment
foundation
Stage 1: IT management processes are ad hoc, project- centric, and have
Creating investment widely variable outcomes. awareness Source: GAO.
According to the framework, the first key step toward an effective
investment management process is to build the investment foundation. An 11
U. S. General Accounting Office, Information Technology Investment
Management: A
Framework for Assessing and Improving Process Maturity (Exposure Draft),
GAO/ AIMD10.1.23 (Washington, D. C.: May 2000).
organization with this foundation (stage 2 maturity) has attained
repeatable, successful investment control processes and basic selection
processes at the project level. Successful management at this level allows
an organization to measure the progress of existing IT projects and to
identify variances in cost, schedule, and performance expectations by
following established, disciplined processes. The organization should also
be able to take corrective action, if appropriate, and should possess
basic capabilities for selecting new project proposals. To accomplish this
level of basic control, an organization should establish an investment
board, identify the business needs and opportunities to be addressed by
each project, and use this knowledge in the selection of new proposals.
The office has satisfied two of the critical processes for stage 2, but it
has not satisfied the other three. Specifically, it has established an
investment governing board, known as the Investment Review Board (IRB) and
developed a guide to direct its operations. It is also defining project
needs in alignment with the agency*s mission goals. However, the office
has not, for example, defined procedures for project oversight. In
addition, while an
IT project and systems inventory exists as part of its *as is*
architecture, a policy specifying how it will be used for investment
management purposes has not been defined. Until EOUSA satisfies all
critical processes for stage 2, it will not have the foundation it needs
to build its investment management capability and it will not have an
effective investment
process. Table 4 summarizes our assessment of stage 2 capabilities.
Tabl e 4: Assessment of EOUSA*s ITIM Efforts against Stage 2 of GAO*s ITIM
Maturity Framework Critical process Description Satisfied? Comments
IT investment board Define and establish (1) the governing Ye s A
governing board (the IRB) has been operation board( s) responsible for
selecting,
defined and established, and guidance controlling, and evaluating IT
investments
directing the board*s operations exists. and (2) a guide directing the
board( s) operations.
IT project oversight Regularly oversee each IT project*s No According to
agency officials, projects are progress toward cost and schedule managed
using earned value management a milestones, using established criteria,
to regularly determine project progress and and require corrective actions
when
control project costs and schedules. milestones are not achieved.
However, there are no procedures defining how the investment board is to
regularly oversee each project*s progress and require corrective actions
when milestones are not achieved.
(Continued From Previous Page)
Critical process Description Satisfied? Comments
IT project and system Create and maintain an IT project and No An IT
project and systems inventory exists identification systems inventory to
assist in managerial
as part of the EA. However, there is no decision making.
policy defining how this inventory is to be used in managerial decision
making. Business needs
Ensure that each IT project supports the Yes Business needs and associated
users have identification for IT organization*s business needs and meets
been identified, and users participate in the projects
users* needs. management of the project. We verified this for the
Enterprise Case Management System project, which officials told us is
representative of how they intend to acquire systems. Proposal selection
Ensure that an established, structured No A selection process using risk
and return process is used to select new IT
criteria is defined. However, officials stated proposals. that this
selection process has not yet been used.
Source: GAO. a Earned value is a management technique that relates
resource planning to schedules and to technical
cost and schedule requirements. There are two major objectives of an
earned value system: to encourage contractors to use effective internal
cost and schedule management control systems, and to permit the customer
to rely on timely data produced by those systems for determining
productoriented
contract status.
EOUSA has not demonstrated that maturing its IT investment management
process is a priority by developing a plan for doing so and devoting
resources to execute the plan. Until the office develops and implements a
plan for establishing mature IT investment management processes
(including all critical processes for building the investment management
foundation), EOUSA will not have the full suite of capabilities it needs
to ensure that project selection and control processes are repeatable or
that it has the best mix of investments to meet agency priorities.
EOUSA Has Not Effective information security management is critical to
EOUSA*s ability to
Implemented Effective ensure the reliability, availability, and
confidentiality of its information
Security Practices assets, and thus it is fundamental to its ability to
perform its mission. Our
research into public- and private- sector organizations with strong
information security programs shows that leading organizations* programs
include (1) establishing a central security focal point with appropriate
resources, (2) continuously assessing business risks, (3) implementing and
maintaining policies and controls, (4) promoting awareness, and (5)
monitoring and evaluating the effectiveness of policies and controls. 12
Currently, EOUSA is not fully satisfying any of these tenets of effective
security. In addition, it has not demonstrated that institutionalizing
effective security practices is a priority by developing a plan to guide
its efforts to address security weaknesses and committing resources to
perform essential security functions. Until such a plan is developed and
effectively implemented, data, systems, and networks are at risk of
inadvertent or deliberate misuse, fraud, improper disclosure, or
destruction* possibly without detection. For example, the reliability and
integrity of case information may be compromised, or sensitive crime
victim information may be improperly disclosed.
Central Security Focal Point Is According to our framework, central
management of key security functions
Established but Has Not Been is the foundation of an effective information
security program, because it
Appropriately Resourced allows knowledge and expertise to be incorporated
and applied on an
enterprisewide basis. Having a central security focal point supported by
appropriate resources is especially important for managing the increased
risks associated with a highly connected computing environment, such as
JCN, where security weaknesses in one segment of an organization*s network
can compromise the security of another segment*s IT assets. In addition,
centralizing the security management function provides a focal point for
coordinating the activities associated with the other four elements of a
strong information security program.
12 U. S. General Accounting Office, Information Security Management:
Learning from Leading Organizations, GAO/ AIMD- 98- 68 (Washington, D. C.:
May 1998).
In June 2001, EOUSA appointed a security officer with responsibility for
centrally managing all aspects of IT security. However, EOUSA has not
assigned sufficient staff to adequately carry out these responsibilities.
For example, no staff has been assigned to monitor firewall logs 13 or
support the development of a centrally managed IT security training
program* activities that fall under the security officer*s purview. Each
of these activities is discussed further in the following sections.
Officials said that they recognize the need for additional staff resources
to perform these activities. They also stated that they were in the
process of hiring two people to support security functions, but they
agreed that this would still not allow for performance of key security
responsibilities. Without an appropriately resourced security program,
security breaches may not be detected or addressed in a timely manner,
awareness of security requirements across the organization may be
inconsistent, and vulnerabilities in the current IT environment may not be
appropriately addressed.
Risks Have Not Always Been According to our framework, identifying and
assessing business risks is an
Assessed essential step in determining what IT security controls are
needed and
what resources should be invested in these controls. Federal guidance
advocates performing risk assessments at least once every 3 years* or when
a significant change in a system or the systems environment (e. g., new
threats) has occurred. These assessments should address the risks
that are introduced through connections to other networks and the impact
on an organization*s mission should network security be compromised. In
line with this guidance, EOUSA*s certification and accreditation 14
process
requires that a risk assessment be completed for each system before any
office can use it.
13 Logs keep track of accesses to and attempts to access the networks that
the firewalls are intended to secure. 14 Certification is the technical
and nontechnical evaluation that is conducted to verify that IT systems
comply with security requirements. Accreditation is the formal declaration
that the appropriate safeguards have been properly implemented and that
the residual risk is acceptable.
According to EOUSA, a major system that recently underwent EOUSA*s
certification and accreditation process is the replacement for the
existing firewall/ VPN system. This system is intended to be the
foundation for secure communications between EOUSA, Justice, and the
geographically dispersed USAOs. Accordingly, we analyzed this system and
found that while the firewall/ VPN replacement system has been certified
and accredited, the existing firewall/ VPN system* which was deployed in
1996 and, as of May 9, 2003, was operating at 75 of the 240 sites 15 *had
not had a risk assessment performed and had not been certified and
accredited. Officials told us that they have not performed such an
assessment on this network because (1) it is not cost- effective to use
limited resources to perform an assessment on a network that is to be
fully replaced by June 30, 2003, and (2) the risks inherent in the network
are minimal, given that it resides on Justice*s JCN, for which they said
they assume Justice had performed risk assessments. We agree that it does
not make sense at this point to perform a risk assessment on the existing
firewall/ VPN system given that the replacement
system is expected to be fully deployed by the end of June 2003. However,
this does not change the fact that EOUSA has operated the network for
about 7 years without understanding its exposure to risk. This is
particularly important, because EOUSA officials could not provide us with
evidence to support the assumption that Justice had performed a risk
assessment for JCN. Moreover, previous studies have shown that Justice has
had long- standing weaknesses in several aspects of its IT security
program. 16 According to EOUSA, its recently established certification and
accreditation program will not allow this to happen again.
Key Security Controls Have Not According to our framework, risk- based,
cost- effective security policies
Been Implemented and related technology controls (such as firewalls
configured to explicit
rules and intrusion detection devices 17 located to monitor key network
assets) and procedural controls (such as contingency plans) are needed to
15 EOUSA is responsible for IT operations at 93 geographically dispersed
USAOs and their branches, which comprise about 240 sites. 16 U. S.
Department of Justice, FY 2000 Performance Report- FY 2002 Performance
Plan, April 2001; U. S. General Accounting Office, Major Management
Challenges and Program Risks: Department of Justice, GAO- 03- 105 (January
2003).
17 Intrusion detection devices are software or hardware systems that
monitor network traffic and help identify cyberthreats.
protect a system from compromise, subversion, and tampering. Federal and
Justice guidance also advocate establishing these policies and controls.
While EOUSA is guided by many Justice security policies, it has not yet
implemented key security controls that are needed to satisfy them. For
example, CIO officials told us that the existing firewall/ VPN system,
which, as of May 9, 2003, was operating at 75 sites, is not based on
explicit firewall rules. Moreover, according to these officials, no
intrusion detection devices monitor the wide- area network (WAN) 18
routers, firewalls, and VPN devices. Rather, the intrusion detection
devices that are currently implemented are located only within the local
area network environment (i. e., within a USAO). Also, the contingency
plan developed for the replacement firewall/ VPN system was not prepared
according to federal guidelines. For example, the contingency plan does
not specify procedures for notifying recovery personnel or assessing
damage to systems. CIO officials told us that they had not implemented
these security controls
because, as previously noted, they believe the risks inherent in the
network are minimal given that it resides on Justice*s JCN, for which they
said they assumed Justice had performed risk assessments. However, as
previously stated, EOUSA provided no evidence to support this assumption,
and Justice has had longstanding security weaknesses.
Until EOUSA implements security controls, it may be unaware of
vulnerabilities, increasing the risk that intruders may take control of
network devices or that data passing through its firewalls can be read or
manipulated. In addition, EOUSA may not be able to respond to security
breaches adequately and in a timely manner. This is particularly
threatening given the sensitivity of the information used by the USAOs in
performing their work. EOUSA Does Not Adequately
According to our framework, promoting user awareness through education
Promote User Awareness
and training is essential to successfully implementing information
security policies, achieving user understanding of security policies, and
ensuring that security controls are instituted properly. This is because
computer
users* and others with access to information resources* are not able to
comply with policies of which they are unaware or which they do not fully
understand. Our framework suggests that a central group be tasked with
18 A wide- area network is a network that provides data communications to
a large number of independent users and spans a relatively large
geographical area.
educating users about current information security risks and helping to
ensure consistent understanding and administration of policies. As
previously mentioned, the security officer is responsible for promoting
awareness of computer security. However, the security officer does not
carry out this responsibility because provision of the resources to do so
has not been viewed as an agency priority. According to the security
officer, each district is thus responsible for managing its own IT
training program, and the security officer does not know to what extent
these programs
address awareness of computer security. Without a centralized approach to
security education and training, the security officer cannot adequately
ensure that users are consistently aware of or fully understand the
organizational policies and procedures with which they are expected to
comply, thus risking the integrity, reliability, and confidentiality of
data and systems. According to EOUSA officials, they plan to hire staff to
develop and implement a centralized program by August 2003.
EOUSA Is Not Monitoring the Our framework recognizes the need to
continuously monitor controls, Effectiveness of Security
through tests and evaluations, to ensure that the controls have been
Controls
appropriately implemented and are operating as intended. Further,
Justice*s policy requires annual testing of security controls and requires
EOUSA to (1) verify that the policies and procedures in component
organizations are consistent with this policy and (2) enforce compliance
with component and Justice security policies, including identifying
sanctions and penalties for noncompliance. In addition, our framework and
related best practices* as well as Justice*s own policy* advocate keeping
summary records of security incidents, to allow measurement of the
frequency of various types of violations and the damage suffered from
these incidents. This type of oversight is critical because it enables
management to identify problems and their causes* and to make the
necessary corrections.
CIO officials told us that testing has never been conducted to determine
whether EOUSA*s policies and procedures are consistent with Justice*s and
whether security controls are generally effective. According to these
officials, testing has not been a priority because they assumed that
Justice was performing tests of the WAN environment. However, Justice
officials told us that, although they had evaluated the contractor*s
management of the WAN*s circuits, they had not performed any tests to
determine the effectiveness of technical and other controls associated
with the WAN. The lack of testing heightens the risk that individuals both
within and outside Justice could compromise EOUSA*s external and internal
security controls
to gain extensive unauthorized access to its networks and to networks to
which it is connected.
EOUSA officials also told us that, contrary to Justice*s policy, they do
not maintain summary records of security incidents. Specifically, the
production firewall/ VPN software and routers at over 240 locations do not
have audit logs that are activated, and the replacement routers,
firewalls, and VPN devices are being implemented with no audit logs
activated. According to these officials, they have not activated the audit
logs because resources have not been allocated to provide for this
security control. This lack of auditing heightens the risk of undetected
intruders using EOUSA*s systems to modify, bypass, or negate its firewalls
and routers. Additionally, without these audit logs the office would be
unable to reconstruct securityrelated incidents.
EOUSA Is Employing Rigorous and disciplined system acquisition processes
and practices can Important Acquisition
reduce the risk of fielding systems that do not perform as intended, are
Management Practices on a
delivered late, or cost more than planned. The Software Engineering Key
System
Institute (SEI), recognized for its expertise in acquiring software-
intensive systems, has published models and guides for determining an
organization*s acquisition process maturity. One of those models, referred
to as the Software Acquisition Capability Maturity Model (SA- CMM), 19
addresses an organization*s acquisition management ability. 20 The SA- CMM
model defines organizational maturity according to five levels (see table
5). 19 Carnegie Mellon University*s Software Engineering Institute,
Software Acquisition Capability Maturity Model, Version 1.02, CMU/ SEI-
99- TR- 002 (April 1999). 20 EOUSA officials told us that they primarily
acquire their systems.
Tabl e 5: SA- CMM Levels and Descriptions Level Description
Level 5: Continuous process improvement is empowered by quantitative
feedback Optimizing from the process and from piloting innovative ideas
and technologies.
Level 4: Detailed measures of the acquisition processes, products, and
services are Quantitative quantitatively and qualitatively understood and
controlled. Level 3: The acquisition organization*s software acquisition
process is documented Defined and standardized. All projects use an
approved, tailored version of the
organization*s standard process for acquiring their products and services.
Level 2: Basic management processes for acquisition projects are
established to Repeatable plan all aspects of the acquisition, manage
requirements, track project
team and contractor team performance, manage the project*s cost and
schedule baselines, evaluate the products and services, and successfully
transition to its support organization. The necessary process discipline
is in place to repeat earlier successes on projects in similar domains.
Level 1: The acquisition process is characterized as ad hoc, and
occasionally even Initial chaotic. Few processes are defined and success
depends on individual effort. Source: SEI.
According to SEI, level 2 (the repeatable level) demonstrates that basic
management processes, known as key process areas, have been established to
track performance, cost, and schedule, and that the organization has the
means to repeat earlier successes on similar projects. An organization
that has these processes in place is in a much better position to
successfully acquire software- intensive systems than an organization that
does not.
As a Justice component, EOUSA must comply with all departmental policies
and procedures, including Justice*s system development life- cycle
management guidance. Since EOUSA officials told us that the Enterprise
Case Management System (ECMS), which is intended to be the enterprise
solution for managing and tracking case workload within the USAOs, is the
first acquisition effort to follow Justice guidance from its inception, we
compared this project, and the Justice guidance used to manage it, against
SEI*s SA- CMM, and we found that the project was being managed in
accordance with the majority of the applicable level 2 practices. Table 6
represents a summary of our findings for this acquisition (see app. I for
an expanded analysis).
Table 6: Assessment of ECMS Acquisition against SEI*s SA- CMM Level 2 Key
Process Area Key Key practices SA- CMM level 2 key process
Total key practices not
area Description practices performed
performed
Software acquisition planning Ensure that reasonable planning for the
acquisition is conducted and that all elements of the project are
included. 15 13 2
Solicitation Ensure that award is made to the contractor most capable of
satisfying the specified requirements. 18 16 2 Requirements development
and
Establish a common and unambiguous definition of acquisition management
requirements to be used by the acquisition team, the system*s users, and
the contractor. 14 14 0
Project management Manage the activities of the project office and
supporting contractor( s) to ensure a timely, efficient, and effective
acquisition. 16 16 0
Contract tracking and oversight Ensure that the activities under contract
are being performed in accordance with contractual requirements and that
products and services will satisfy contract requirements. 17 16 1
Evaluation Determine that the acquired products and services satisfy
contract requirements before accepting and supporting them. 15 6 9
Source: GAO. More specifically, the office has performed all of the key
practices in the
requirements development and management and project management key process
areas. These include (1) establishing a written policy for developing and
managing system- related contractual requirements; (2) having bi-
directional traceability between the contractual requirements and the
contractor*s work products and services; (3) measuring and reporting to
management on the status of requirements development and management
activities; (4) designating responsibility for project management; (5)
keeping plans current during the life of the project as replanning occurs,
issues are resolved, requirements are changed, and new risks are
discovered; and (6) tracking the risks associated with cost, schedule,
resources, and the technical aspects of the project.
EOUSA has also performed the majority of the key practices in the
remaining four process areas. However, it does not have written policies
for either the contract tracking and oversight or the software acquisition
planning key process areas. Policies in general are key to establishing
well- defined and enduring processes and procedures. In these two areas,
policies would ensure that the office*s approach to tracking and
overseeing contractors and planning the acquisition is defined in a
repeatable and
measurable fashion. In addition, during the solicitation process, the
office did not document its plans for solicitation activities, which would
provide those involved with objectives for the solicitation process and a
defined way to manage and control solicitation activities and decisions.
In evaluation, the office has yet to satisfy 9 of the 15 required
practices.
Officials told us that they intend to satisfy them but that they do not
have a plan for addressing those practices or for implementing all of the
required practices on future system acquisitions. According to these
officials,
developing such a plan is currently not a priority. By developing and
implementing a plan for satisfying all of these key process areas on ECMS
and future acquisitions, EOUSA can increase its chances of successfully
acquiring needed system capabilities on time and within budget.
Conclusions EOUSA has taken important steps to define and implement four
key IT management disciplines. Nevertheless, key aspects of each
discipline have
yet to be institutionalized, leaving the office challenged in its ability
to achieve the department*s strategic goal of improving the integrity,
security, and efficiency of its IT systems. Critical to the office*s
success going forward will be treating institutionalization of each of
these management
disciplines as priority matters by developing integrated plans of action
for addressing the weaknesses that we identified in each and effectively
implementing these plans* including assignment of appropriate resources
and measurement and reporting of progress. Without taking these steps,
EOUSA is unlikely to fully establish the IT management capabilities it
needs.
Recommendations To strengthen the office*s IT management capacity and
increase its chances of improving the integrity, security, and efficiency
of its IT systems, we
recommend that the Attorney General direct the EOUSA Director to treat
institutionalization of EA management, IT investment management, IT
security management, and system acquisition management as priorities
by developing and implementing action plans to address the weaknesses in
each discipline that are identified in this report. These plans should, at
a minimum, provide for accomplishing the following:
For EA management,
establish a committee or group representing the enterprise that is
responsible for directing, overseeing, or approving the EA;
ensure that EA products are under configuration management; define,
approve, and implement a policy for IT investment compliance with the EA;
specify metrics for measuring EA benefits; and define, approve, and
implement a policy for maintaining the EA. For IT investment management,
regularly oversee each IT project*s progress toward cost and schedule
milestones, using established criteria, and require corrective actions
when milestones have not been achieved;
define and implement a policy for using the IT project and systems
inventory for managerial decision making; and
ensure that an established, structured process is used to select new IT
proposals.
For IT security management,
allocate the appropriate resources to enable the responsibilities of the
security officer to be fully performed;
ensure that risk assessments are performed on all existing and future
systems;
implement intrusion detection devices to monitor activity at the
routers, firewalls, and VPN devices, and implement other network security
controls as noted in the report;
develop and implement a centralized approach to security education and
training; and
perform regular tests to determine compliance with policies and
procedures and the effectiveness of security controls.
For system acquisition management,
develop and implement a policy for contract tracking and oversight;
develop and implement a policy for system acquisition planning; and
address the remaining key practices associated with evaluation as ECMS
progresses in the life cycle; and ensure that the Software Engineering
Institute acquisition practices
identified in this report are used in future system acquisitions. In
developing these plans, the Director should ensure that each plan (1) is
integrated with the other three plans; (2) defines clear and measurable
goals, objectives, and milestones; (3) specifies resource needs; and (4)
assigns clear responsibility and accountability for implementing the plan.
In implementing each plan, the Director should ensure that the needed
resources are provided and that progress is measured and reported
periodically to the Attorney General. Agency Comments and
In written comments on a draft of this report signed by the EOUSA Director
Our Evaluation
(reprinted in app. III), the office agreed with our findings relative to
enterprise architecture management, IT investment management, and system
acquisition management. EOUSA also agreed with our recommendations in
these three areas and stated that it intends to implement the
recommendations. However, EOUSA stated that it disagreed with our findings
and our recommendations regarding information security management,
although at the same time it cited certain actions that it intends to
take, such as implementing a centralized security training program and
monitoring security audit logs, that are consistent with our security
findings and associated recommendations. Further, the office disagreed
that the state of its efforts to institutionalize best management
practices in the four areas is due to it not treating each area as an
office priority. It also disagreed with our conclusion that the state of
its efforts to
institutionalize best practices currently limits its ability to meet
Justice*s strategic goal of improving its IT systems, and that the USAOs
will be challenged in their ability to effectively and efficiently meet
mission goals
and priorities. Each of these three areas of disagreement is addressed
below.
First, with respect to information security management, EOUSA stated that
it has one of the strongest security programs in Justice, and perhaps the
federal government. To support this statement, the office cited 10
security initiatives it has implemented, such as certification and
accreditation of more than eight systems, real- time encryption of all
data in laptops and handheld devices, and conduct of vulnerability
assessments and penetration testing. It also noted, among other things,
that it had added 10 field security positions and 2 headquarters
positions, and that its data are monitored 24 hours a day, seven days a
week, and have never been compromised. We do not question these statements
concerning the office*s information security program and associated
activities because (1) the
purpose and scope of our review was not to compare EOUSA to other Justice
component organizations or other federal agencies, and thus EOUSA*s
relative standing is not relevant to the findings in our report and (2)
the message of our report is not that EOUSA has not taken steps to improve
its information security posture, but rather that the office*s information
security management efforts, including ongoing and complete
improvement steps, are weak in a number of areas relative to information
security management best practices. Accordingly, we make recommendations
aimed at addressing identified weaknesses, including a recommendation to
implement network intrusion detection devices and other security controls.
While EOUSA*s comments cited plans that are consistent with many of our
security- related recommendations, it disagreed with the recommendation
relative to its wide area network on
the grounds that this network is managed, secured, and monitored by
Justice and Sprint. We understand that the WAN is not managed by EOUSA,
and accordingly our recommendation was aimed at actively monitoring the
network routers, firewalls, and VPN devices, which are managed by EOUSA.
To avoid any confusion about this recommendation, we have clarified its
wording to better reflect our intentions. Similarly, in light of the
recent progress that EOUSA has made replacing its VPN system, we have
adjusted our finding and recommendation concerning the office*s exposure
to risk from its old VPN system.
Second, with respect to our statements that EOUSA has not treated
institutionalization of each of the four IT management disciplines*
enterprise architecture management, IT investment management, system
acquisition management, and information security management* as agency
priorities, the office stated that these statements were unfair and
that it did not agree with them. To support its position, EOUSA made the
following two points: (1) it has made tremendous progress, as evidenced by
our report recognizing those best practices that it is satisfying, and (2)
it
has received the highest level of support from Justice, as evidenced by
the establishment of the EOUSA CIO position in 2001, the progress that has
been made in the last 2 years compared to other Justice component
organizations, and EOUSA*s being viewed by Justice senior management as a
leader in IT management. We do not challenge EOUSA*s two points because
they are not relevant to our position regarding treating
institutionalization of each of the four IT management disciplines as
agency priorities. Our position is based on two facts that EOUSA did not
dispute:
(1) plans for addressing the weaknesses cited in our report do not exist
and (2) limitations in resources to address these weaknesses were cited by
EOUSA officials as the reason why the weaknesses exist. In our view, if
each of these areas were an agency priority, then plans would be in place
to address the weaknesses, and resources to execute the plans would be
committed.
Third, with respect to our conclusion that EOUSA is currently limited in
its ability to meet Justice*s strategic goal of improving its IT systems,
and that the USAOs are thereby challenged in their ability to effectively
and efficiently meet their mission goals and objectives, the office
disagreed but did not offer any comments to counter our conclusion beyond
those cited above. Given that any organization*s ability to effectively
leverage
technology is determined in large part by its institutionalized
capabilities in these four IT disciplines, we have not modified our
conclusion.
EOUSA provided additional comments that have been incorporated in the
report as appropriate. EOUSA*s written comments are reproduced in appendix
III, along with our detailed evaluation of each comment.
As arranged with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days
after the date of this letter. At that time, we will send copies of this
report to interested congressional committees. We will also send copies to
the Director of the Office of Management and Budget, the Attorney General
of the United States, the EOUSA Director, and the EOUSA CIO. We will also
send copies to others upon request. In addition, copies will be available
at no charge on our Web site at www. gao. gov.
Should you or your offices have questions on matters discussed in this
report, please contact me at (202) 512- 3439. I can also be reached by E-
mail at hiter@ gao. gov. An additional GAO contact and staff
acknowledgments are listed in appendix IV.
Randolph C. Hite Director, Information Technology Architecture and Systems
Issues
Appendi Appendi xes x I
Objective, Scope, and Methodology Our objective was to determine the
extent to which the Executive Office for United States Attorneys (EOUSA)
has institutionalized key information technology (IT) management
capabilities to achieve the Department of Justice*s strategic goal of
improving the integrity, security, and efficiency of its IT systems. To
meet this objective, we focused on whether EOUSA had institutionalized
four key IT management disciplines: enterprise architecture management, IT
investment management, information security management, and system
acquisition management.
To evaluate EOUSA*s enterprise architecture (EA) management, we first
solicited responses to an EA management questionnaire, reviewed EA plans
and products, and interviewed officials to verify their responses. Next,
we compared the information that we had collected with GAO*s
February 2002 EA management maturity framework 1 to determine the extent
to which EOUSA was employing effective EA management practices. This
framework is based on the Practical Guide to Federal Enterprise
Architecture, published by the Chief Information Officers* (CIO) Council.
2 We did not use the revised framework issued in April 2003 3 because, by
then, we had already completed our work.
To evaluate EOUSA*s IT investment management (ITIM), we used GAO*s ITIM
framework 4 and assessed the extent to which EOUSA had satisfied the
critical processes associated with stage 2 of the five- stage framework*
building the investment foundation. We focused on stage 2
processes because officials told us that they had only recently begun
defining and implementing the specific practices that are associated with
this stage. To conduct our assessment, we reviewed relevant
EOUSA and Justice policies, procedures, guidance, and documentation*
including the office*s investment management guide and associated
memorandums, project proposals, and budget
1 U. S. General Accounting Office, Information Technology: Enterprise
Architecture Use across the Federal Government Can Be Improved, GAO- 02- 6
(Washington, D. C.: February 2002).
2 Chief Information Officers Council, A Practical Guide to Federal
Enterprise Architecture,
Version 1. 0 (February 2001). 3 U. S. General Accounting Office, A
Framework for Assessing and Improving Enterprise Architecture Management,
Version 1.1, GAO- 03- 584G (Washington, D. C.: April 2003).
4 U. S. General Accounting Office, Information Technology Investment
Management: A Framework for Assessing and Improving Process Maturity
(Exposure Draft), GAO/ AIMD10.1.23 (Washington, D. C.: May 2000).
documents. We also interviewed the CIO and the senior official who is
responsible for implementing IT investment management. We then compared
this information with our maturity framework to determine the extent to
which the office was employing effective IT investment management
practices. To evaluate EOUSA*s information security management, we used
our
executive guide for information security management, 5 as well as Justice
policy and guidance and relevant EOUSA U. S. Attorney Procedures. 6 We
reviewed internal Justice and other reports identifying security
weaknesses at Justice and EOUSA and information on how these weaknesses
will be addressed. We also reviewed the certification and accreditation
package and the deployment schedule for the virtual private network 7 that
the office is currently deploying, because EOUSA and the USAOs rely on
this network to carry out its mission. We interviewed Justice officials
and EOUSA officials within the Office of
the CIO about the office*s security management. To evaluate EOUSA*s
system acquisition management, we used the Software Engineering
Institute*s Software Acquisition Capability Maturity Model, 8 focusing on
six of the seven key process areas that are defined for level 2 of the
model*s five- level maturity scale. 9 We focused on level 2 processes
because they represent the minimum level of
maturity needed to effectively manage system acquisition projects. We used
the office*s acquisition of the Enterprise Case Management System as a
case study because officials stated that it is representative of how 5 U.
S. General Accounting Office, Information Security Management: Learning
from Leading Organizations, GAO/ AIMD- 98- 68 (Washington, D. C.: May
1998). 6 See, for example, U. S. Department of Justice, Information
Technology Security (DOJ 2640. 2D (July 2001) and EOUSA, Access to
Sensitive But Unclassified IT Resources, USAP 3- 16.010.30.001( M) (March
2002).
7 A virtual private network uses a public or shared telecommunication
infrastructure to provide remote users with secure access to an
organization's network. 8 Carnegie Mellon University*s Software
Engineering Institute, Software Acquisition Capability Maturity Model,
Version 1.02, CMU/ SEI- 99- TR- 002 (April 1999). 9 The six key process
areas that we evaluated are software acquisition planning, solicitation,
requirements development and management, project management, contract
tracking and oversight, and evaluation. We did not include the seventh key
process area*
transition to support* in our evaluation because the system that we
assessed had not yet progressed to the point that this process area was
relevant.
they intend to acquire systems. In addition, this system will be critical
in providing fundamental support to the U. S. Attorneys as they work to
achieve mission goals. We reviewed key project documentation, such as the
concept of operations, project plan, and requirements traceability matrix,
and we interviewed system acquisition officials. We also reviewed the
Justice guidance used to manage the project. We then compared this
information to the Software Acquisition Capability
Maturity Model to determine the extent to which the office was employing
effective system acquisition management practices.
We performed our work at EOUSA headquarters in Washington, D. C., from
November 2002 to May 2003, in accordance with generally accepted
government auditing standards.
Assessment of ECMS Acquisition Practices against Level 2 of SEI*s Software
Acquisition
Appendi x II
Capability Maturity Model Table 7: Software Acquisition Planning Common
feature CMM key practice Satisfied? Comments
Commitment 1 The acquisition organization has a written No EOUSA does not
have a written policy for planning the policy for planning the software
acquisition. software acquisition.
Commitment 2 Responsibility for software acquisition Yes Responsibility
for software acquisition activities was activities is designated.
designated to the ECMS project manager.
Ability 1 A group that is responsible for planning the Yes A group
responsible for planning exists and includes the software acquisition
exists. project manager, administrative contract officer*s
technical representative, and assistant directors of Case Management
staff.
Ability 2 The acquisition organization provides Yes The acquisition
organization provided experienced experienced software acquisition
software acquisition management personnel to support management personnel
to support project
project software acquisition planning. software acquisition planning.
Ability 3 Adequate resources are provided for Yes According to EOUSA
officials, adequate resources were software acquisition planning
activities. provided for software acquisition planning activities.
Activity 1 Software acquisition planning personnel are Yes Software
acquisition planning personnel were involved involved in system
acquisition planning. in system acquisition planning.
Activity 2 The project*s software acquisition planning Yes The project*s
software acquisition planning was is accomplished in conjunction with
system
accomplished in conjunction with system acquisition acquisition planning.
planning. Activity 3 The software acquisition strategy for the
No There is no software acquisition strategy document. project is
developed and documented. Activity 4 Software acquisition planning
addresses the Yes Software acquisition planning addresses most critical
elements of the software acquisition
elements of the software acquisition process. process. Activity 5 The
project*s software acquisition planning Yes Software acquisition planning
information is included in is documented, and the planning
the project management plan, which has been updated documentation is
maintained over the life of once. the project. Activity 6 Life- cycle
support of the software is
Yes Certain life- cycle support provisions (user training, included in
software acquisition planning system growth) are documented in the project
documentation.
management plan. Activity 7 Life- cycle cost and schedule estimates for
Yes Life- cycle cost and schedule estimates for the initial the software
products and services being release of ECMS were prepared by the project
team acquired are prepared and independently and independently reviewed by
the administrative reviewed.
contract officer*s technical representative. Measurement 1 Measurements
(e. g., planned vs. completed
Yes Measurements (e. g., estimated vs. actual cost and works) are made and
used to determine the
schedule) were made by the project team and used to status of the software
acquisition planning determine the status of the software acquisition
activities and resultant products.
planning activities and resultant products.
(Continued From Previous Page)
Common feature CMM key practice Satisfied? Comments
Verification 1 Software acquisition planning activities are Yes The
project team reviews software acquisition planning reviewed by acquisition
organization
activities on a periodic basis. management on a periodic basis.
Verification 2 Software acquisition planning activities are Yes The
project manager reviews software acquisition reviewed by the project
manager on both a
planning activities on both a periodic and event- driven periodic and
event- driven basis.
basis. Source: Key practice data from SEI; analysis and comments from GAO.
Table 8: Solicitation Common feature CMM key practice Satisfied? Comments
Commitment 1 The acquisition organization has a written Yes The
acquisition organization used the Department of policy for the conduct of
the solicitation. Transportation*s Value Added Niche Information
Technology Services (VANITS) vehicle, which provides federal, state, and
local government clients with access to specialized technology services
and support.
Commitment 2 Responsibility for the software portion of the Yes
Responsibility for the software portion of the solicitation solicitation
was designated. was designated to a technical point of contact and an
administrative contract officer*s technical
representative. Commitment 3 A selection official was designated to be
Yes The technical point of contact and the administrative responsible for
the selection process and contract officer*s technical representative were
the decision.
responsible for the selection process and the decision. Ability 1 A group
that is responsible for coordinating
Yes A group consisting of assistant directors of the and conducting the
solicitation activities
information technology staff exists. With the technical exists. point of
contact as the chair, this group conducted an evaluation of vendors*
proposals.
Ability 2 Adequate resources were provided for the Yes Adequate resources
were provided for solicitation solicitation activities. activities. EOUSA
budgeted and paid a fee for using
services provided under the VANITS vehicle. Ability 3 Individuals
performing the solicitation Yes According to EOUSA officials, individuals
performing activities have experience or receive
the solicitation activities have formal training or training.
experience. Ability 4 The groups supporting the solicitation (e. g.,
No The supporting groups received an orientation, but this end user,
system engineering, and
did not cover solicitation procedures. application domain experts) receive
orientation on the solicitation*s objectives and procedures.
Activity 1 The project*s solicitation activities were No The project team
did not document its plans for performed in accordance with its plans.
solicitation activities.
Activity 2 Solicitation activities are conducted in a Yes The project team
followed standard procedures manner compliant with relevant laws,
required by the Governmentwide Acquisition Contracts policies, and
guidance.
(GWAC). Activity 3 The software and evaluation requirements
Yes The project team incorporated the software and are incorporated into
the solicitation evaluation requirements into the solicitation package
package and resulting contract. and resulting contract.
Activity 4 The project*s proposal evaluation activities Yes According to
EOUSA officials, the project*s proposal were performed in accordance with
its
evaluation activities were performed in accordance with plans. its plans.
Activity 5 Cost and schedule estimates for the Yes Cost and schedule
estimates for the software activity software activity were prepared. were
prepared by the project team.
(Continued From Previous Page)
Common feature CMM key practice Satisfied? Comments
Activity 6 The software cost and schedule estimates Yes Software cost and
schedule estimates were were independently reviewed for
independently reviewed for comprehensiveness and comprehensiveness and
realism.
realism by the administrative contracting officer*s representative.
Activity 7 The selection official uses proposal Yes The selection official
used proposal evaluation results evaluation results to support his or her
to support his decision. decision to select an offerer.
Activity 8 The project team and the offerer( s) review Yes The team
reviewed four proposals and asked the project*s software requirements and
contractors to provide presentations to ensure mutual plans during
negotiations to ensure mutual understanding. understanding.
Measurement 1 Measurements were made and used to Yes The VANITS program
office kept the project team determine the status of the solicitation
informed of the status of all activities. Measurements activities and
resultant products.
used to determine the status included the length of time taken for each
activity. Verification 1 The activities for solicitation were reviewed Yes
The activities for solicitation were reviewed bi- weekly by acquisition
organization management on by the designated selection official or
acquisition a periodic basis.
organization management. Verification 2 The activities for solicitation
were reviewed
Yes The activities for solicitation were reviewed by the by the project
manager or designated
project manager on both a periodic and an event selection official on both
a periodic and an
driven basis. event- driven basis. Source: Key practice data from SEI;
analysis and comments from GAO.
Table 9: Requirements Development and Management Common feature CMM key
practice Satisfied? Comments
Commitment 1 The acquisition organization has a written Yes The project
management plan includes guidelines for policy for developing and managing
defining and controlling technical and nontechnical software- related
requirements.
(software- related) requirements. Commitment 2 Responsibility for
requirements development
Yes The ECMS project team is responsible for and management is designated.
requirements development and management.
Ability 1 A group that is responsible for performing Yes A Joint
Application Development group is responsible requirements development and
for performing requirements development. The management activities exists.
contractor is responsible for performing requirements management.
Ability 2 Adequate resources are provided for Yes According to EOUSA
officials, adequate resources requirements development and
were provided for requirements development and management activities.
management activities. Ability 3 Individuals performing requirements
Yes According to EOUSA officials, individuals performing development and
management activities requirements development and management activities
have experience or receive training. have experience or received training.
Activity 1 The project team performs its activities in Yes EOUSA officials
reported that the project team accordance with its documented
performs its activities in accordance with its requirements development
and
documented requirements development and management plans.
management plans. Activity 2 The project team develops, baselines, and
Yes According to EOUSA officials, the project team maintains software-
related contractual develops, baselines, and maintains software- related
requirements.
contractual requirements. Activity 3 The project team appraises requests
for
Yes The project team reviews requests for changes to changes to system
requirements for their
system requirements for their impact on ECMS. impact on the software being
acquired.
Activity 4 The project team appraises all changes to Yes The project team
reviews all changes to the software the software- related contractual
related contractual requirements for their impact on requirements for
their impact on performance, architecture, supportability, system
performance, architecture, supportability, resource utilization, and
contract schedule and cost. system resource utilization, and contract
schedule and cost.
Activity 5 Bi- directional traceability between the Yes Bi- directional
traceability between the contractual contractual requirements and the
contractor requirements and the contractor*s team software work team*s
software work products and services products and services is maintained by
the project
is maintained throughout the effort. team. Activity 6 The end user and
other affected groups are
Yes EOUSA officials reported that the end user and other involved in the
development of all software affected groups are involved in the
development of all related contractual requirements and any
software- related contractual requirements and any subsequent change
activity.
subsequent change activity. Measurement 1 Measurements are made and used
to
Yes Measurements are made and used by the project team determine the
status of the requirements to determine the status of the requirements
development and management activities
development and management activities and resultant and resultant
products.
products.
(Continued From Previous Page)
Common feature CMM key practice Satisfied? Comments
Verification 1 Requirements development and Yes Requirements development
and management management activities are reviewed by activities are
reviewed by the project team (and the acquisition organization management
(and
contractor) on a periodic basis. the contractor) on a periodic basis.
Verification 2 Requirements development and Yes Requirements development
and management management activities are reviewed by the activities are
reviewed by the project manager on both project manager on both a periodic
and
a periodic and event- driven basis. event- driven basis. Source: Key
practice data from SEI; analysis and comments from GAO.
Table 10: Project Management Common feature CMM key practice Satisfied?
Comments
Commitment 1 The acquisition organization has a written Yes A policy memo
was issued requiring all information policy for executing the software
project. technology projects to follow a streamlined life- cycle
methodology. Commitment 2 Responsibility for project management is Yes
Responsibility for project management is designated designated. to the
ECMS project manager.
Ability 1 A team that is responsible for performing the Yes A team that is
responsible for performing the project*s project*s software acquisition
management
software acquisition management exists. It includes a exists. project
manager and case management staff.
Ability 2 Adequate resources for the project team are Yes According to
EOUSA officials, adequate resources for provided for the duration of the
software
the project team are provided for the duration of the acquisition project.
software acquisition project. Ability 3 When project trade- offs are
necessary, the
Yes When project trade- offs are necessary, the project project manager is
permitted to alter the
manager is permitted to alter the performance, cost, or performance, cost,
or schedule software schedule software acquisition baseline after
acquisition baseline. appropriate review. Ability 4 The project team has
experience or Yes EOUSA officials reported that the project team
receives training in project software members have experience or received
training in acquisition management activities.
project software acquisition management activities. Activity 1 The project
team performs its activities in
Yes EOUSA officials reported that the project team accordance with its
documented software
performs its activities in accordance with its project acquisition
management plans.
management plan. Activity 2 The roles, responsibilities, and authority for
Yes The roles, responsibilities, and authority for the project the project
functions are documented,
functions are documented in the ECMS project plan, maintained, and
communicated to affected
maintained, and communicated to affected groups. groups.
Activity 3 The project team*s commitments, and Yes The project team*s
commitments, and changes to changes to commitments, are
commitments, are communicated to affected groups communicated to affected
groups.
via an on- line discussion forum. Activity 4 The project team tracks the
risks associated
Yes Project- wide risks are documented in the risk with cost, schedule,
resources, and the management plan. Ancillary risks that affect project
technical aspects of the project.
execution, and plans for mitigating those risks, are documented in the
weekly reports. Activity 5 The project team tracks project issues,
Yes According to EOUSA officials, the project team tracks status,
execution, funding, and expenditures
project issues, status, execution, funding, and against project plans and
takes action.
expenditures against project plans and takes action. Activity 6 The
project team implements a corrective
Yes The project team identifies, records, and tracks issues action system
for the identification,
using Rational*s ClearQuest product. These data are recording, tracking,
and correction of then used to correct problems discovered during the
problems discovered during the software
software acquisition. The team is moving toward using acquisition.
Merant*s PVCS Dimensions software.
(Continued From Previous Page)
Common feature CMM key practice Satisfied? Comments
Activity 7 The project team keeps its plans current Yes The project team
updates its plans during the life of during the life of the project as re-
planning
the project as re- planning occurs, issues are resolved, occurs, issues
are resolved, requirements
requirements are changed, and new risks are are changed, and new risks are
discovered.
discovered. Measurement 1 Measurements are made and used to
Yes Measurements are made and used by the ECMS determine the status of
project management
project team to determine the status of project activities and the
resultant products.
management activities and the resultant products. Verification 1 Project
management activities are reviewed Yes Project management activities are
reviewed by by acquisition organization management on acquisition
organization management on a bi- weekly
a periodic basis. basis. Verification 2 Project management activities are
reviewed Yes Project management activities are reviewed by the
by the project manager on both a periodic project manager on both a
periodic and an and an event- driven basis. event- driven basis. Source:
Key practice data from SEI; analysis and comments from GAO.
Table 11: Contract Tracking and Oversight Common feature CMM key practice
Satisfied? Comments
Commitment 1 The acquisition organization has a written No The acquisition
organization does not have a written policy for the contract tracking and
oversight
policy for the contract tracking and oversight of the of the contracted
software effort.
contracted software effort. Commitment 2 Responsibility for contract
tracking and
Yes Responsibility is designated to the project manager oversight is
designated. and the administrative contracting officer*s
representative. Commitment 3 The project team includes contracting Yes
These specialists include the operations staff, the specialists in the
execution of the contract. contract officer*s technical representative,
and contracting and procurement staff.
Ability 1 A group that is responsible for managing Yes The project
management team is responsible for contract tracking and oversight
activities managing contract tracking and oversight activities. exists.
Ability 2 Adequate resources are provided for Yes According to EOUSA
officials, adequate resources are contract tracking and oversight
activities. provided for contract tracking and oversight activities.
Ability 3 Individuals performing contract tracking and Yes According to
EOUSA officials, individuals performing oversight activities have
experience or contract tracking and oversight activities have receive
training.
experience or receive training. Activity 1 The project team performs its
activities in
Yes The project team performs its activities in accordance accordance with
its documented contract with its documented contract tracking and
oversight tracking and oversight plans.
plans. Several reporting mechanisms are used to monitor and control the
contractor*s performance, including sticking to the project schedule and
reporting any problems that are encountered.
Activity 2 The project team reviews required Yes The project team reviews
required contractor software contractor software planning documents
planning documents, which provide a basis for which, when satisfactory,
are used to overseeing the contractor team*s software engineering oversee
the contractor team*s software
efforts. engineering effort.
Activity 3 The project team conducts periodic reviews Yes There are weekly
meetings and monthly written status and interchanges with the contractor
team. reports between the project team and the contractor.
Activity 4 The actual cost and schedule of the Yes Actual cost and
schedule of the contractor*s software contractor*s software engineering
effort are engineering effort are compared to the planned costs compared
to planned schedules and
and schedule, and issues are identified. Issues so far budgets, and issues
are identified.
are primarily related to the contractor*s staff getting the security
clearances needed to do the work.
Activity 5 The size, critical computer resources, and Yes The contractor
provides information about the size, technical activities associated with
the
critical computer resources, and technical activities to contractor team*s
work products are tracked, the ECMS project team for tracking purposes and
and issues are identified.
issue identification.
(Continued From Previous Page)
Common feature CMM key practice Satisfied? Comments
Activity 6 The project team reviews and tracks the Yes The project team
reviews and tracks the development development of the software engineering
of the software engineering environment. environment required to provide
life- cycle support for the acquired software, and issues are identified.
Activity 7 Any issues found by the project team during Yes According to
EOUSA officials, any issues found by the contract tracking and oversight
are recorded project team during contract tracking and oversight in the
appropriate corrective action system,
are recorded in the appropriate corrective action action is taken, and the
issue is tracked to system, action is taken, and the issue is tracked to
closure.
closure. Activity 8 The project team ensures that changes to
Yes The project team ensures that changes to the the software- related
contractual
software- related contractual requirements are requirements are
coordinated with all coordinated with all affected groups and individuals,
affected groups and individuals, such as the
including the administrative contracting officer*s contracting official,
contractor, and end user.
representative and end users. Measurement 1 Measurements are made and used
to Yes Measurements are made and used by the determine the status of the
contract tracking administrative contracting officer*s representative to
and oversight activities and resultant
determine the status of the contract tracking and products.
oversight activities and resultant products. Verification 1 Contract
tracking and oversight activities are Yes The administrative contracting
officer*s representative reviewed by acquisition organization reviews
contract tracking and oversight activities on a
management on a periodic basis. periodic basis. Verification 2 Contract
tracking and oversight activities are Yes The project manager, on both a
periodic and an event reviewed by the project manager on both a
driven basis, reviews contract tracking and oversight periodic and event-
driven basis.
activities. Source: Key practice data from SEI; analysis and comments from
GAO.
Table 12: Evaluation Common feature CMM key practice Satisfied? Comments
Commitment 1 The acquisition organization has a written Yes The
acquisition organization has defined guidelines for policy for managing
the evaluation of the testing and certifying ECMS. acquired software
products and services.
Commitment 2 Responsibility for evaluation activities is Yes
Responsibility for evaluation activities is designated to designated. the
ECMS project team.
Ability 1 A group that is responsible for planning, Yes The ECMS project
team is responsible for planning, managing, and performing evaluation
managing, and performing evaluation activities for the activities for the
project exists.
project. Ability 2 Adequate resources are provided for
Yes According to EOUSA officials, the evaluation activities evaluation
activities. have been budgeted. Ability 3 Individuals performing
evaluation activities Yes Individuals performing evaluation activities
have have experience or receive training. experience or receive training.
The contractors were required to submit resumes along with their
proposals.
Ability 4 Members of the project team and groups No Because of ECMS*s
stage in the life cycle (design supporting the software acquisition
receive phase), this key practice has not yet been addressed. orientation
on the objectives of the evaluation approach.
Activity 1 The project team performs its activities in No Because of
ECMS*s stage in the life cycle (design accordance with its documented
evaluation
phase), this key practice has not yet been addressed plans.
Activity 2 The project*s evaluation requirements are No Because of ECMS*s
stage in the life cycle (design developed in conjunction with the
phase), this key practice has not yet been addressed. development of the
system or software technical requirements.
Activity 3 The project*s evaluation activities are No Because of ECMS*s
stage in the life cycle (design planned to minimize duplication and take
phase), this key practice has not yet been addressed. advantage of all
evaluation results, where appropriate.
Activity 4 The project team appraises the contractor Yes The project team
assesses the contractor team*s team*s performance over the full period of
performance continuously. the contract for compliance with requirements.
Activity 5 Planned evaluations are performed on the No Because of ECMS*s
stage in the life cycle (design evolving software products and services
phase), this key practice has not yet been addressed. prior to acceptance
and operational use.
Activity 6 Results of the evaluations are analyzed and No Because of
ECMS*s stage in the life cycle (design compared with the contract*s
requirements
phase), this key practice has not yet been addressed. to establish an
objective basis to support the decision to accept the products and
services or to take further action.
Measurement 1 Measurements are made and used to No Because of ECMS*s stage
in the life cycle (design determine the status of the evaluation
phase), this key practice has not yet been addressed. activities and
resultant products.
(Continued From Previous Page)
Common feature CMM key practice Satisfied? Comments
Verification 1 Evaluation activities are reviewed by No Because of ECMS*s
stage in the life cycle (design acquisition organization management on a
phase), this key practice has not yet been addressed. periodic basis.
Verification 2 Evaluation activities are reviewed by the No Because of
ECMS*s stage in the life cycle (design project manager on both a periodic
and an phase), this key practice has not yet been addressed. event- driven
basis. Source: Key practice data from SEI; analysis and comments from GAO.
Appendi x III
Comments from the Department of Justice Note: GAO comments supplementing
those in the report text appear at the end of this appendix. See comment
1. See comment 2.
See comment 3.
See comment 4. See comment 5. See comment 6. See comment 7. See comment 8.
See comment 9. See comment 10. See comment 11.
See comment 12. See comment 13. See comment 14.
See comment 15. See comment 16.
See comment 17.
See comment 18. See comment 19.
See comment 20. See comment 21.
See comment 22. See comment 23.
See comment 24. See comment 25.
See comment 26. See comment 27.
The following are GAO*s comments on the Department of Justice*s letter
dated June 16, 2003.
GAO Comments 1. We disagree. Our position that institutionalization has
not been a priority is based on two facts that EOUSA did not dispute: (1)
plans for
addressing the weaknesses cited in our report do not exist and (2)
limitations in resources to address these weaknesses were cited by EOUSA
officials as the reason why the weaknesses exist. If each of these areas
were an agency priority, then plans would be in place to address the
weaknesses, and resources to execute the plans would be committed.
2. We do not question EOUSA*s statement that it has made *tremendous
progress.* Our work focused on determining the extent to which EOUSA
currently satisfies key practices in the four IT management disciplines.
It did not include developing a baseline from which to measure progress.
To EOUSA*s credit, our review showed that the office has satisfied many
key practices in each discipline, and we have noted this in our report.
3. We agree and include both of these facts in our report. 4. We disagree.
EOUSA*s comments did not include any information to
refute our conclusion. Given that it did not have a plan for fully
implementing best practices for each discipline, and had not allocated
adequate resources to support such a plan, we have not modified our
conclusion.
5. We do not question these statements about the position of EOUSA and the
USAOs relative to other Justice components. Such a comparison was not part
of the scope of our work.
6. We disagree. EOUSA has not gained this maturity level. Rather,
according to EOUSA, the contractor that maintains its LIONS application is
certified as a level 3 software developer. In contrast, our work focused
on EOUSA*s capabilities as a software acquirer, and thus addresses a
different organization, discipline, and maturity model.
7. See comment 1.
8. We do not question this statement because the position of EOUSA and the
USAOs relative to other Justice components or other law enforcement
entities was not part of the scope of our work.
9. As noted in our report, EOUSA satisfied about 80 percent of the
elements of just stage 2 of the EA management framework. It has satisfied
about 60 percent of the elements (12 out of 19) of the entire framework.
10. We have modified the report to reflect this comment. 11. We agree.
However, according to GAO*s IT Investment Management
Framework, to satisfy the proposal selection critical process, EOUSA would
need to demonstrate the use of the criteria it has defined. Because it has
not yet done so, it is not satisfying the critical process and thus has
met two out of five elements of stage 2 of the framework.
12. We disagree. Our assessment is based on EOUSA*s satisfaction of key
practices laid out in our executive guide for information security
management. 1 This assessment showed that EOUSA has not fully satisfied
any of these key practices. For example, EOUSA does not (1) have a central
security focal point with appropriate resources, (2) adequately promote
user awareness, and (3) regularly monitor the effectiveness of security
controls. Until EOUSA addresses these and other security weaknesses we
identify in our report, it will not have implemented effective security
practices.
13. See comment 8. 14. We do not question this statement because
determining whether the
data of the United States Attorneys have never been compromised and are
monitored 24 hours a day, 7 days a week was not within the scope of our
work and EOUSA did not provide any evidence supporting its assertions.
15. See comment 1. Additionally, our finding is that the
institutionalization of information security management has not been an
agency priority.
1 U. S. General Accounting Office, Information Security Management:
Learning from Leading Organizations, GAO/ AIMD- 98- 68 (Washington, D. C.:
May 1998).
16. We do not question these security initiatives. Additionally, we
emphasize that our message is not that EOUSA has not taken steps to
improve its information security posture, but rather that the office*s
information security management efforts, including ongoing and completed
improvement steps, are weak in a number of areas relative to information
security management best practices.
17. We agree, but would add that our recommendation could be addressed by
actively monitoring activity at the routers, firewalls, and wide area
network devices, which we understand are remotely managed by EOUSA. To
avoid any potential confusion on this point, we have clarified our
recommendation. Implementing an intrusion detection system to monitor
activity at the routers, firewalls, and other network devices would enable
EOUSA to detect hostile attempts to manage those devices.
18. We do not question EOUSA*s statement that it has been working to
resolve vulnerabilities identified during a security audit conducted by
the Justice Inspector General. The scope of the Inspector General*s audit,
however, was narrower than ours in that it focused on EOUSA*s local area
network environment.
19. We agree that given EOUSA*s recent progress in deploying the
replacement network its exposure to risk is currently limited. We have
modified the security risk assessment section of the report and the
associated recommendation to reflect this change in circumstances.
20. We agree and thus do not conclude that EOUSA*s risk assessment program
is inadequate. Rather, based on the fact that a risk assessment was not
performed on the network that EOUSA has operated since 1996 and, until
recently, relied exclusively on, we conclude that EOUSA has not always
performed risk assessments. Additionally, to recognize the recent change
in circumstances we have modified our recommendation concerning risk
assessments.
21. We do not question these statements. We support the use of automated
tools to review audit logs, particularly because these logs were not being
reviewed, and EOUSA attributed this to a lack of resources. We also
support EOUSA*s plan to conduct regular tests to determine
compliance with policies and procedures. Both of these planned actions are
consistent with our recommendations.
22. We do not question this statement. However, as noted in our report,
the office did not have a plan to address the issues that are discussed in
our report.
23. We do not question these statements because our review did not address
contingency plans for all certified and accredited systems. As stated in
the report, while a contingency plan was developed for the replacement
network, it was not prepared in accordance with federal guidelines. For
example, the plan did not specify procedures for notifying recovery
personnel. To clarify our position, we have added examples to the report
of this plan*s noncompliance with federal guidelines.
24. We support EOUSA*s stated commitment to establish a centralized
security training program. Establishing such a program is consistent with
our recommendations.
25. We have modified the report to reflect that the Enterprise Case
Management System is the first acquisition to follow the Justice lifecycle
methodology from its inception.
26. See comment 6. 27. We have modified the report to reflect that EOUSA*s
acquisitions are processed through the department and must comply with all
departmental policies and procedures.
Appendi x IV
GAO Contact and Staff Acknowledgments GAO Contact Lester P. Diamond, (202)
512- 7957 Acknowledgments In addition to the individual named above,
Nabajyoti Barkakati, Jamey
Collins, Joanne Fiorino, Anh Q. Le, Sabine R. Paul, and William F.
Wadsworth made key contributions to this report.
(310257)
a
GAO United States General Accounting Office
To varying degrees, EOUSA has partially defined and implemented certain IT
management disciplines that are critical to successfully achieving the
Justice Department*s strategic goal of improving the integrity, security,
and
efficiency of its IT systems. However, it has yet to institutionalize any
of these disciplines, meaning that it has not defined existing policies
and procedures in accordance with relevant guidance, and it has yet to
fully implement what it has defined. In particular, while EOUSA has
developed an enterprise architecture* a blueprint for guiding operational
and
technological change* the architecture was not developed in accordance
with certain best practices. In addition, while the office has implemented
certain process controls for selecting, controlling, and evaluating its IT
investments, it has not yet implemented others that are necessary in order
to develop an effective foundation for investment management. Further, it
has not implemented important management practices that are associated
with an effective security program. In contrast, it has defined* and is
implementing on a major system that we reviewed* most, but not all, of the
management practices associated with effective systems acquisition.
Institutionalization of these IT management disciplines has not been an
agency priority and is not being guided by plans of action or sufficient
resources. Until each discipline is given the priority it deserves, EOUSA
will not have the IT management capabilities it needs to effectively
achieve the department*s strategic goal of improving the integrity,
security, and
efficiency of its IT systems.
EOUSA*s Institutionalization of Four Key IT Management Disciplines
Management discipline
Fully Institutionalized?
Comment
Enterprise architecture management No Has an approved enterprise
architecture but, for
example, has yet to develop a policy for maintaining it. IT investment
management
No Has several basic elements of proper IT investment management but has
not yet, for example, used its defined investment selection process.
Information security management
No Is not fully satisfying any of the tenets of effective security, such
as monitoring the effectiveness of security controls and promoting
security awareness. System acquisition management
No Is successfully employing most of the practices associated with
effective software acquisition management on one key project but does not
have, for example, a policy for software acquisition planning.
Source: GAO. The Executive Office for United States Attorneys (EOUSA) of
the
Department of Justice is responsible for managing information technology
(IT) resources for the United States
Attorneys* Offices. GAO was asked to determine the extent to which EOUSA
has institutionalized key IT management capabilities that are critical to
achieving Justice*s strategic goal of improving the integrity, security,
and efficiency of its IT systems. To strengthen EOUSA*s IT
management capacity and to increase its chances of effectively leveraging
IT to improve its mission performance, GAO recommends that the Attorney
General direct the Director of
EOUSA to (1) designate institutionalization of each of the IT management
disciplines as priorities and (2) develop and implement action plans in
each of the four IT disciplines to address
the weaknesses that are identified in this report. EOUSA agreed with the
majority of GAO*s findings and recommendations, and stated that it will
address most of the
recommendations. It also stated that it has made notable progress in
institutionalizing the IT management disciplines, particularly information
security, and that each is currently an office priority. www. gao. gov/
cgi- bin/ getrpt? GAO- 03- 751. To view the full product, including the
scope
and methodology, click on the link above. For more information, contact
Randolph C. Hite at 202- 512- 3439 or hiter@ gao. gov. Highlights of GAO-
03- 751, a report to
congressional requesters
July 2003
INFORMATION TECHNOLOGY
Executive Office for U. S. Attorneys Needs to Institutionalize Key IT
Management Disciplines
Page i GAO- 03- 751 EOUSA IT Management
Contents
Page ii GAO- 03- 751 EOUSA IT Management
Page 1 GAO- 03- 751 EOUSA IT Management United States General Accounting
Office Washington, D. C. 20548
Page 1 GAO- 03- 751 EOUSA IT Management
A
Page 2 GAO- 03- 751 EOUSA IT Management
Page 3 GAO- 03- 751 EOUSA IT Management
Page 4 GAO- 03- 751 EOUSA IT Management
Page 5 GAO- 03- 751 EOUSA IT Management
Page 6 GAO- 03- 751 EOUSA IT Management
Page 7 GAO- 03- 751 EOUSA IT Management
Page 8 GAO- 03- 751 EOUSA IT Management
Page 9 GAO- 03- 751 EOUSA IT Management
Page 10 GAO- 03- 751 EOUSA IT Management
Page 11 GAO- 03- 751 EOUSA IT Management
Page 12 GAO- 03- 751 EOUSA IT Management
Page 13 GAO- 03- 751 EOUSA IT Management
Page 14 GAO- 03- 751 EOUSA IT Management
Page 15 GAO- 03- 751 EOUSA IT Management
Page 16 GAO- 03- 751 EOUSA IT Management
Page 17 GAO- 03- 751 EOUSA IT Management
Page 18 GAO- 03- 751 EOUSA IT Management
Page 19 GAO- 03- 751 EOUSA IT Management
Page 20 GAO- 03- 751 EOUSA IT Management
Page 21 GAO- 03- 751 EOUSA IT Management
Page 22 GAO- 03- 751 EOUSA IT Management
Page 23 GAO- 03- 751 EOUSA IT Management
Page 24 GAO- 03- 751 EOUSA IT Management
Page 25 GAO- 03- 751 EOUSA IT Management
Page 26 GAO- 03- 751 EOUSA IT Management
Page 27 GAO- 03- 751 EOUSA IT Management
Page 28 GAO- 03- 751 EOUSA IT Management
Page 29 GAO- 03- 751 EOUSA IT Management
Appendix I
Appendix I Objective, Scope, and Methodology
Page 30 GAO- 03- 751 EOUSA IT Management
Appendix I Objective, Scope, and Methodology
Page 31 GAO- 03- 751 EOUSA IT Management
Page 32 GAO- 03- 751 EOUSA IT Management
Appendix II
Appendix II Assessment of ECMS Acquisition Practices against Level 2 of
SEI*s Software Acquisition Capability Maturity Model
Page 33 GAO- 03- 751 EOUSA IT Management
Appendix II Assessment of ECMS Acquisition Practices against Level 2 of
SEI*s Software Acquisition Capability Maturity Model
Page 34 GAO- 03- 751 EOUSA IT Management
Appendix II Assessment of ECMS Acquisition Practices against Level 2 of
SEI*s Software Acquisition Capability Maturity Model
Page 35 GAO- 03- 751 EOUSA IT Management
Appendix II Assessment of ECMS Acquisition Practices against Level 2 of
SEI*s Software Acquisition Capability Maturity Model
Page 36 GAO- 03- 751 EOUSA IT Management
Appendix II Assessment of ECMS Acquisition Practices against Level 2 of
SEI*s Software Acquisition Capability Maturity Model
Page 37 GAO- 03- 751 EOUSA IT Management
Appendix II Assessment of ECMS Acquisition Practices against Level 2 of
SEI*s Software Acquisition Capability Maturity Model
Page 38 GAO- 03- 751 EOUSA IT Management
Appendix II Assessment of ECMS Acquisition Practices against Level 2 of
SEI*s Software Acquisition Capability Maturity Model
Page 39 GAO- 03- 751 EOUSA IT Management
Appendix II Assessment of ECMS Acquisition Practices against Level 2 of
SEI*s Software Acquisition Capability Maturity Model
Page 40 GAO- 03- 751 EOUSA IT Management
Appendix II Assessment of ECMS Acquisition Practices against Level 2 of
SEI*s Software Acquisition Capability Maturity Model
Page 41 GAO- 03- 751 EOUSA IT Management
Appendix II Assessment of ECMS Acquisition Practices against Level 2 of
SEI*s Software Acquisition Capability Maturity Model
Page 42 GAO- 03- 751 EOUSA IT Management
Appendix II Assessment of ECMS Acquisition Practices against Level 2 of
SEI*s Software Acquisition Capability Maturity Model
Page 43 GAO- 03- 751 EOUSA IT Management
Page 44 GAO- 03- 751 EOUSA IT Management
Appendix III
Appendix III Comments from the Department of Justice
Page 45 GAO- 03- 751 EOUSA IT Management
Appendix III Comments from the Department of Justice
Page 46 GAO- 03- 751 EOUSA IT Management
Appendix III Comments from the Department of Justice
Page 47 GAO- 03- 751 EOUSA IT Management
Appendix III Comments from the Department of Justice
Page 48 GAO- 03- 751 EOUSA IT Management
Appendix III Comments from the Department of Justice
Page 49 GAO- 03- 751 EOUSA IT Management
Appendix III Comments from the Department of Justice
Page 50 GAO- 03- 751 EOUSA IT Management
Appendix III Comments from the Department of Justice
Page 51 GAO- 03- 751 EOUSA IT Management
Appendix III Comments from the Department of Justice
Page 52 GAO- 03- 751 EOUSA IT Management
Appendix III Comments from the Department of Justice
Page 53 GAO- 03- 751 EOUSA IT Management
Page 54 GAO- 03- 751 EOUSA IT Management
Appendix IV
GAO*s Mission The General Accounting Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities
and to help improve the performance and accountability of the federal
government for the American people. GAO examines the use of public funds;
evaluates federal programs and policies; and provides analyses,
recommendations, and other assistance to help Congress make informed
oversight, policy, and funding decisions. GAO*s commitment to good
government is reflected in its core values of accountability, integrity,
and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO*s Web site (www. gao. gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as *Today*s Reports,* on its
Web site daily. The list contains links to the full- text document files.
To have GAO e- mail this
list to you every afternoon, go to www. gao. gov and select *Subscribe to
e- mail alerts* under the *Order GAO Products* heading. Order by Mail or
Phone The first copy of each printed report is free. Additional copies are
$2 each. A check
or money order should be made out to the Superintendent of Documents. GAO
also accepts VISA and Mastercard. Orders for 100 or more copies mailed to
a single address are discounted 25 percent. Orders should be sent to:
U. S. General Accounting Office 441 G Street NW, Room LM Washington, D. C.
20548 To order by Phone: Voice: (202) 512- 6000 TDD: (202) 512- 2537 Fax:
(202) 512- 6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm E- mail:
fraudnet@ gao. gov Automated answering system: (800) 424- 5454 or (202)
512- 7470
Public Affairs Jeff Nelligan, Managing Director, NelliganJ@ gao. gov (202)
512- 4800 U. S. General Accounting Office, 441 G Street NW, Room 7149
Washington, D. C. 20548
United States General Accounting Office Washington, D. C. 20548- 0001
Official Business Penalty for Private Use $300 Address Service Requested
Presorted Standard Postage & Fees Paid
GAO Permit No. GI00
*** End of document. ***