Customs Service Modernization: Management Improvements Needed on
High-Risk Automated Commercial Environment Project (13-MAY-02,
GAO-02-545).
The U.S. Customs Service requested $206.9 million for its
Automated Commercial Environment (ACE) project in February 2002.
ACE, Customs' new import processing system is the first project
under the Customs Modernization Program. Customs' second
expenditure plan satisfies the conditions specified in the
appropriations act. The plan provides for (1) meeting the Office
of Management and Budget's capital planning and investment
control review requirements; (2) complying with Customs'
enterprise architecture; and (3) complying with federal
acquisition rules, requirements, guidelines, and systems
acquisition management practices. ACE will fundamentally change
Customs' and many other organizations' business processes by
introducing new system capabilities. Further, ACE will be
available around-the-clock to support important commercial and
enforcement systems. Customs did not meet key commitments made in
its first ACE expenditure plan because of underestimating funding
requirements. Actual requirements were 90 percent higher than
estimated. This history casts uncertainty on Customs' ability to
reliably estimate costs and meet future commitments. Further,
Customs lacks management controls in four areas: enterprise
architecture, human capital, software acquisition management, and
cost estimation. Since Customs has compressed its ACE acquisition
plans from five to four years, the degree of overlap of program
increments has increased. This may increases the risk that ACE
capabilities will not be delivered on time and within budget.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-02-545
ACCNO: A03262
TITLE: Customs Service Modernization: Management Improvements
Needed on High-Risk Automated Commercial Environment Project
DATE: 05/13/2002
SUBJECT: Appropriated funds
Appropriations
Emergency preparedness
Funds management
Internal controls
Program management
Strategic planning
National preparedness
Customs Service Automated Commercial
Environment System
Customs Service International Trade Data
System
Customs Service Modernization Program
SEI Software Acquisition Capability
Maturity Model
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-02-545
A
Report to Congressional Committees
May 2002 CUSTOMS SERVICE MODERNIZATION Management Improvements Needed on
High- Risk Automated Commercial Environment Project
GAO- 02- 545
Lett er
May 13, 2002 The Honorable Byron L. Dorgan Chairman The Honorable Ben
Nighthorse Campbell Ranking Minority Member Subcommittee on Treasury and
General Government Committee on Appropriations United States Senate
The Honorable Ernest J. Istook, Jr. Chairman The Honorable Steny H. Hoyer
Ranking Minority Member Subcommittee on Treasury, Postal Service,
and General Government Committee on Appropriations House of Representatives
Pursuant to the U. S. Customs Service?s fiscal year 2002 appropriation, 1
Customs submitted to Congress in February 2002 its second expenditure plan,
requesting $206.9 million for its Automated Commercial Environment (ACE)
project. ACE is intended to be Customs? new import processing system and the
first project under the Customs Modernization Program. As required by the
act, we reviewed the expenditure plan. Our objectives were to (1) determine
whether the second plan satisfied the legislative
conditions specified in the act, (2) determine whether the plan is
consistent with our open ACE recommendations, and (3) provide observations
about the second plan and Customs? management of ACE.
On March 8 and 11, 2002, we briefed your offices on the results of this
review. This report officially transmits to you the results of our work and
the recommendations we made to the commissioner of Customs. The full
briefing, including our scope and methodology, is reprinted as appendix I.
In brief, we determined that Customs? expenditure plan satisfies the
legislative conditions specified in the appropriations act. The plan
provides for (1) meeting the Office of Management and Budget?s (OMB) capital
planning and investment control review requirements; (2) complying with 1
Public Law 107- 67, dated Nov. 12, 2001.
Customs? enterprise architecture; 2 and (3) complying with federal
acquisition rules, requirements, guidelines, and systems acquisition
management practices. Further, it was reviewed and approved by Customs?
Investment Review Board, 3 the Department of the Treasury, and OMB.
Regarding the second objective, activities described in the plan are
consistent with our open recommendations for Customs to (1) justify and make
investment decisions incrementally; (2) strengthen ACE software acquisition
management; (3) immediately transfer responsibility and accountability for
the International Trade Data System 4 (ITDS) pilot to the ACE modernization
program manager; (4) include any plans for further investing in the ITDS
pilot, including cost, benefit, and risk justification, in the next ACE
expenditure plan; and (5) clarify the roles and responsibilities of the ACE
modernization independent verification and validation 5 (IV& V)
agent to ensure that independence is not compromised. Finally, we made the
following observations:
ACE is technically and managerially complex and challenging. Planned ACE
functional and performance parameters are demanding, and the system is
estimated to cost at least $1.5 billion just to put in place, not including
operation and maintenance. Also, ACE aims to fundamentally
change Customs? and many other organizations? business processes through the
introduction of new system capabilities. Further, the system is to be
available around- the- clock to provide critical information and support
important commercial and enforcement systems.
Customs did not meet key commitments made in its first ACE expenditure
plan because of a significant underestimation of funding requirements
(actual requirements were about 90 percent higher than 2 An enterprise
architecture is an institutional blueprint for guiding and constraining
investments in business process change and systems.
3 Customs? Investment Review Board makes information technology funding
decisions on the basis of comparisons and trade- offs among competing
project proposals. 4 ITDS is a component of the ACE project that is to
support governmentwide collection, use, and dissemination of trade data. 5
The purpose of IV& V is to provide an independent review of system processes
and products. The use of IV& V is a recognized best practice for large and
complex systems development and acquisition projects. To be effective, IV& V
should be performed by an entity that is independent of the processes and
products that are being reviewed.
estimated). This history casts uncertainty on Customs? ability to reliably
estimate costs and meet future commitments.
Despite progress, Customs does not yet have important ACE management
controls in place in the following four areas:
1. Enterprise architecture: Customs? enterprise architecture has not yet
been updated and extended to support ACE engineering tasks; consequently,
Customs risks having an enterprise architecture that is not sufficiently
complete to support ACE design activities. As a result, near- term system
design and development decisions could later
necessitate system rework to align ACE with the agency?s operating vision,
embodied in the updated enterprise architecture. If such rework is
necessary, promised system capabilities are unlikely to be delivered
on time and within budget. 2. Human capital: ACE is being managed by the
Customs Modernization Office (CMO). However, this office does not have the
people in place to
perform critical system acquisition functions. Moreover, the CMO does not
have an effective strategy for meeting its human capital needs. This also
increases the risk that promised system capabilities will not be delivered
on time and within budget. 3. Software acquisition management: Customs has
yet to establish
software acquisition process controls that are recognized as best management
practices. 6 In particular, Customs has not begun to establish process
controls for determining whether acquired software products and services
satisfy contract requirements before acceptance, nor to establish related
controls for effective and efficient transfer of acquired software products
to the support organization responsible for maintenance. These control
weaknesses further reduce the level of assurance that ACE capabilities will
be delivered on time and within budget.
6 These best practices are described in the Software Acquisition Capability
Maturity Model (SA- CMM), developed by the Software Engineering Institute
(SEI). In this model, SEI has provided criteria for characterizing an
organization?s software development and acquisition processes according to
five levels of maturity, with level 2 providing the minimum level of
acceptable effectiveness. The Capability Maturity Model is a service mark of
Carnegie Mellon University, and CMM is registered in the U. S. Patent and
Trademark Office.
4. Cost estimation: Customs has not validated its ACE expenditure plan cost
estimates, and its estimates of management reserve costs are not justified.
As a result, its plan does not provide Congress with credible cost
information needed for overseeing the project.
Customs has compressed its ACE acquisition plans from 5 to 4 years. This
compression increases the degree of overlap of program increments, which in
turn further increases the risk that ACE capabilities will not be delivered
on time and within budget.
Recommendations for To improve Customs? ACE modernization management, we
recommend
Executive Action that the Customs Service commissioner direct the chief
information officer,
as the designated modernization executive, to take the following actions:
Before building each ACE release (i. e., beginning detailed design and
development), certify to Customs? House and Senate appropriations
subcommittees that the enterprise architecture has been sufficiently
extended to provide the requisite enterprise design content and has been
updated to ensure consistency and integration across business areas.
Immediately develop and implement a CMO human capital management strategy
that provides both near- and long- term solutions to the CMO?s human capital
capacity limitations, including defining the office?s skill and capability
needs in terms that will allow Customs to attract qualified individuals and
that will provide sufficient rewards and training, linked to performance, to
promote their retention.
Develop and implement process controls for the SEI SA- CMM level 2 key
process areas and the level 3 acquisition risk management key process area.
By September 30, 2002, conduct and report to Customs? House and Senate
appropriations subcommittees on the results of an internal assessment of the
CMO?s maturity against the SEI SA- CMM level 2 key
process areas and the level 3 acquisition risk management key process area.
Develop and implement a rigorous and analytically verifiable cost
estimating program that embodies the tenets of effective estimating as
defined in SEI?s institutional and project- specific estimating models.
Limit future expenditure plan requests for management reserve funds to 10
percent of the total funds requested for the program or adequately justify
any management reserve requests in excess of 10 percent.
Address the risks associated with the accelerated ACE acquisition
strategy, including (1) immediately analyzing the risks associated with the
degree of design, development, and testing concurrency across ACE increments
that is inherent in Customs? 4- year, schedule- driven
acquisition strategy; (2) reconsidering the merits of this accelerated
strategy; and (3) within 90 days of the date of this briefing, reporting to
Customs? House and Senate appropriations subcommittees on the agency?s
strategy for going forward and its plans for mitigating the inherent risks
associated with this strategy.
Agency Comments In written comments on a draft of this report, the director,
Office of Planning, U. S. Customs Service, concurred with our
recommendations and
described specific actions that are being taken to respond to each. The
director?s comments are reprinted in appendix II. We are sending copies of
this report to the chairmen and ranking minority members of other Senate and
House committees and subcommittees that have authorization and oversight
responsibilities for the Customs Service.
We are also sending copies to the secretary of the treasury, the
commissioner of the Customs Service, and the director of OMB.
Should you or your staff have any questions on matters discussed in this
report, please contact me at (202) 512- 3439. I can also be reached by e-
mail at HiteR@ gao. gov. Key contributors to this report were Mark Bird,
Barbara Collier, Tamra Goldstein, Randolph Tekeley, Scott Pettis, and Aaron
Thorne.
Randolph C. Hite Director, Information Technology Architecture and Systems
Issues
Appendi xes Customs? Second Automated Commercial
Appendi x I
Environment (ACE) Expenditure Plan Customs? Second Automated Commercial
Environment (ACE) Expenditure Plan
Briefing for the staffs of the Subcommittee on Treasury and General
Government
Senate Committee on Appropriations and Subcommittee on Treasury, Postal
Service, and General Government
House Committee on Appropriations March 8 and 11, 2002
Page 1 of 60
Briefing Overview
Introduction
Objectives, Scope, and Methodology
Results in Brief
Background
Results
Conclusions
Recommendations for Executive Action
Agency Comments Page 2 of 60
Introduction
The U. S. Customs Service is acquiring a new trade processing system known
as the Automated Commercial Environment (ACE)/ International Trade Data
System (ITDS). 1 This project is intended to fundamentally transform not
only Customs? internal import processing operations, but also the manner in
which it and numerous other federal agencies interact with each other and
with the trade community. According to Customs, the project will, for
example,
enable more efficient import processing;
minimize the gap between revenue that should be collected and revenue that
is actually collected;
increase the flow of legitimate commerce; and
provide the trade community a single interface with 104 federal agencies
that have trade- related missions.
1 Hereafter referred to as ACE. Page 3 of 60
Introduction (cont.)
Customs? FY 2001 and FY 2002 appropriations acts 2 provided $130 million and
$300 million, respectively, for ACE. The acts specified that these funds not
be obligated until Customs submits to the Congress for approval ACE
expenditure plans that
meet the Office of Management and Budget?s (OMB) capital planning and
investment control review requirements;
comply with the Customs Service?s enterprise architecture;
comply with federal acquisition rules, requirements, guidelines, and systems
acquisition management practices;
are reviewed and approved by the Customs Investment Review Board, the
Department of the Treasury, and OMB; and
are reviewed by GAO. 2 Public Laws 106- 554 and 107- 67, respectively.
Page 4 of 60
Introduction (cont.)
On February 1, 2002, Customs submitted its second ACE expenditure plan to
its House and Senate appropriations subcommittees, seeking release of $206.9
million from the ACE appropriations.
Page 5 of 60
Objectives, Scope and Methodology Objectives
As agreed, our objectives were to
determine whether the second ACE expenditure plan satisfies the legislative
conditions,
determine whether the plan is consistent with our open ACE recommendations,
and
provide observations about the plan and Customs management of ACE. Page 6 of
60
Objectives, Scope and Methodology (cont.) Scope and Methodology
To accomplish our objectives, we
analyzed the plan against the legislative conditions, to determine if the
conditions were met;
analyzed the plan to determine consistency of actions taken by Customs in
response to our open recommendations;
interviewed ACE and ITDS program management, support contractor, and
modernization contractor officials to understand the scope and content of
plans, and to clarify information in supporting documentation;
analyzed ACE supporting documentation to ensure consistency with information
in the expenditure plan and to determine progress made in implementing
program management capabilities (e. g., acquisition management, enterprise
architecture, human capital, and cost and schedule estimating);
Page 7 of 60
Objectives, Scope and Methodology (cont.)
determined progress made against commitments in first expenditure plan;
coordinated with the Treasury inspector general to avoid duplication of
effort in reviewing ACE;
did not evaluate the ACE Investment Strategy and Cost Benefit Analysis,
which we received on February 13, 2002; and
as agreed, did not independently validate Customs? provided information,
including cost estimates and project content and status information. We
performed our work from December 2001 through March 2002 in accordance with
generally accepted government auditing standards.
Page 8 of 60
submitting it to the House and Senate Appropriations Committees.
Results in Brief: Objective 1 Customs? expenditure plan satisfies the
legislative conditions.
Legislative conditions Satisfies Does not satisfy 1. Meets OMB capital
planning and investment control review
recommendations.
manager. ACE expenditure plan.
compromised. 3. Immediately transfer responsibility and accountability
for the ITDS pilot to the ACE modernization program 4. Include any plans for
further investing in the ITDS pilot, including cost, benefit, and risk
justification, in the next
5. Clarify the roles and responsibilities of the ACE modernization
independent verification and validation (IV& V) agents to ensure that
independence is not
Results in Brief: Objective 2 Customs? expenditure plan is consistent with
open GAO
Consistent GAO open recommendations Complete In progress 1. Justify and make
investment decisions incrementally.
Results in Brief: Objective 3 Other observations: Customs faces considerable
risk in acquiring and implementing ACE.
1. ACE is technically and managerially complex Adds risk
and challenging. +
2. Customs did not meet key commitments made Adds risk
in its first ACE expenditure plan. +
3. Despite progress, Customs does not yet have Adds risk
important ACE management controls in place. +
4. Customs has accelerated its ACE acquisition Adds risk
plans.
CUMULATIVE RISK IS HIGH
Page 11 of 60
Background ACE Overview
ACE is intended to support Customs? efforts to protect our nation?s borders,
enforce trade laws, and facilitate the movement of legitimate trade. For
example, ACE is to
support all Customs functions associated with trade compliance (such as
defining, capturing, and managing regulations for import and export
compliance; establishing and managing accounts for trade participants; and
verifying credentials and managing compliance levels);
support other functions such as data analysis for trade law enforcement and
border security purposes;
support the trade information collection, processing, and dissemination
needs of 104 other government agencies; and
provide the trade community a single interface to the federal government.
Page 12 of 60
Background (cont.)
According to Customs, ACE will permit, for example, more efficient import
and export operations; increased legitimate cargo flow; reduced paper
burden; and more efficient and effective account management, including
revenue billing and collection operations.
We have testified that the need for Customs to modernize its trade
processing capability is undeniable because, among other things, Customs?
existing import processing system was paper- intensive, error- prone,
transaction- based, and out of step with the just- in- time inventory
practices of the trade community. 4
4 U. S. General Accounting Office, U. S. Customs Service: Observations on
Selected Operations and Program Issues , GAO/ T- GGD/ AIMD- 00- 150
(Washington, D. C.: Apr. 20, 2000).
Page 13 of 60
Background (cont.) ACE Acquisition Strategy
Customs is using an integration contractor, supported by subcontractors, to
develop and deploy ACE. The contractor and its subcontractors are
collectively called the e- Customs Partnership or eCP.
Initial modernization contract activities include establishing a management
foundation (e. g., people, processes, tools) for Customs? overall
modernization program. ACE is to be the first modernized system under this
program.
Customs? current plans call for release of ACE capabilities in four
increments implemented over 4 years. Each increment is to contain discrete
functionality, while building on the functionality in the previous
increments.
Page 14 of 60
Background (cont.) ACE Management Structure
Page 15 of 60
expenditure plan. Congress for $45 million.
Customs
Background (cont.)
Customs was investing in an ITDS pilot project in parallel with, and
separate from, ITDS efforts under the ACE modernization program; and
Customs had defined conflicting roles and responsibilities for its
independent verification and validation (IV& V) contractor.
Accordingly, we made the following recommendations, which Customs agreed to
implement:
Immediately transfer responsibility and accountability for the ITDS pilot to
the ACE modernization program manager.
Include any plans for further investing in the ITDS pilot, including cost,
benefit, and risk justification, in the next ACE expenditure plan.
Clarify the roles and responsibilities of the ACE modernization IV& V agent
to ensure that independence is not compromised.
Page 17 of 60
Background (cont.)
On April 24 and 25, 2001, Customs? House and Senate appropriation
subcommittees, respectively, released to Customs an additional $45 million
in response to Customs? first ACE expenditure plan.
On April 27, 2001, Customs awarded a 5- year, indefinite- delivery
indefinitequantity contract, with options to extend the contract to not more
than 15 years, to International Business Machines (IBM) Global Services.
On February 1, 2002, Customs submitted its second ACE expenditure plan,
seeking release of $206.9 million.
The following two figures illustrate the recent history of ACE
appropriations, expenditure plans, funding, and major contract events.
Page 18 of 60
Background (cont.) ACE funding releases shown by fiscal year appropriations
FY 2001 appropriations = $130 million FY 2002 appropriations = $300 million
March 2001 $5 million
April 2001 funding
$45 million release
funding release
$127 million $173 million
$80 million Second expenditure plan for $206.9 million
Next expenditure plan (planned for May 2002)
Page 19 of 60
Background (cont.) Illustrated chronology of ACE expenditure plans and
funding
Contractor authorized to
Contract proceed
awarded Initial three $5M
$45M task orders
release release
signed First
Second plan
plan Next plan
submitted submitted 12/ 00 3/ 01 6/ 01 9/ 01 12/ 01 3/ 02 6/ 02
Page 20 of 60
Background (cont.)
Customs? second ACE expenditure plan requests funding for the following:
Areas of expenditure Amount (millions) Funds to cover first expenditure plan
cost variance 5 $22.4 National Customs Automation Program (NCAP) operations
& maintenance $3.1 Program management and support $23.5 SAP 6 infrastructure
$7.3 ACE increment 1, release 1.0 design, development, test, and deployment
$104.0 ACE increment 2, detailed requirements and planning $11.2 Customs
Modernization Office (CMO) management reserve $35.5 Total $206.9
5 The second expenditure plan requests funding equivalent to the difference
between the negotiated price for the task orders ($ 47.4 million) and the
funding released as a result of its first expenditure plan ($ 25 million). 6
These funds will be used to acquire hardware and supporting infrastructure
for Customs? SAP Implementation Project, a joint project of the Office of
Finance and the Office of Information Technology. SAP AG is a vendor that
makes a software package called R/ 3, which is to be used by ACE to account
for payments, collections, and refunds. Additionally, R/ 3 is to replace
Customs? existing core budget, accounting, procurement, and property
accountability systems.
Page 21 of 60
Background (cont.)
Customs attributes its first expenditure plan variance to underestimates or
omissions in the following areas:
Labor: $12.9 million (58 percent of the total variance) Variance (millions)
Labor rates and skill mix $6.8 Tasks (e. g., change management, Stakeholder
Relations Council) $4.5 Other (e. g., performance- based contracting) $1.6
Other direct cost: $9. 9 million (42 percent of the total variance) Task
order planning $2.0 Facilities and support infrastructure $6. 7 Other (e.
g., performance- based contracting, travel costs) $1.2 Total $22.8
Page 22 of 60
1. Plan meets OMB capital planning and investment control review
requirements, including OMB Circular A- 11, part 3. Legislative conditions
Satisfies? Results of our analysis
regulations;
Results: Objective 1
Customs? expenditure plan satisfies the legislative conditions.
2. Plan complies with Customs?
4. Plan is reviewed and approved by Customs Investment Review Board (IRB),
Treasury, and OMB.
Results: Objective 1 (cont.)
Legislative conditions Satisfies? Results of our analysis
recommendations. 1. Justify and make investment decisions incrementally, and
for each increment:
Use disciplined investment
processes; Require a favorable return on investment and compliance with
Customs architecture before making an investment decision; and Prepare
realistic and supportable
benefit expectations; Validate actual costs and benefits
once an increment is piloted. Complete In progress
resources;
increment;
process; and
Results: Objective 2
Customs? expenditure plan is consistent with GAO open GAO recommendations
Consistent? Actions planned or taken
2. Strengthen ACE software
3. Immediately transfer responsibility and accountability for ITDS pilot to
the ACE modernization program manager.
4. Include any plans for further investing in the ITDS pilot (including
cost, benefit, and risk justification) in the next ACE expenditure plan.
5. Clarify the roles and responsibilities of the ACE modernization IV& V
agent to ensure that independence is not compromised.
Complete In progress
pilot.
Results: Objective 2 (cont.)
GAO recommendations Consistent? Actions planned or taken
Results: Objective 3 Other observations: Customs faces considerable risk in
acquiring and implementing ACE.
1. ACE is technically and managerially complex Adds Risk and challenging.
+ 2. Customs did not meet key commitments made
Adds Risk in its first ACE expenditure plan.
+ 3. Despite progress, Customs does not yet have
Adds Risk important ACE management controls in place.
+ 4. Customs has accelerated its ACE acquisition
Adds Risk plans.
CUMULATIVE RISK IS HIGH
Page 29 of 60
example, ACE is to
Results: Objective 3 (cont.)
ACE is a large undertaking, as measured by the level of development
resources needed. In particular, the cost of developing the entire system is
estimated to be between $1.5 and $1.6 billion, and the number of
professional staff needed to develop the system is estimated to be about 500
to 700.
ACE is also intended to support and satisfy the needs of 104 other
government agencies with international trade- related missions as well as
the needs of thousands of importers. Accordingly, acquiring a system that
meets this diverse set of needs will require an enormous level of
stakeholder participation, cooperation, and interaction.
ACE is intended to fundamentally change not only the way that Customs
operates internally but also the way that importers and other government
agencies operate. The ability of these organizations to accept and absorb
such operational change is not guaranteed; their uncertainty has been
recognized as a reason that COTS- based modernization efforts have not
consistently met expectations.
Page 32 of 60
Results: Objective 3 (cont.) Observation 2: Customs did not meet key
commitments made in its
first ACE expenditure plan.
One indication of future performance, and thus risk, is past performance.
Customs? past performance in acquiring ACE fell short of the commitments it
made in the first expenditure plan.
In the first plan, Customs requested $25 million for three ACE tasks: (1)
program management, (2) enterprise engineering, and (3) requirements
development. Customs reports that it met commitments for task 3, but it did
not meet all of the commitments for tasks 1 and 2. Specifically, because the
cost to complete the three tasks collectively increased $22.8 million over
what Customs requested in the first plan, Customs funded the first two tasks
for only 6 months, rather than 18 months as specified in its expenditure
plan. (See table on next page.)
Page 33 of 60
Results: Objective 3 (cont.)
Task Summary of commitment in first plan Actual result Impact on second plan
1. Program Establish a program office to manage the
Work was $22.4 million to
management modernization program for an 18- month period. performed for 6
fund the
months. remaining 12
2. Enterprise Establish and maintain the systems engineering
Work was months on the
engineering capability for implementing ACE for an 18 month performed for 6
first two task period. This work includes, for example, updating
months. orders.
and maintaining Customs? Enterprise Architecture, Enterprise Life Cycle
Model, and Technical Reference Model.
3. ACE Develop high- level ACE system requirements, and
ACE system None.
requirements detailed increment 1 requirements, as well as ACE
requirements, and planning
project plan outlining the transition from ACS to including the
ACE. This work was to be done within 6 months. detailed
requirements for ACE increment 1, along with project plan, were delivered
within 6 months.
Page 34 of 60
Results: Objective 3 (cont.)
According to Customs, the $25 million request in the first expenditure plan
was based on a 1999 estimate. Customs officials acknowledged that derivation
of this estimate lacked sufficient rigor because it was not based on
detailed work statements.
Customs? inability to meet its initial commitments casts uncertainty on its
ability to meet future commitments and thus introduces an element of risk to
the ACE project.
Page 35 of 60
Results: Objective 3 (cont.) Observation 3: Despite progress, Customs does
not yet have key
management controls for ACE. a. Extension of the enterprise architecture to
support ACE engineering tasks is not complete.
Enterprise architectures are corporate blueprints to guide and constrain
operational and technological change, particularly investments in large
system development or acquisition efforts, such as ACE. To effectively guide
and constrain these efforts, enterprise architectures should be updated to
ensure that they reflect changes in the strategic business direction of the
enterprise. Further, they should be updated in a way that ensures
consistency and integration across the full scope of the enterprise?s
business areas; for Customs, these areas are commercial, enforcement, and
administration. Also, these architectures should be sufficiently extended to
recognize systemspecific business requirements upon which later design
decisions will be based. Such extensions are both normal and a necessary
part of methodologically sound system design and development. To minimize
risk, these architectural activities should be completed before system
design and development begins.
Page 36 of 60
Results: Objective 3 (cont.)
Customs recognizes the need to ensure that its ACE system architecture is
further defined, and it has instituted processes for providing such
assurance. For example, it is having eCP extend the ACE system architecture
to include logical data models and data flows and expand the standards for
Web services. However, Customs has yet to commit to work on its enterprise
architecture that eCP officials say is needed to ensure that Customs?
enterprise architecture is internally consistent, sufficiently scoped, and
adequately extended.
Instead, Customs? plans to provide ?just enough? enterprise architecture
?justin-
time? to guide and constrain the design of future ACE increments. According
to eCP officials, parallel definition of the enterprise and system
architectures is not typically done. The reason it is not typically done is
because it risks not having sufficient enterprise context available to
effectively design the system.
We believe that the reason for Customs? parallel approach is the priority
being given to meeting the ACE schedule. In our view, such a schedule-
driven approach risks producing an enterprise architecture extension that is
not sufficiently complete to support ACE design activities; this, in turn,
could result in costly system rework and a failure to deliver system
capabilities on time and within budget.
Page 37 of 60
Results: Objective 3 (cont.) b. Customs does not have an effective strategy
for meeting its modernization human capital needs.
Effective human capital management is fundamental to any organization (or
organizational component), such as the CMO. To be effective, information
technology (IT) human capital management strategies should provide for
identifying knowledge, skills, and experience requirements;
continuously assessing existing capabilities; and
developing effective strategies and implementing plans to fill any gaps,
including developing compensation packages to attract and retain qualified
individuals.
Since the CMO?s establishment in June 2000, its approach to meeting its
human capital needs has been to identify a set of CMO positions, develop
position descriptions for each, advertise these positions, and select among
applicants.
Using this approach, the CMO currently does not have the human capital
capacity it needs to effectively manage ACE. (See table on next page.) To
compensate, Customs? approach has been to rely on support contractors.
Page 38 of 60
Results: Objective 3 (cont.)
Status of CMO positions:
Position title Level Filled? Number of months vacant Executive Director SES
Yes -
Deputy Director GS- 15 No 2 Requirements Executive GS- 15 Yes -
Requirements Executive GS- 15 Yes -
Executive Support GS- 14 Yes -
Executive Support GS- 7 Yes -
Requirements Management Lead GS- 14 No 7 Business Management Lead GS- 14 No
5 Communications Lead GS- 14 No 10 Program Control Lead GS- 14 No 0 Systems
Engineering Lead GS- 13/ 14 No 9 Program Analyst GS- 9/ 13 No 0 Program
Analyst GS- 9/ 13 No 5
Page 39 of 60
Results: Objective 3 (cont.)
In our view, the CMO?s lack of adequate human capital can be traced to not
having defined and implemented an effective human capital strategy. Such a
strategy would, at a minimum, completely and correctly define the CMO?s
human capital needs (both the proper skill levels and the commensurate
compensation packages needed to attract and retain them) and provide
Customs? full commitment (resources and management attention) for pursuing
short- term and long- term actions to meet them. Unless Customs addresses
its human capital weaknesses, it increases the risk of not delivering ACE
capabilities on time and within budget.
Page 40 of 60
Results: Objective 3 (cont.) c. Software acquisition management processes
are incomplete.
Effective IT management processes, such as processes for acquiring software,
are essential to software- intensive system acquisitions. SEI, recognized
for its expertise in software processes, has defined the tenets of effective
software acquisition. The SA- CMM identifies, among other things, a number
of key process areas that are necessary to effectively manage software-
intensive system acquisition.
As discussed earlier, Customs is addressing our recommendations aimed at
correcting the ACE software acquisition management weaknesses that we
identified in our February 1999 report, 12 and Customs officials told us
that they plan to eventually assess their SA- CMM capabilities, although no
time frame has been set.
12 U. S General Accounting Office, Customs Service Modernization: Serious
Management and Technical Weaknesses Must Be Corrected, GAO/ AIMD- 99- 41,
Washington, D. C., Feb. 26, 1999.
Page 41 of 60
Results: Objective 3 (cont.)
Using Customs? reported data on the status of its efforts to address these
weaknesses, we determined that one of the SEI key process areas is generally
complete (i. e., the vast majority of practices in a key process area are in
place), five key process areas are works in progress (i. e., one or more
practices in a key process area are in place), and two key process areas are
not started (i. e., no practices in a key process area are in place).
Page 42 of 60
the project are included. the contractor. effective software acquisition.
contract requirements. capable of satisfying the specified requirements.
Requirements development and management: establishing a common and
unambiguous definition of software acquisition
Contract tracking and oversight: ensuring that the software
Results: Objective 3 (cont.)
Status at Customs on SEI key process areas: SEI SA- CMM key process area
Generally Software acquisition planning: ensuring that reasonable planning
for the software acquisition is conducted and that all elements of
Solicitation: ensuring that award is made to the contractor most
requirements understood by the acquisition team, system user, and Project
office management: managing the activities of the project office and
supporting contractor( s) to ensure a timely, efficient, and
activities under contract are being performed in accordance with contract
requirements, and that products and services will satisfy
complete Work in
transition to support. organization.
the acquisition process. products being acquired to the eventual software
support
Results: Objective 3 (cont.)
SEI SA- CMM key process area Generally Evaluation: determining that the
acquired software products and services satisfy contract requirements prior
to acceptance and
Transition to support: providing for the transition of the software
Acquisition risk management: 13 identifying risks as early as possible,
adjusting acquisition strategy to mitigate those risks, and developing and
implementing a risk management process as an integral part of
13 Acquisition risk management is an SEI SA- CMM level 3 key process area.
complete Work in
progress Not started
Results: Objective 3 (cont.)
As shown in the table above, Customs has yet to complete many key processes
and, in particular, the evaluation and transition to support key process
areas have not been started. The purpose of evaluation is to determine that
the acquired software products and services satisfy contract requirements
before acceptance. Evaluation should include developing and managing
evaluation requirements in conjunction with developing software technical
requirements.
Transition to support is to provide for the effective and efficient ?hand
off? of the acquired software products to the support organization
responsible for software maintenance and should include designating a group
that is responsible for coordinating transition and support activities.
Customs officials agree that its software acquisition processes are
incomplete and told us that Customs delayed addressing the evaluation and
transition to support key process areas because resources (i. e., staff)
were needed for higher priority activities. In the absence of an evaluation
capability, Customs said that they are using a support contractor to perform
evaluation activities.
Page 45 of 60
Results: Objective 3 (cont.)
Because Customs has not yet established effective software acquisition
processes, it does not have adequate assurance that ACE capabilities will be
delivered on time and within budget.
Page 46 of 60
Results: Objective 3 (cont.) d. Expenditure plan cost estimates have not
been validated by Customs.
Producing reliable cost estimates for inclusion in ACE expenditure plans is
essential. Without such estimates, Customs cannot provide congressional
committees with the information needed to exercise important ACE oversight.
Ensuring that cost estimates are reliable requires, at a minimum, that
Customs satisfy itself that rigorous, analytically verifiable methods are
used to derive the estimates. To this end, SEI has defined models for (1)
establishing an institutional estimating capability and (2) validating the
reliability of a projectspecific estimate. 14
As discussed earlier, Customs? cost estimates for the initial ACE tasks in
its first expenditure plan were not reliable because Customs did not use a
rigorous approach in deriving them.
14 Software Engineering Institute, Checklists and Criteria for Evaluating
the Cost and Schedule Estimating Capabilities of Software Organizations (
CMU/ SEI- 95- SR- 005 and A Manager s Checklist for Validating Software Cost
and Schedule Estimates ( CMU/ SEI- 95- SR- 004 .
Page 47 of 60
Results: Objective 3 (cont.)
For its second expenditure plan, Customs did not use any method to derive
contract- related costs. Instead, it simply passed along estimates developed
by the ACE contractor, except for Customs? internal (CMO) costs and the
management reserve. According to Customs, it plans to use an independent
contractor to validate cost estimates after it submits future expenditure
plans and to use this information in negotiating task orders.
For the management reserve of $35.5 million requested in the second
expenditure plan, Customs used a contractor to derive this estimate. This
estimate represents about 21 percent of the total funds requested (excluding
the reserve) in the expenditure plan. We found that this reserve estimate
lacked analytical justification and relied extensively on the
unsubstantiated assumptions and subjective judgments of a few individuals.
In fact, we could have used the same methodology used by the contractor,
made our own equally valid assumptions and judgments, and produced estimates
either considerably higher or lower.
Page 48 of 60
Results: Objective 3 (cont.)
Moreover, available information from studies and comparable system
undertakings, such as the Internal Revenue Service?s Business Systems
Modernization (BSM), do not support this level of management reserve. For
example, the BSM management reserve is 10 percent, and a study covering
about 500 Defense contracts showed a mean management reserve of about 4
percent.
We conclude that the management reserve cost estimate appears overstated,
and the reliability of the other estimates in the second expenditure plan is
uncertain. Without reliable estimates, Customs increases the risk that it
will repeat its first expenditure plan performance, and that it will not
meet expenditure plan commitments.
Page 49 of 60
Results: Objective 3 (cont.) Observation 4: Customs has accelerated its ACE
acquisition plans.
When a system is incrementally acquired, it is important to carefully manage
the extent to which design, development, testing, and implementation of the
increments overlap (i. e., occur concurrently). The goal is to ensure that a
given increment has been sufficiently matured (so that it is free of
material defects and performs as intended) before later, dependent
increments are built.
While some degree of increment concurrency is appropriate, extensive overlap
of increments? detailed design, development, and testing activities is
generally not desirable because it increases the risk of overall system
rework. Reasons for this include, carrying over unresolved defects into
later increments; maintaining control over a proliferating number of system
versions; stretching critical resources (such as people, development and
testing laboratories); and introducing incremental change into the
organization with such frequency that it exhausts the organization?s
capacity to absorb the change.
Page 50 of 60
Results: Objective 3 (cont.)
As previously stated, Customs has committed to implementing our prior
recommendation to incrementally acquire and invest in ACE. Until recently,
Customs? incremental strategy provided for acquiring and deploying ACE in
four major increments spanning 5 years, with the first three of the four
increments consisting of two subincrements or releases. Even under this 5-
year strategy, Customs? plans provided for some overlap among increments.
For example, design activities in one increment were generally occurring
concurrently with development activities in later increments. (See figure on
next page.)
In January 2002, Customs decided to accelerate its plans for acquiring and
deploying ACE, electing to deliver ACE?s four increments in 4 years rather
than 5 because, according to the Commissioner of Customs, ACE is important
to our country?s security and the future of trade facilitation. While
Customs has yet to develop a program schedule detailing the extent to which
this compressed schedule will increase concurrency among system increment
design, development, and testing, Customs and its integration contractor
acknowledge that increased concurrency among increments will occur.
Page 51 of 60
Results: Objective 3 (cont.)
ACE 5- year implementation plan:
CY2001 CY2001 CY2002 CY2002 CY2003 CY2003 CY2004CY2005 CY2004CY2005 CY2006
CY2006 CY2007 CY2007 I I crement crement 1, 1, Release Release 1
1 I I crement crement 1, 1, Release Release 2
2 I I crement crement 2, 2, Release Release 1
1 I I crement crement 2, 2, Release Release 2
2 I I crement crement 3, 3, Release Release 1
1 I I crement crement 3, 3, Release Release 2
2 Increme Increme t t 4
4 Requirements Requirements & & pla pla ning
ning Desig Desig Develop Develop
Deploy Deploy
Page 52 of 60
Results: Objective 3 (cont.)
The contractor has also acknowledged that this increased concurrency will
exacerbate preexisting risks, such as
the possibility that an increment?s requirements will not be fully
understood before design and development is to begin;
the possibility that the delivery of COTS and hardware products will not be
timely; and
the possibility that changes will be required to Customs? legacy systems
with which ACE must interface.
The contractor has also stated that the compressed schedule introduced new
risks, such as
funding requirements will be so high that budget approval is unlikely;
staffing requirements will be so high that physical housing of all personnel
will be difficult; and
the trade community will not have sufficient lead time to make changes
necessary to support ACE.
Page 53 of 60
Results: Objective 3 (cont.)
Combined with the other risks that we cited earlier (i. e., unresolved
defects carrying over to later increments, version control, overtaxing core
resources, and capacity for change absorption), the chances of not
delivering ACE capabilities on time and within budget are increased
appreciably.
Page 54 of 60
Conclusions
Customs? second ACE expenditure plan satisfies the legislative conditions
and is consistent with our open recommendations. Nevertheless, ACE
introduces a host of unavoidable risks, including the complexity of the
work, the dependence of successful implementation on the interaction of a
large and diverse set of stakeholders, and these stakeholders? ability to
absorb the considerable operational changes to be introduced through ACE.
One indicator of Customs? ability to successfully address these risks and
deliver ACE capabilities on time and within budget is its past performance.
Customs did not fulfill important commitments that it made in its first
expenditure plan because the plan?s estimated costs for certain tasks were
not reliable. Because of this track record, the risks in moving forward on
ACE are amplified.
Page 55 of 60
Conclusions (cont.)
Another indicator of Customs? ability to successfully acquire and implement
ACE is the effectiveness of the management controls it has in place to
govern its system acquisition and implementation activities. For various
reasons cited by Customs officials, including limited resources and schedule
demands, Customs has yet to establish several important management controls,
such as an updated enterprise architecture, adequate human capital capacity,
complete software acquisition management processes, and reliable cost
estimating. The absence of these controls further compounds the risks facing
Customs on ACE.
Notwithstanding these risks, Customs has recently decided to develop and
deploy the system faster. This decision exacerbates an already risky
undertaking by increasing the difficulty of the work without first providing
for a commensurate increase in the capabilities needed to perform the work.
Page 56 of 60
Recommendations for Executive Action
To address the risks discussed in this briefing, we recommend that the
Customs Service Commissioner direct the Chief Information Officer, as the
designated Modernization Executive, to take the following actions:
Before building each ACE release (i. e., beginning detailed design and
development), certify to Customs? House and Senate appropriation
subcommittees that the enterprise architecture has been sufficiently
extended to provide the requisite enterprise design content and has been
updated to ensure consistency and integration across business areas.
Immediately develop and implement a CMO human capital management strategy
that provides both near- term and long- term solutions to the CMO?s human
capital capacity limitations, including defining the office?s skill and
capability needs in terms that will allow Customs to attract qualified
individuals and that will provide sufficient rewards and training, linked to
performance, that promote their retention.
Page 57 of 60
Recommendations for Executive Action
Develop and implement process controls for the SEI SA- CMM level 2 key
process areas and the level 3 acquisition risk management key process area.
By September 30, 2002, conduct and report to Customs? House and Senate
appropriation subcommittees on the results of an internal assessment of the
CMO?s maturity against the SEI SA- CMM level 2 key process areas, and the
level 3 acquisition risk management key process area.
Develop and implement a rigorous and analytically verifiable cost estimating
program that embodies the tenets of effective estimating as defined in SEI?s
institutional and project- specific estimating models.
Limit future expenditure plan requests for management reserve funds to 10
percent of the total funds requested for the program or adequately justify
any management reserve requests in excess of 10 percent.
Page 58 of 60
Recommendations for Executive Action
Address the risks associated with the accelerated ACE acquisition strategy,
including (1) immediately analyzing the risks associated with the degree of
design, development, and testing concurrency across ACE increments that is
inherent in Customs? 4- year, schedule- driven acquisition strategy; (2)
reconsidering the merits of this accelerated strategy; and (3) within 90
days of the date of this briefing, reporting to Customs? House and Senate
appropriations subcommittees on the agency?s strategy going forward and its
plans for mitigating the inherent risks associated with this strategy.
Page 59 of 60
Agency Comments
In commenting on a draft of this briefing, the Customs Modernization Office
Executive Director agreed with our findings, conclusions, and
recommendations.
Page 60 of 60
Appendi x II Comments from the U. S. Customs Service
(310235)
GAO?s Mission The General Accounting Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance to
help Congress make informed oversight, policy, and funding decisions. GAO?s
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of The fastest and easiest way to obtain copies of GAO
documents at no cost is through the Internet. GAO?s Web site (www. gao. gov)
contains abstracts and fulltext GAO Reports and
files of current reports and testimony and an expanding archive of older
Testimony
products. The Web site features a search engine to help you locate documents
using key words and phrases. You can print these documents in their
entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as ?Today?s Reports,? on its Web
site daily. The list contains links to the full- text document files. To
have GAO e- mail this list to you every afternoon, go to www. gao. gov and
select ?Subscribe to daily E- mail alert for newly released products? under
the GAO Reports heading.
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out to
the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U. S. General Accounting Office 441 G Street NW, Room LM Washington, D. C.
20548
To order by Phone: Voice: (202) 512- 6000 TDD: (202) 512- 2537 Fax: (202)
512- 6061
To Report Fraud, Contact: Waste, and Abuse in
Web site: www. gao. gov/ fraudnet/ fraudnet. htm E- mail: fraudnet@ gao. gov
Federal Programs Automated answering system: (800) 424- 5454 or (202) 512-
7470
Public Affairs Jeff Nelligan, managing director, NelliganJ@ gao. gov (202)
512- 4800 U. S. General Accounting Office, 441 G Street NW, Room 7149
Washington, D. C. 20548
a
GAO United States General Accounting Office
Why GAO Did This Study
The U. S. Customs Service is in the early stages of a multiyear,
multibillion- dollar project: the Automated Commercial Environment (ACE), a
new import processing system that is to support effective and efficient
movement of goods into the United States. By congressional mandate, Customs?
spending plans for ACE must meet certain conditions, including being
reviewed by GAO. In this study, GAO addresses whether Customs? latest plan
satisfies congressional conditions and is consistent with open GAO
recommendations, and it identifies opportunities for strengthening project
management.
May 2002 CUSTOMS SERVICE MODERNIZATION Management Improvements Needed on
High- Risk Automated Commercial Environment Project
This is a test for developing highlights for a GAO report. The full report,
including GAO's objectives, scope, methodology, and analysis is available at
www. gao. gov/ cgi- bin/ getrpt? GAO- 02- 545. For additional information
about the report, contact Randolph C. Hite (202- 512- 3439). To provide
comments on this test highlights, contact Keith Fultz (202- 512- 3200) or E-
mail HighlightsTest@ gao. gov.
Highlights of GAO- 02- 545, a report to the Subcommittee on Treasury and
General Government, Senate Committee on Appropriations, and the Subcommittee
on Treasury, Postal Service, and General Government, House Committee on
Appropriations.
What GAO Recommends
To increase the chances of delivering needed system capabilities on time and
within budget, GAO is making recommendations to the commissioner aimed at
improving Customs? management of ACE, including strengthening system
alignment with Customs? enterprise architecture, cost estimating, human
capital capacity, software process maturity, and sequencing of incremental
releases. Customs concurred with GAO?s recommendations and described
specific actions that it is taking to respond to each.
United States General Accounting Office
What GAO Found
Customs? February 2002 ACE spending plan is the second in a series of
legislatively required plans. This plan covers certain project management
tasks as well as the definition, design, and development of the first ACE
increment. GAO found that the plan meets the legislative conditions that
Congress imposed on Customs and is consistent with GAO?s open
recommendations. Nevertheless, investment in ACE is a high- risk endeavor
for several reasons:
The system?s size, performance parameters, and organizational impact make
it technically and managerially complex. For example, the estimated cost for
acquisition alone is about $1.5 billion, the system is to operate around-
the- clock, and it is to drive fundamental business process reform not only
within Customs, but also within numerous other federal agencies and
countless private sector importers. Customs fell far short of key
commitments made in its first spending
plan because it severely underestimated costs. This track record casts doubt
on Customs? ability to meet future commitments. Despite progress, Customs
still lacks important acquisition
management controls. For example, it needs to update its enterprise
architecture (its agencywide modernization blueprint) to support system
design and development, and it needs to advance its acquisition office?s
human capital and software management processes. Customs has recently
decided to compress its time frame for delivering
the system from 5 to 4 years. This exacerbates the level of project risk by
introducing more overlap among incremental system releases.
Because of the system?s national importance, Customs is taking a schedule-
driven approach to acquiring ACE. However, without the management capacity
to effectively acquire such a large and complex system, particularly in
light of Customs? performance to date and the accelerated acquisition and
deployment schedule, this approach could backfire. Full system capabilities
may take longer and cost more to acquire, deploy, and make operational,
because the system delivered under the accelerated schedule could require
considerable rework.
ACE Is High Risk for Several Reasons
Cumulative risk is high Size and complexity Past performance Management
control weaknesses
Accelerated schedule G A O Accountability Integrity Reliability
Highlights
G A O Accountability Integrity Reliability
Highlights
Page 1 GAO- 02- 545 Customs Service Modernization United States General
Accounting Office
Washington, D. C. 20548 Page 1 GAO- 02- 545 Customs Service Modernization
A
Page 2 GAO- 02- 545 Customs Service Modernization
Page 3 GAO- 02- 545 Customs Service Modernization
Page 4 GAO- 02- 545 Customs Service Modernization
Page 5 GAO- 02- 545 Customs Service Modernization
Page 6 GAO- 02- 545 Customs Service Modernization
Appendix I
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 7 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 8 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 9 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 10 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 11 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 12 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 13 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 14 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 15 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 16 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 17 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 18 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 19 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 20 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 21 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 22 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 23 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 24 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 25 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 26 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 27 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 28 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 29 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 30 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 31 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 32 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 33 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 34 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 35 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 36 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 37 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 38 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 39 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 40 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 41 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 42 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 43 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 44 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 45 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 46 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 47 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 48 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 49 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 50 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 51 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 52 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 53 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 54 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 55 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 56 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 57 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 58 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 59 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 60 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 61 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 62 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 63 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 64 GAO- 02- 545 Customs Service Modernization
Appendix I Customs? Second Automated Commercial Environment (ACE)
Expenditure Plan
Page 65 GAO- 02- 545 Customs Service Modernization
Page 66 GAO- 02- 545 Customs Service Modernization
Appendix II
Appendix II Comments from the U. S. Customs Service
Page 67 GAO- 02- 545 Customs Service Modernization
Appendix II Comments from the U. S. Customs Service
Page 68 GAO- 02- 545 Customs Service Modernization
Appendix II Comments from the U. S. Customs Service
Page 69 GAO- 02- 545 Customs Service Modernization
Appendix II Comments from the U. S. Customs Service
Page 70 GAO- 02- 545 Customs Service Modernization
Appendix II Comments from the U. S. Customs Service
Page 71 GAO- 02- 545 Customs Service Modernization
Appendix II Comments from the U. S. Customs Service
Page 72 GAO- 02- 545 Customs Service Modernization
Appendix II Comments from the U. S. Customs Service
Page 73 GAO- 02- 545 Customs Service Modernization
Appendix II Comments from the U. S. Customs Service
Page 74 GAO- 02- 545 Customs Service Modernization
United States General Accounting Office Washington, D. C. 20548- 0001
Official Business Penalty for Private Use $300
Address Correction Requested Presorted Standard
Postage & Fees Paid GAO Permit No. GI00
*** End of document. ***