Cargo Container Inspections: Preliminary Observations on the
Status of Efforts to Improve the Automated Targeting System
(30-MAR-06, GAO-06-591T).
U.S. Customs and Border Protection's (CBP) Automated Targeting
System (ATS)--a computerized model that CBP officers use as a
decision support tool to help them target oceangoing cargo
containers for inspection--is part of CBP's layered approach to
securing oceangoing cargo. GAO reported in February 2004 on
challenges CBP faced in targeting oceangoing cargo containers for
inspection and testified before Congress in March 2004 about the
findings in that report. The report and testimony outlined
recommendations aimed at (1) better incorporating recognized
modeling practices into CBP's targeting strategy, (2)
periodically adjusting the targeting strategy to respond to
findings that occur during the course of its operation, and (3)
improving implementation of the targeting strategy. This
statement for the record discusses preliminary observations from
GAO's ongoing work related to ATS and GAO's 2004 recommendations
addressing the following questions: (1) What controls does CBP
have in place to provide reasonable assurance that ATS is
effective at targeting oceangoing cargo containers with the
highest risk of smuggled weapons of mass destruction? (2) How
does CBP systematically analyze security inspection results and
incorporate them into ATS? and (3) What steps has CBP taken to
better implement the rest of its targeting strategy at the
seaports?
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-591T
ACCNO: A50480
TITLE: Cargo Container Inspections: Preliminary Observations on
the Status of Efforts to Improve the Automated Targeting System
DATE: 03/30/2006
SUBJECT: Cargo security
Comparative analysis
Computer modeling
Counterterrorism
Deep water ports
Harbors
Homeland security
Inspection
Internal controls
Operational testing
Performance measures
Risk management
Smuggling
Terrorism
Weapons of mass destruction
Customs Service Automated Targeting
System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-591T
* [1]Summary
* [2]Background
* [3]CBP Currently Does Not Have Reasonable Assurance That ATS Is
* [4]CBP Does Not Yet Have Performance Measures to Gauge the Effe
* [5]CBP Is Not Yet Using the Results of Random Inspections to As
* [6]CBP Has Not Yet Tested the Effectiveness of ATS in Targeting
* [7]CBP Is Working to Address Peer Review Recommendations
* [8]Although CBP Strives to Refine ATS for Intelligence Informat
* [9]CBP Adjusts ATS for Targeting Cargo Containers for Inspectio
* [10]CBP Targeting Officers at the Seaports Have Provided Few Sug
* [11]CBP Is Not Using Inspection Results to Systematically Adjust
* [12]CBP Has Taken Steps to Better Implement the Targeting Strate
* [13]CPB Has Implemented a Testing and Certification Process for
* [14]Despite CBP Action to Address Longshoremen's Safety Concerns
* [15]GAO Contacts and Acknowledgments
* [16]Appendix I: Scope and Methodology
* [17]Order by Mail or Phone
Statement forthe Record to the Permanent Subcommittee on Investigations,
Committee on Homeland Security and Governmental Affairs, U.S. Senate
United States Government Accountability Office
GAO
For Release on Delivery Expected at 9:30 a.m. EST
Thursday, March 30, 2006
CARGO CONTAINER INSPECTIONS
Preliminary Observations on the Status of Efforts to Improve the Automated
Targeting System
Statement for the Record by Richard M. Stana, Director Homeland Security
and Justice Issues
GAO-06-591T
On April 19, 2007, references to GAO-04-352SU were revised in the
footnotes on pages 1, 8, 9, 12, 13, and 14. The references did not
identify the document as SU, which indicates a "sensitive but
unclassified" document not available to the public. That document was not
and is not available on the GAO Web site.
Mr. Chairman and Members of the Subcommittee:
I appreciate the opportunity to comment on our ongoing work on the U.S.
Customs and Border Protection's (CBP) Automated Targeting System (ATS)--a
computerized model that CBP officers use as a decision support tool to
help them target oceangoing cargo containers for inspection.^1
In the aftermath of the terrorist attacks of September 11, 2001, there is
heightened concern that terrorists will attempt to smuggle a weapon of
mass destruction (e.g., a nuclear, biological, or radiological explosive
device) into the United States using one of the 11 million cargo
containers that arrive at our nation's seaports. Because of the large
volume of imported containers, CBP maintains that it is unable to
physically inspect all oceangoing containers without disrupting the flow
of commerce. Thus, CBP uses a multilayered strategy for addressing the
threat posed by the movement of oceangoing containers, of which ATS is a
key component.^2 CBP uses ATS to review documentation and assign a risk
score for all containers destined for U.S. ports. CBP officers located at
domestic ports or at 1 of the 40 foreign ports that participate in the
Container Security Initiative (CSI) then use these scores to help them
make decisions on the extent of additional documentary review and possible
physical inspection that will be conducted at the seaport.
We previously reported in February 2004 on the challenges CBP faced in
targeting oceangoing cargo containers for inspection^3 and testified
before this Subcommittee in March 2004 about the findings in that report.
^4 The report and testimony outlined recommendations aimed at (1)
incorporating recognized modeling practices into its targeting strategy,
such as conducting simulated events and initiating an external peer
review,^5 (2) periodically adjusting the targeting strategy to respond to
findings that occur during the course of its operation, and (3) improving
implementation of the targeting strategy at domestic seaports. This
subcommittee and other congressional requesters asked that we ascertain
whether CBP had implemented the recommendations we made to improve the
targeting strategy. Our work, in response to this request, has been under
way since last October, and we expect to complete the work and provide
this subcommittee and our other requesters with a report on the final
results later this year. In this statement, I will discuss our preliminary
observations on the status of these recommendations as part of the
following questions:
^1A model is a physical, mathematical, or otherwise logical representation
of a system, entity, phenomenon, or process.
^2In addition to ATS, CBP's multilayered strategy includes the (1)
Compliance Measurement Program, which randomly selects additional
containers to be physically examined; (2) the Container Security
Initiative, whereby CBP places staff at foreign seaports to work with
foreign counterparts to inspect high-risk containers before they are
shipped to the United States; and (3) the Customs-Trade Partnership
Against Terrorism which is a cooperative program between CBP and members
of the international trade community in which private companies agree to
improve the security of their supply chains. A supply chain consists of
all stages involved in fulfilling a customer request, including stages
conducted by manufacturers, suppliers, transporters, retailers, and
customers.
^3GAO, Homeland Security: Challenges Remain in the Targeting of Oceangoing
Cargo Containers for Inspection, GAO-04-352SU (Washington, D.C.: Feb. 20,
2004).
^4GAO, Homeland Security: Summary of Challenges Faced in Targeting
Oceangoing Cargo Containers for Inspection, [18]GAO-04-557T , (Washington,
D.C.: Mar. 31, 2004).
o What controls does CBP have in place to provide reasonable
assurance that ATS is effective at targeting oceangoing cargo
containers with the highest risk of smuggled weapons of mass
destruction?
o How does CBP systematically analyze security inspection results
and incorporate them into ATS?
o What steps has CBP taken to better implement the rest of its
targeting strategy at the seaports?
To address these questions, we interviewed CBP officials in headquarters
and visited six seaports: Baltimore, Charleston, Los Angeles-Long Beach,
Miami, New York-New Jersey, and Savannah. Because we did not select a
random sample of ports to visit, the results from these visits cannot be
generalized to ports nationwide. We also met with CBP's contractor
responsible for conducting CBP's peer review of ATS and longshoremen's
union representatives. We reviewed CBP's policies and procedures for
targeting and inspecting shipments, and its documentation on intelligence
gathering and dissemination, targeting strategies, random inspections,
training, and radiation safety as well as its peer review report. We also
examined information on officers trained and certified in CBP's Sea Cargo
Targeting Training course. We did not independently validate the
reliability of CBP's targeting results or test the effectiveness of ATS.
We conducted our work in response to this request from October 2005
through March 2006 in accordance with generally accepted government
auditing standards. Appendix I contains more detailed information on our
scope and methodology.
^5External peer review is a process that includes an assessment of the
model by independent and qualified external peers.
Summary
CBP has not yet put key controls in place to provide reasonable assurance
that ATS is effective at targeting oceangoing cargo containers with the
highest risk of containing smuggled weapons of mass destruction.^6 To
provide assurance that ATS targets the highest-risk cargo containers as
intended, CBP is (1) working to develop and implement performance measures
related to the targeting of cargo containers, (2) planning to compare the
results of its random inspections with its ATS inspection results, (3)
working to develop and implement a testing and simulation environment, and
(4) addressing recommendations contained in a 2005 peer review of ATS. CBP
expects to begin using performance measures in June 2006 and enter the
final phase of software development for its testing and simulation
environment at the same time. However, to date, none of these four
initiatives has been fully implemented. Thus, CBP does not yet have key
internal controls in place to be reasonably confident that ATS is
providing the best information to allocate resources for targeting and
inspecting containers that are the highest risk and not overlook
inspecting containers that pose a threat to the nation. ^7
CBP does not yet have a comprehensive, integrated system in place to
analyze security inspection results and incorporate them into ATS. An
integrated system would allow any of the various systems that CBP uses to
manage cargo inspection data to communicate with one another for the
purpose of analyzing combined data. CBP currently adjusts ATS based on
intelligence information it receives and has initiated a process to track
suggestions submitted by CBP targeting officers at the seaports for
modifying ATS. However, CBP has not yet implemented plans to refine ATS
based on findings from routine security inspections. Without a more
comprehensive feedback system, CBP is limited in refining ATS, a fact that
could hinder the overall effectiveness of the targeting strategy.
^6For purposes of this statement, when we state that CBP uses ATS to
target oceangoing cargo containers to identify weapons of mass
destruction, we are also including the different components that could be
used to create a weapon of mass destruction.
^7Internal control is an integral component of an organization's
management that provides reasonable assurance that the following
objectives are achieved: (1) effectiveness and efficiency of operations,
(2) reliability of financial reporting, and (3) compliance with applicable
laws and regulations.
CBP has taken steps to improve implementation of the targeting strategy at
the seaports. It has implemented a testing and certification process for
its officers who complete the Sea Cargo Targeting Course that should
provide better assurance of effective targeting practices. CBP has also
made a good faith effort to address longshoremen's safety concerns
regarding radiation emitted by nonintrusive inspection equipment by taking
actions such as working with longshoremen's unions and other maritime
organization to develop public radiation tests on the nonintrusive
inspection equipment. ^8 Nevertheless, CBP has not been able to persuade
one longshoremen's union to permit changes in the procedure for staging
containers to increase inspection efficiency at some West Coast seaports
where the union's members work.
Background
Oceangoing cargo containers have an important role in the movement of
cargo between global trading partners. Approximately 90 percent of the
world's trade is transported in cargo containers. In the United States
almost half of incoming trade (by value) arrives by containers aboard
ships. If terrorists smuggled a weapon of mass destruction into the nation
using a cargo container and detonated such a weapon at a seaport, the
incident could cause widespread death and damage to the immediate area,
perhaps shut down seaports nationwide, cost the U.S. economy billions of
dollars, and seriously hamper international trade.
The Department of Homeland Security and CBP are responsible for addressing
the threat posed by terrorist smuggling of weapons in oceangoing
containers. To carry out this responsibility, CBP uses a layered security
strategy. One key element of this strategy is ATS. CBP uses ATS to review
documentation, including electronic manifest information submitted by the
ocean carriers on all arriving shipments, to help identify containers for
additional inspection.^9 CBP requires the carriers to submit manifest
information 24 hours prior to a United States-bound sea container being
loaded onto a vessel in a foreign port. ATS is a complex mathematical
model that uses weighted rules that assign a risk score to each arriving
shipment in a container based on manifest information. As previously
discussed, CBP officers use these scores to help them make decisions on
the extent of documentary review or physical inspection to be conducted.
^8Nonintrusive inspection equipment uses technology to help determine the
contents of a container without opening it.
^9Cargo manifests are prepared by the ocean carrier to describe the
contents of a container.
ATS is an important part of other layers in the security strategy. Under
its CSI program, CBP places staff at designated foreign seaports to work
with foreign counterparts to identify and inspect high-risk containers for
weapons of mass destruction before they are shipped to the United States.
At these foreign seaports, CBP officials use ATS to help target shipments
for inspection by foreign customs officials prior to departing for the
United States. Approximately 73 percent of cargo containers destined for
the United States originate in or go through CSI ports.
ATS is also an important factor in the Customs-Trade Partnership Against
Terrorism (C-TPAT) program. C-TPAT is a cooperative program linking CBP
and members of the international trade community in which private
companies agree to improve the security of their supply chains in return
for a reduced likelihood that their containers will be inspected.
Specifically, C-TPAT members receive a range of benefits, some of which
could change the ATS risk characterization of their shipments, thereby
reducing the probability of extensive documentary and physical inspection.
CBP Currently Does Not Have Reasonable Assurance That ATS Is Effective
CBP does not yet have key controls in place to provide reasonable
assurance that ATS is effective at targeting oceangoing cargo containers
with the highest risk of containing smuggled weapons of mass destruction.
To address this shortcoming, CBP is (1) developing and implementing
performance metrics to measure the effectives of ATS, (2) planning to
compare the results of randomly conducted inspections with the results of
its ATS inspections, (3) developing and implementing a simulation and
testing environment, and (4) addressing recommendations contained in a
2005 peer review. To date, none of these control activities have been
fully completed or implemented.^10 Thus, CBP does not yet have key
internal controls in place to be reasonably certain that ATS is providing
the best available information to allocate resources for targeting and
inspecting containers that are the highest risk and thus not overlook
inspecting containers that pose a high threat to the nation.
^10The Comptroller General's internal control standards state that
internal control activities help ensure that management's directives are
carried out. Further, they state that the control activities should be
effective and efficient in accomplishing the agency's control objectives.
GAO, Standards for Internal Control in the Federal Government,
GAO/AIMD-00-21.3.1, p. 11 (Washington, D.C.: November 1999).
CBP Does Not Yet Have Performance Measures to Gauge the Effectiveness of ATS in
Targeting Cargo Containers, But is Working to Develop Them
CBP does not yet have performance measures in place to help it determine
the effectiveness of ATS at targeting oceangoing cargo containers with the
highest risk of smuggled weapons of mass destruction. The Comptroller
General's internal control standards include the establishment and review
of performance measures as one example of a control activity to help an
entity ensure it is achieving effective results.^11 In July 2005, CBP
contracted with a consulting firm to develop such performance metrics. CBP
officials and personnel from this consulting firm told us that the firm's
personnel analyzed shipment information in ATS over a 2-year period to
obtain additional insights into ATS's performance and to determine whether
ATS is more effective at targeting cargo containers for terrorism related
risk than a random sampling inspection approach. CBP officials told us
that the consulting firm's personnel prepared a draft of the results of
their analyses and that, as of March 21, 2006, CBP officials are reviewing
these analyses. They also said that the consulting firm's personnel are
documenting the methodology for their analyses and related performance
measures that CBP can use in the future. CBP officials expect to receive
this methodology and the performance measures in April 2006, and told us
that they expect to begin using the measures in June 2006. CBP officials
also told us that they initially planned to have performance measures
developed by August 31, 2005, but that this process has taken longer than
expected because of delays in (1) obtaining security clearances for the
consulting firm's personnel, (2) obtaining workspace for the firm's staff,
and (3) arranging for the appropriate levels of access to CBP's
information systems.
CBP Is Not Yet Using the Results of Random Inspections to Assess ATS
Effectiveness
Currently, CBP is not using the results of its random sampling program to
assess the effectiveness of ATS. As part of its Compliance Measurement
Program, CBP plans to randomly select 30,000 shipments based on entry
information submitted by the trade community and examine those shipments
to ensure compliance with supply chain security during fiscal year
2006.^12
11See GAO/AIMD-00-21.3.1, pps. 11 and 14.
At this time, CBP is unable to compare the examination results from its
random sampling program with its ATS inspection results, as we recommended
in our 2004, report because CBP does not yet have an integrated,
comprehensive system in place to compare multiple sets of data--like
results of random inspections with results of routine ATS inspections that
were triggered by ATS scores and other operational circumstances. Such a
comparison would allow examination of if and why the outcomes of ATS's
weighted rule sets are not consistent with the expected outcomes possible
in the universe of cargo containers, based on sample projections.
Furthermore, the Comptroller General's standards for internal control
state that information should be recorded and communicated to management
and others within the entity who need it in a form that enables them to
carry out their responsibilities.^13
CBP Has Not Yet Tested the Effectiveness of ATS in Targeting Cargo Containers
for Inspection but Has Plans to Do So
Currently, CBP does not conduct simulated events (e.g., covert tests and
computer-generated simulations)--a key control activity--to test and
validate the effectiveness of ATS in targeting oceangoing cargo containers
with the highest risk of containing smuggled weapons of mass destruction
and has not yet implemented a dedicated simulation and testing
environment. Without testing and validation, CBP lacks a vital mechanism
for evaluating ATS's ability to identify high-risk containers.
In July 2005, CBP contracted with a consulting firm to obtain assistance
in the development of a computer-generated simulation and testing
environment. CBP officials report that they have the simulation
environment infrastructure in place and have processed mock manifest data
to simulate cargo linked to terrorism in the new environment. CBP is
currently reviewing the results of this test. Further, CBP officials told
us that the consulting firm is continuing to work with CBP to develop
system requirements so that officers can effectively use the simulation
environment. CBP expects to receive the consulting firm's final input for
the simulation and testing environment by June 2006. CBP officials said
that they cannot estimate when this simulation and testing environment
will be fully operational until CBP receives the consulting firm's final
product. As with the development of performance measures, CBP officials
also told us that this process has taken longer than expected because of
delays in (1) obtaining security clearances for the consulting firm's
personnel, (2) obtaining workspace for the firm's staff, and (3) arranging
for the appropriate levels of access to CBP's information systems.
^12Entry information is documentation to declare items arriving in the
United States. Entry information allows CBP to determine what is included
in a shipment. Entry information provides more detail on a container's
contents than manifest information.
^13See GAO/AIMD-00-21.3.1, p. 18.
As we reported in 2004, terrorism experts suggested that testing ATS by
covertly simulating a realistic event using probable methods of attack
would give CBP an opportunity to examine how ATS would perform in an
actual terrorist situation.^14 CBP officials told us that although they
are considering implementing this kind of practice, they do not currently
have a program in place to conduct such tests. The Director of CBP's
Management Inspections and Integrity Assurance office told us that in
mid-April 2006, his office will be presenting a proposal to the Acting
Commissioner and other senior management to request initiation of a
program to conduct testing of the CSI program that will include testing
ATS to help ensure that it is appropriately targeting the highest-risk
cargo in the CSI program.
CBP Is Working to Address Peer Review Recommendations
In response to our 2004 recommendation that CBP initiate an external peer
review of ATS, CBP contracted with a consulting firm to evaluate CBP's
targeting methodology and recommend improvements.^15 Specifically, the
contractor identified strengths of the CBP targeting methodology and
compared ATS with other targeting methodologies. However, the peer review
did not evaluate the overall effectiveness of ATS because CBP did not have
the systems in place to allow the contractor to do so.
The contractor's final report, issued in April 2005, identified many
strengths in the ATS targeting methodology, such as a very capable and
highly dedicated team and the application of a layered approach to
targeting. It also made several recommendations to improve the targeting
methodology that included control activities, such as (1) the development
of performance measures, (2) the development of a simulation and testing
environment, (3) the development and implementation of a structured plan
for continual rules enhancement, and (4) an evaluation and determination
of the effectiveness of the ATS targeting rules, several of which
reinforced the recommendations we made in our 2004 report.^16
14 GAO-04-352SU .
^15 GAO-04-352SU .
CBP issued a detailed plan, which projected delivery dates, for responding
to the recommendations made in the contractor's final report. However,
about half of these dates have not been met. For example, CBP projected
that it would have its testing and simulation environment in place by
September 30, 2005. Although CBP has been working on this effort, the
environment has not yet been implemented. As previously discussed, CBP
officials said that they cannot provide a current estimate of when this
simulation and testing environment will be fully operational.
Although CBP Strives to Refine ATS for Intelligence Information and Officer
Feedback, It Is Not Yet Positioned to Use Inspection Results
CBP strives to refine ATS to include intelligence information it acquires
and feedback it receives from its targeting officers at the seaports, but
it is not able to systematically adjust ATS for inspection results. CBP
does not have a comprehensive, integrated system in place to report
details on security inspections nationwide that will allow management to
analyze those inspections and refine ATS. CBP officials said that they are
developing a system that will allow them to do so but did not know when it
will be fully operational. CBP officials cautioned that because an
inspection does not identify any contraband or a weapon of mass
destruction or its components, it may not necessarily indicate that a
particular rule is not operating as intended. They noted that terrorist
incidents may happen infrequently, and the rule therefore might operate
only when weapons, materials, or other dangerous contraband is actually
shipped. However, without analyzing and using security inspection results
to adjust ATS, CBP is limited in refining ATS, a fact that could hinder
the effectiveness of CBP's overall targeting strategy.
CBP Adjusts ATS for Targeting Cargo Containers for Inspection Based on
Intelligence
CBP adjusts ATS's rules and weights for targeting cargo containers for
inspection in response to intelligence received on an ongoing basis. CBP's
Office of Intelligence (OINT) is responsible for acquiring, reviewing,
analyzing, and disseminating intelligence. OINT officials told us they
receive information from the intelligence community, which includes
federal agencies such as the Central Intelligence Agency and the Federal
Bureau of Investigation.^17 According to OINT officials, OINT disseminates
information to CBP's offices at the seaports to, among other things,
support these offices' targeting efforts related to cargo containers. For
example, the targeting officers may use information provided by OINT to
search ATS for information about shipments and containers. OINT officials
said they also disseminate information to CBP's senior management to
inform them about risks associated with cargo containers. CBP uses
intelligence information to refine its targeting of cargo containers for
inspection by incorporating the intelligence information into ATS to
readily identify containers whose manifest information may match or be
similar to data contained in the intelligence information.
^16 GAO-04-352SU .
CBP documentation and our observations showed that CBP headquarters
personnel incorporate intelligence information into ATS by adjusting ATS's
existing rules and weights and creating new rules and weights that result
in a higher risk score being assigned to a container whose manifest
information may match or be similar to data contained in the intelligence
information. CBP officers can also conduct queries or create lookouts in
ATS that will search all manifest data in the system to identify those
containers whose manifest information may match or be similar to data
contained in the intelligence information.^18 Once ATS identifies these
containers, CBP officers are to then designate these containers for
inspection. When CBP receives credible intelligence information that
requires immediate action, CBP officials also report that they can
initiate a special operation to address specific concerns identified in
the intelligence data. CBP officials at the six seaports we visited
reported that they sometimes receive intelligence information from local
sources such as state and local law enforcement. Officials at five of
these seaports reported that they will use such information to help them
make decisions regarding targeting efforts. Additionally, officials at
five of the six seaports we visited said that if the information they
receive has national implications, they will notify CBP headquarters
personnel, who will make a determination regarding potential adjustments
to ATS.
^17The intelligence community is a federation of executive branch agencies
and organizations that work separately and together to conduct
intelligence activities necessary for the conduct of foreign relations and
the protection of the national security of the United States.
^18A query is a search an individual officer creates to seek information
from ATS about shipments and containers based on specific criteria to
assist in the officer's targeting decisions. A lookout is a query that CBP
headquarters or officers at the seaports can create that will notify all
officers making targeting decisions when a shipment's manifest data are
similar to or match the search criteria. .
CBP Targeting Officers at the Seaports Have Provided Few Suggestions for
Adjusting ATS
In the late summer of 2005, CBP headquarters initiated a process to
formally track its targeting officers' suggestions to enhance ATS for
targeting cargo containers for inspection. Targeting officers at all six
seaports we have visited are aware of the process for providing
suggestions to CBP headquarters. According to documentation maintained by
headquarters, CBP officers at the seaports have provided few suggestions
to date.
CBP headquarters officials said that although they have received few
suggestions for modifying ATS, they do not believe this is an indication
of ATS's effectiveness. These officials stated that overall the feedback
they have received from CBP targeting officers at the seaports related to
the operation and usefulness of ATS has been positive.
We reviewed the report CBP uses to track these suggestions and found that
since it was established, CBP headquarters has received 20 suggestions for
enhancing the ATS component responsible for targeting oceangoing cargo
containers for inspection. Some of these suggestions relate to modifying
ATS's rules, while others focused on other aspects of ATS such as
enhancing the organization and presentation of ATS screens by changing the
size of an icon and the fonts or text used.
CBP Is Not Using Inspection Results to Systematically Adjust ATS, but It Is
Developing a System to Allow it to Do So
CBP is not using inspection results to systematically adjust ATS for
targeting cargo containers for inspection because CBP does not yet have a
comprehensive, integrated system in place that can report sufficient
details for analyzing inspection results. CBP officials said that although
they can analyze inspection results on a case-by-case basis to identify
opportunities to refine ATS, such as when an inspection results in a
seizure of some type of contraband, they currently do not have a reporting
mechanism in place that will allow them to view inspection results
nationwide to identify patterns for systematically adjusting ATS. CBP is
developing the Cargo Enforcement Reporting Tracking System (CERTS) to
document, among other things, all cargo examinations so that documentation
substantiating the examinations will be available for analysis by
management to adjust ATS. CBP officials said they will begin testing CERTS
in the spring of 2006. CBP officials told us that once testing of CERTS is
complete, they will be in a better position to estimate when CERTS can be
fully implemented.
CBP officials cautioned that because an inspection does not identify any
contraband or a weapon of mass destruction or its components, it may not
necessarily indicate that a particular rule is not operating as intended.
They noted that terrorist incidents may happen infrequently and the rule
therefore might operate only when weapons, materials, or other dangerous
contraband is actually shipped. However, without using inspection results
to adjust ATS, CBP may not be targeting and inspecting containers with the
highest risk of containing smuggled weapons of mass destruction.
CBP Has Taken Steps to Better Implement the Targeting Strategy at the Seaports
CBP has implemented a testing and certification process for its officers
who complete the Sea Cargo Targeting Course that should provide better
assurance of effective targeting practices. CBP has also made a good faith
effort to address longshoremen's safety concerns regarding radiation
emitted by nonintrusive inspection equipment. Nevertheless, it has not
been able to persuade one longshoremen's union to permit changes in the
procedure for staging containers to increase inspection efficiency.
CPB Has Implemented a Testing and Certification Process for Officers Who Target
Cargo Containers for Inspection
In our 2004 report, we recommended that CBP establish a testing and
certification process for CBP staff who complete the national targeting
training to provide reasonable assurance that they have sufficient
expertise to perform targeting work. ^19 CBP has implemented such a
testing and certification process.
CBP conducted two evaluations that assessed its targeting training
program--a job performance assessment and a job task analysis. With the
results of these evaluations, CBP concluded that a certification component
should be added to the training program and the Sea Cargo Targeting
Training course content should remain unchanged. CBP officials then
updated the course materials to encompass the inclusion of the
certification component. In October 2004, CBP began certifying officers
who successfully completed the Sea Cargo Targeting Training course. Since
the establishment of the testing and certification component for the Sea
Cargo Targeting Training course, CBP data indicate that it has trained and
certified 278 of its officers responsible for targeting cargo as of March
24, 2006.^20
19 GAO-04-352SU .
While CBP has conducted a job performance assessment prior to the
incorporation of a certification program for Sea Cargo Targeting Training,
it has not yet formally assessed the impact that revised training and
certification has had on officers' targeting of oceangoing cargo
containers. However, a CBP official said that CBP has recently initiated
planning efforts to begin such an evaluation and expects to complete the
evaluation in May 2006. Nevertheless, supervisory officers from five of
the six CBP offices at the seaports we visited said that the mandatory
training and certification program has been beneficial. These supervisory
officers told us that the training and certification improves the
confidence of targeters, provides the ability for officers to improve
their targeting productivity, and provides an opportunity for officers to
gain a broader perspective into the targeting environment by examining
passenger and outbound targeting.
Despite CBP Action to Address Longshoremen's Safety Concerns, Efficiency
Concerns Remain on the West Coast
In our 2004 report,^21 we discussed concerns that longshoremen had
regarding the safety of driving cargo containers through the gamma ray
imaging system, one type of nonintrusive inspection equipment used to
examine containers to detect potential contraband or weapons of mass
destruction. Because this equipment emits radiation as it takes images of
the inside of cargo containers, some longshoremen expressed concerns about
the health effects of this radiation. As a result of these safety
concerns, the longshoremen's union representing West Coast longshoremen
established a policy that prevents its members from driving containers
through the gamma ray imaging system. In response, CBP altered its
procedures at ports affected by this policy. For example, at some West
Coast ports, CBP allows longshoremen to stage cargo containers away from
the dock, in rows at port terminals, so that CBP officers can then drive
the gamma ray imaging system over a group of containers.
^20A CBP official estimated that CBP has approximately 300 officers
responsible for targeting oceangoing cargo containers. However, CBP is
currently surveying its offices to determine a more precise estimate and
will have this information available within the next month.
^21 GAO-04-352SU .
However, this procedure can be space-intensive and time-consuming compared
to the procedure utilized at East and Gulf Coast ports, whereby the gamma
ray imaging system machinery is operated by a CBP officer and parked in
place while longshoremen drive the cargo containers through the
machinery.^22 At other West Coast ports, the longshoremen get out of the
trucks after transporting the cargo containers so that CBP officials can
drive the gamma ray imaging system cargo over the container. This is also
time-consuming compared to the procedure utilized at the East and Gulf
Coast ports.
In response to our recommendation that CBP work with longshoremen to
address their safety concerns, CBP engaged in two efforts: (1)
establishing CBP's radiation threshold in accordance with the Nuclear
Regulatory Commission's (NRC) federal guidelines for public radiation
exposure and advertising this threshold to longshoremen through the
unions, and (2) working with longshoremen's unions and other maritime
organizations to develop public radiation tests on nonintrusive inspection
equipment. Officials from the West Coast union that prohibits its members
from driving through the gamma ray imaging system told us that the union
is satisfied with CBP efforts to operate the gamma ray imaging system in
an alternative format, to comply with the union's policy of receiving no
amount of man-made radiation. Despite CBP efforts to assure this union
that the amount of radiation emitted by the gamma ray imaging system is
within safe levels, a union representative told us that CBP will not
convince the union to change its policy unless it eliminates radiation
emission from inspection equipment.
- - - - -
In closing, ATS is an integral part of CBP's layered security strategy. A
well-functioning ATS is crucial to the effective screening of cargo
containers at domestic and CSI foreign ports, as well as cargo shipped by
the trade community participating in C-TPAT. While CBP is working to make
improvements to ATS, our ongoing work indicates that it is not yet in a
position to gauge the effectiveness of ATS. We are continuing to review
CBP's plans and actions to improve ATS and will report to this
subcommittee and the other requesters later this year.
^22 GAO-04-352SU .
GAO Contacts and Acknowledgments
Contact points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this statement. For further
information about this testimony, please contact me at 202-512-8777 or at
[email protected]. Debra Sebastian, Assistant Director; Chan-My J. Battcher;
Lisa L. Berardi; Wayne A. Ekblad; and Jessica A. Evans made key
contributions to this report. Additional assistance was provided by
Frances Cook, Kathryn E. Godfrey, Nancy A. Hess, Arthur L. James, Jr.,
Stanley J. Kostyla, and Vanessa R. Taylor.
Appendix I: Scope and Methodology
To address each of our objectives, we met with U.S. Customs and Border
Protection (CBP) officials in headquarters and six seaports including,
Baltimore, Charleston, Los Angeles-Long Beach, Miami, New York-Newark, and
Savannah. These seaports were selected based on the number of cargo
containers arriving at the seaport and their geographic dispersion as
reported by the U.S. Department of Transportation. At these locations, we
also observed targeting and inspection operations. Because we did not
select a random, probability sample of ports to visit, the results from
these visits cannot be generalized to ports nationwide. We also spoke with
CBP's contractor responsible for conducting CBP's peer review and
longshoremen's union representatives.
To evaluate how CBP provides assurance that the Automated Targeting System
(ATS) targets the highest-risk oceangoing cargo containers for inspection,
we reviewed CBP documentation and prior GAO work on performance measures.
Additionally, we reviewed CBP's peer review report. To gain an
understanding of CBP's random sampling program, we met with CBP officials
responsible for this program and reviewed and analyzed CBP documentation,
including procedures for examining the randomly selected shipments and
documenting the results of the inspections completed for those shipments.
We did not independently validate the reliability of CBP's targeting
results.
To assess how CBP adjusts ATS to respond to findings that occur during the
course of its operational activities, we met with CBP officials
responsible for gathering and disseminating intelligence and for
incorporating intelligence into CBP's targeting operations. Further, we
reviewed CBP policies and procedures on intelligence gathering and
disseminating as well as intelligence received and resulting changes to
ATS rules and weights. We did not assess the quality of intelligence
received or the appropriateness of adjusted rules and weights. To
determine how targeting officers' feedback and inspection results are used
to adjust ATS rules and weights, we met with CBP officials responsible for
collecting and maintaining data on suggestions provided by targeting
officers and reviewed CBP data on the suggestions received over a 7 month
period. Regarding inspection results, we reviewed CBP's policies and
procedures for documenting inspection results. Additionally, we reviewed
CBP's manuals identifying the specific details of an inspection completed
and observed officers entering inspection results into the ATS findings
module during our site visits. Further, during these visits, we discussed
how CBP offices at the seaports may use inspection results to enhance
their targeting efforts. Last, we met with CBP officials and reviewed CBP
documentation on its current and planned findings module.
To determine the status of recommendations from GAO's February 2004 report
to (1) establish a testing and certification process for CBP staff who
complete the national targeting training to provide assurance that they
have sufficient expertise to perform targeting work and (2) work with
longshoremen's unions to address fully their safety concerns so that the
noninstrusive inspection equipment can be used to conduct inspections
efficiently and safely, we reviewed and analyzed data on the number of
officers trained and certified in sea cargo targeting. We also reviewed
CBP's Sea Cargo Training Manual as well as CBP evaluations assessing the
quality of its Sea Cargo Training course. We did not assess the quality of
this training. Regarding longshoremen's union concerns, we reviewed
scientific literature related to radiation safety and the Nuclear
Regulatory Commission guidelines on radiation levels. We also spoke with
longshoremen's representatives to discuss whether CBP had addressed their
concerns since we issued our 2004 report. Last, we also met with CBP's
Radiation Safety Officer to gain a further understanding of the potential
risks associated with CBP's inspection equipment and actions he took to
address longshoremen's concerns. We did not assess the appropriateness of
radiation safety levels used by CBP.
We conducted our work from October 2005 through March 2006 in accordance
with generally accepted government auditing standards.
(440502)
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
[email protected] Automated answering system: (800) 424-5454 or (202)
512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
www.gao.gov/cgi-bin/getrpt?GAO-06-591T .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Richard Stana at (202) 512-8777 or
[email protected].
Highlights of [26]GAO-06-591T , a statement for the record to the
Permanent Subcommittee on Investigations, Committee on Homeland Security
and Governmental Affairs, United States Senate
March 30, 2006
CARGO CONTAINER INSPECTIONS
Preliminary Observations on the Status of Efforts to Improve the Automated
Targeting System
U.S. Customs and Border Protection's (CBP) Automated Targeting System
(ATS)--a computerized model that CBP officers use as a decision support
tool to help them target oceangoing cargo containers for inspection-- is
part of CBP's layered approach to securing oceangoing cargo. GAO reported
in February 2004 on challenges CBP faced in targeting oceangoing cargo
containers for inspection and testified before this subcommittee in March
2004 about the findings in that report. The report and testimony outlined
recommendations aimed at (1) better incorporating recognized modeling
practices into CBP's targeting strategy, (2) periodically adjusting the
targeting strategy to respond to findings that occur during the course of
its operation, and (3) improving implementation of the targeting strategy.
This statement for the record discusses preliminary observations from
GAO's ongoing work related to ATS and GAO's 2004 recommendations
addressing the following questions:
o What controls does CBP have in place to provide reasonable
assurance that ATS is effective at targeting oceangoing cargo
containers with the highest risk of smuggled weapons of mass
destruction?
o How does CBP systematically analyze security inspection results
and incorporate them into ATS?
o What steps has CBP taken to better implement the rest of its
targeting strategy at the seaports?
CBP has not yet put key controls in place to provide reasonable assurance
that ATS is effective at targeting oceangoing cargo containers with the
highest risk of containing smuggled weapons of mass destruction. To
provide assurance that ATS targets the highest-risk cargo containers as
intended, CBP is (1) working to develop and implement performance measures
related to the targeting of cargo containers, (2) planning to compare the
results of its random inspections with its ATS inspection results, (3)
working to develop and implement a testing and simulation environment, and
(4) addressing recommendations contained in a 2005 peer review of ATS. CBP
expects to begin using performance measures in June 2006 and enter the
final phase of software development for its testing and simulation
environment at the same time. However, to date, none of these four
initiatives has been fully implemented. Thus, CBP does not yet have key
internal controls in place to be reasonably confident that ATS is
providing the best information to allocate resources for targeting and
inspecting containers that are the highest risk and not overlook
inspecting containers that pose a threat to the nation.
CBP does not yet have a comprehensive, integrated system in place to
analyze security inspection results and incorporate them into ATS. CBP
currently adjusts ATS based on intelligence information it receives and
has initiated a process to track suggestions submitted by CBP targeting
officers at the seaports for modifying ATS. However, CBP has not yet
implemented plans to refine ATS based on findings from routine security
inspections. Without a more comprehensive feedback system, CBP is limited
in refining ATS, a fact that could hinder the overall effectiveness of the
targeting strategy.
CBP has taken steps to improve implementation of the targeting strategy at
the seaports. It has implemented a testing and certification process for
its officers who complete the Sea Cargo Targeting Course that should
provide better assurance of effective targeting practices. CBP has also
made a good faith effort to address longshoremen's safety concerns
regarding radiation emitted by nonintrusive inspection equipment by taking
actions such as working with longshoremen's unions and other maritime
organization to develop public radiation tests on the nonintrusive
inspection equipment. Nevertheless, CBP has not been able to persuade one
longshoremen's union to permit changes in the procedure for staging
containers to increase inspection efficiency at some West Coast seaports
where the union's members work.
References
Visible links
18. http://www.gao.gov/cgi-bin/getrpt?GAO-04-557T
26. http://www.gao.gov/cgi-bin/getrpt?GAO-06-591T
*** End of document. ***