Transportation Security Administration: Oversight of Explosive
Detection Systems Maintenance Contracts Can Be Strengthened
(31-JUL-06, GAO-06-795).
Mandated to screen all checked baggage by using explosive
detection systems at airports by December 31, 2003, the
Transportation Security Administration (TSA) has deployed two
types of screening equipment: explosive detection systems (EDS),
which use computer-aided tomography X-rays to recognize
explosives, and explosive trace detection (ETD) systems, which
use chemical analysis to detect explosive residues. This report
discusses (1) EDS and ETD maintenance costs, (2) factors that
played a role in these costs, and (3) the extent to which TSA
conducts oversight of maintenance contracts. GAO reviewed TSA's
contract files and processes for reviewing contractor cost and
performance data.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-795
ACCNO: A57705
TITLE: Transportation Security Administration: Oversight of
Explosive Detection Systems Maintenance Contracts Can Be
Strengthened
DATE: 07/31/2006
SUBJECT: Baggage
Contract administration
Contract oversight
Contract performance
Cost analysis
Explosives detection systems
Explosives trace detection
Federal procurement
Homeland security
Maintenance costs
Maintenance services contracts
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-795
* Results in Brief
* Background
* Results
* Conclusions
* Recommendations
* Agency Comments and Our Evaluation
* GAO Contact
* Staff Acknowledgements
* GAO's Mission
* Obtaining Copies of GAO Reports and Testimony
* Order by Mail or Phone
* To Report Fraud, Waste, and Abuse in Federal Programs
* Congressional Relations
* Public Affairs
GAO
Report to Congressional Committees
United States Government Accountability Office
July 2006 July 2006
TRANSPORTATION SECURITY ADMINISTRATION
Oversight of Explosive Detection Systems Maintenance Contracts Can Be
Strengthened
TSA EDS Maintenance Costs TSA EDS Maintenance Costs TSA EDS Maintenance
Costs TSA EDS Maintenance Costs TSA EDS Maintenance Costs TSA EDS
Maintenance Costs TSA EDS Maintenance Costs TSA EDS Maintenance Costs TSA
EDS Maintenance Costs TSA EDS Maintenance Costs TSA EDS Maintenance Costs
TSA EDS Maintenance Costs TSA EDS Maintenance Costs TSA EDS Maintenance
Costs TSA EDS Maintenance Costs TSA EDS Maintenance Costs TSA EDS
Maintenance Costs TSA EDS Maintenance Costs TSA EDS Maintenance Costs TSA
EDS Maintenance Costs TSA EDS Maintenance Costs TSA EDS Maintenance Costs
TSA EDS Maintenance Costs TSA EDS Maintenance Costs TSA EDS Maintenance
Costs TSA EDS Maintenance Costs TSA EDS Maintenance Costs TSA EDS
Maintenance Costs TSA EDS Maintenance Costs TSA EDS Maintenance Costs TSA
EDS Maintenance Costs TSA EDS Maintenance Costs TSA EDS Maintenance Costs
TSA EDS Maintenance Costs TSA EDS Maintenance Costs TSA EDS Maintenance
Costs TSA EDS Maintenance Costs TSA EDS Maintenance Costs TSA EDS
Maintenance Costs TSA EDS Maintenance Costs TSA EDS Maintenance Costs TSA
EDS Maintenance Costs TSA EDS Maintenance Costs TSA EDS Maintenance Costs
TSA EDS Maintenance Costs TSA EDS Maintenance Costs TSA EDS Maintenance
Costs TSA EDS Maintenance Costs TSA EDS Maintenance Costs TSA EDS
Maintenance Costs TSA EDS Maintenance Costs TSA EDS Maintenance Co
GAO-06-795
Contents
Letter 1
Results in Brief 3
Background 6
Results 8
Conclusions 14
Recommendations 15
Agency Comments and Our Evaluation 15
Appendix I Information for Congressional Committees 17
Appendix II Agency Comments 47
Appendix III GAO Contact and Staff Acknowledgements 51
Tables
Table 1: Number of EDS and ETD Machines and Annual Per-Machine Maintenance
Cost, Fiscal Year 2005 and Fiscal Year 2006 9
Table 2: Mean Downtime Requirement for EDS Contractors, 2005 through 2009
12
Figure
Figure 1: EDS and ETD Machines Used by TSA to Screen Checked Baggage 7
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office
Washington, DC 20548
July 31, 2006
The Honorable Judd Gregg Chairman The Honorable Robert C. Byrd Ranking
Member Subcommittee on Homeland Security Committee on Appropriations
United States Senate
The Honorable Harold Rogers Chairman The Honorable Martin Olav Sabo
Ranking Member Subcommittee on Homeland Security Committee on
Appropriations House of Representatives
After the terrorist attacks of September 11, 2001, which highlighted the
vulnerability of U.S. aircraft to acts of terrorism, Congress passed and
the President signed into law the Aviation and Transportation Security Act
(ATSA), mandating, among other things, that all checked baggage at U.S.
airports be screened using explosive detection systems by December 31,
2002.1 To meet this requirement, the Transportation Security
Administration (TSA) deployed two types of equipment to screen checked
baggage for explosives: (1) explosive detection systems (EDS) that use
specialized X-rays to detect characteristics of explosives that may be
contained in baggage as it moves along a conveyor belt and (2) explosive
trace detection (ETD) machines, whereby an individual (i.e., a baggage
screener, or transportation security officer) swabs baggage and then
inserts the swab into the ETD machine, which in turn can detect chemical
residues that may indicate the presence of explosives within a bag.2
1Aviation and Transportation Security Act, Pub. L. No. 107-71, S: 110(b),
115 Stat. 597, 615 (2001). Section 425 of the subsequently enacted
Homeland Security Act of 2002, Pub. L. No. 107-296, 116 Stat. 2135,
2185-86, in effect, extended this mandate to December 31, 2003. See 49
U.S.C. S: 44901(d).
Pursuant to ATSA, TSA assumed operational responsibility for conducting
the screening of checked baggage, which includes the procurement,
installation, and maintenance of EDS and ETD machines. By the end of
fiscal year 2006, TSA will have deployed over 1,400 EDS and 6,600 ETD
machines at baggage-screening locations in over 400 airports nationwide,
according to TSA budget documents. TSA has used contractors to perform
preventative and corrective maintenance on these EDS and ETD machines.
House Conference Report 109-241, which accompanied the Department of
Homeland Security Appropriations Act, 2006 (Public Law 109-90) directed
that we report on the reasons for cost increases in maintaining TSA's
explosive detection systems, including TSA's contracting practices that
may have affected cost increases.3 With regard to this requirement, on
April 19, 2006, we provided you with information on the results of our
review (see app. I), which has been updated as appropriate in this report.
Subsequently, in May 2006, the House Appropriations Committee stated in
its report accompanying the Department of Homeland Security Appropriations
Bill for fiscal year 2007 its long-standing concerns about the increasing
costs for EDS and ETD maintenance and specifically, TSA's recovery of any
excess award fees from a previous EDS and ETD contractor.4 Further, the
report stated that TSA should submit a report to the House Appropriations
Committee on any actions it has taken to collect excessive award fees, how
much has been received to date, and specific plans to obligate these
collections.
This report addresses the following questions:
o What are the historical, current, and projected costs for the
maintenance of EDS and ETD machines?
o What factors played a role in EDS and ETD maintenance costs,
and what factors could affect future costs?
o What has TSA done to control EDS and ETD maintenance costs?
o To what extent does TSA oversee the performance of EDS and ETD
maintenance contractors?
2To satisfy the ATSA mandate, TSA interpreted the term explosive detection
system to include both explosive detection systems (EDS) and explosive
trace detection (ETD) machines.
3 See H.R. Conf. Rep. No. 109-241, at 52 (2005).
4Department of Homeland Security Appropriations Act, 2007, H.R. 5441,
109th Cong. (2006); H.R. Rep. No. 109-476, tit. II, at 49-50 (2006).
To address our objectives, we reviewed TSA contract files and related cost
data, TSA processes for reviewing contract performance data, budget
documents for fiscal years 2003 through 2007, acquisition and strategic
plans, a Department of Homeland Security's Office of the Inspector General
(DHS OIG) report,5 GAO standards for internal controls,6 and our previous
work on TSA's acquisition function.7 We interviewed TSA headquarters
officials, DHS OIG officials, and EDS and ETD contractor representatives.
For purposes of our review, we focused on amounts obligated under
contracts to maintain the machines. We did not review TSA's negotiations
for maintenance services or the process for awarding contracts. Nor did we
assess other direct or indirect costs that may have been related to TSA or
DHS employees engaged in contract administration or other related items.
We performed our work from January 2006 through July 2006 in accordance
with generally accepted government auditing standards.
Results in Brief
According to TSA budget documents, TSA obligated almost $470 million from
fiscal year 2002 through fiscal year 2005 for EDS and ETD maintenance. In
fiscal year 2006, TSA estimates it will spend $199 million and has
projected it will spend $234 million in fiscal year 2007.8 TSA did not
provide us with projections of EDS and ETD maintenance costs beyond 2007.
TSA officials told us that future EDS and ETD maintenance costs will be
influenced by the number, type, quantity, and locations of machines
necessary to support system configurations at airports and on decisions
related to the deployment of new technologies and the refurbishment of
existing equipment, among other things. The current contracts have
negotiated maintenance prices per machine through March 2009, if TSA
decides to exercise option years in the contracts.
5U.S. Department of Homeland Security, Office of Inspector General,
Evaluation of TSA's Contract for the Installation and Maintenance of
Explosive Detection Equipment at United States Airports, OIG-04-44
(Washington, D.C.: September 2004).
6GAO, Standards for Internal Control in the Federal Government,
GAO/AIMD-00-21 .3.1 (Washington, D.C.: November 1999).
7GAO, Transportation Security Administration: High-Level Attention Needed
to Strengthen Acquisition Function, GAO-04-544 (Washington, D.C.: May
2004).
Different factors played a role in EDS and ETD maintenance costs.
According to a September 2004 DHS OIG report,9 TSA did not follow sound
contracting practices in administering the contract with Boeing Service
Company (Boeing) that was primarily for the installation and maintenance
of EDS and ETD machines from June 2002 through March 2005. According to
DHS OIG officials, TSA's failure to control costs under the Boeing
contract contributed to increases in maintenance costs. Among other
things, the DHS OIG report stated that TSA has paid provisional award fees
totaling $44 million through December 2003 without any evaluation of
Boeing's performance. In response to the DHS OIG, TSA agreed to recover
any excessive award fees paid to Boeing, if TSA determined that such fees
were not warranted. In responding to our draft report, DHS told us that
TSA and Boeing have reached an agreement in principle on this matter and
the documentation is in the approval process with closure anticipated in
July 2006. Moreover, TSA did not develop life-cycle cost estimates before
any of the maintenance contracts were executed, and, as a result, TSA did
not have a sound estimate of maintenance costs for all the years the
machines are expected to be in operation. DHS also stated in its comments
on our draft report that a TSA contractor expects to complete a prototype
life-cycle cost model by September 2006 and that TSA anticipated that the
EDS model would be completed 12 months after the prototype was approved.
Without such an analysis, TSA may not be identifying cost efficiencies and
making informed procurement decisions on future purchases of EDS and ETD
machines and maintenance contracts.
8 Amounts attributed to maintenance also include utility costs, such as
electricity, that generally amount to less than 10 percent of the overall
amount allocated for maintenance each fiscal year, according to TSA
officials. Further, TSA officials told us they could provide us with
amounts obligated for fiscal years 2005 and 2006, but could not provide us
with the amounts expended for this time period.
9 OIG-04-44.
TSA has taken several actions to control EDS and ETD maintenance costs,
such as entering into firm-fixed-price contracts starting in March 2005,
which have certain advantages to the government because price certainty is
guaranteed. Also, TSA included several performance requirements in the
current contracts, including metrics related to machine reliability,
maintainability, and availability, and specific cost data related to
maintenance and repair, and required monthly performance reviews. For EDS
contractors, TSA also incorporated provisions to specify that the full
agreed price will be paid only if mean downtime requirements10 are met.
Although TSA has policies for monitoring contracts, TSA officials provided
no evidence that they are reviewing required contractor-submitted
performance data, such a mean downtime data. TSA officials told GAO that
they perform such reviews, but do not document their activities because
there are not TSA policies and procedures requiring them to do so. As a
result, without adequate documentation, TSA does not have reasonable
assurance that contractors are performing as required and that full
payment is justified based on meeting mean downtime requirements.
We are recommending that the Secretary of Homeland Security direct TSA to
establish a timeline to close out the Boeing contract and report to
congressional committees on its actions to recover any excessive fees
awarded to Boeing; establish a timeline for completing life-cycle cost
models for EDS, which TSA recently began; and revise policies and
procedures to require documentation of the monitoring of EDS and ETD
maintenance contracts to provide reasonable assurance that contractor
maintenance cost data and performance data are recorded and reported in
accordance with TSA contractual requirements and that self-reported
contractor mean downtime data are valid, reliable, and justify the full
payment of the contract amount.
We provided a draft of this report to DHS for review. DHS, in its written
comments, concurred with our findings and recommendations and described
actions that it has initiated or plans to take to address the issues
identified. For a reprint of DHS's comments, see appendix II.
10Mean downtime is a performance requirement in EDS and ETD maintenance
contracts. Mean downtime is calculated by the number of hours a machine is
out of service in a month divided by the number of times that machine is
out of service per month. For example, if a machine has a total downtime
of 50 hours per month and is out of service 5 times in that month, the MDT
would be equal to 50 divided by 5, which is 10 hours.
Background
With the passage of the Aviation and Transportation Security Act (ATSA) in
November 2001, TSA assumed from the Federal Aviation Administration (FAA)
the majority of the responsibility for civil aviation security, including
the commercial aviation system.11 ATSA required that TSA screen 100
percent of checked baggage using explosive detection systems by December
31, 2002. As it became apparent that certain airports would not meet the
December 2002 deadline, the Homeland Security Act of 2002 in effect
extended the deadline to December 31, 2003, for noncompliant airports.12
Under ATSA, TSA is responsible for the procurement, installation, and
maintenance of explosive detection systems used to screen checked baggage
for explosives. Airport operators and air carriers continued to be
responsible for processing and transporting passengers' checked baggage
from the check-in counter to the airplane.
Explosive detection systems include EDS and ETD machines (fig. 1). EDS
uses computer-aided tomography X-rays adapted from the medical field to
automatically recognize the characteristic signatures of threat
explosives. By taking the equivalent of hundreds of X-ray pictures of a
bag from different angles, EDS examines the objects inside of the baggage
to identify characteristic signatures of threat explosives. TSA has
certified, procured, and deployed EDS manufactured by three companies-L-3
Communications Security and Detection Systems (L-3); General Electric
InVision, Inc.13 (GE InVision); and Reveal Imaging Technologies, Inc.
(Reveal). In general, EDS is used for checked baggage screening. ETD
machines work by detecting vapors and residues of explosives. Human
operators collect samples by rubbing bags with swabs, which are then
chemically analyzed in the ETD machine to identify any traces of explosive
materials. ETD machines are used for both checked baggage and passenger
carry-on baggage screening. TSA has certified, procured, and deployed ETD
machines from three manufacturers, Thermo Electron Corporation, Smiths
Detection, and General Electric Company.
11Pub. L. No. 107-71, S: 101, 115 Stat. at 597. See 49 U.S.C. S: 114(d).
12Pub. L. No. 107-296, S: 425, 116 Stat. at 2185-86. See 49 U.S.C. S:
44901(d).
13 General Electric InVision, Inc. is an entity of General Electric
Company.
Figure 1: EDS and ETD Machines Used by TSA to Screen Checked Baggage
TSA's EDS and ETD maintenance contracts provide for preventative and
corrective maintenance. Preventative maintenance includes scheduled
activities, such as changing filters or cleaning brushes, to increase
machine reliability and are performed monthly, quarterly, or yearly based
on the contractors' maintenance schedules. Corrective maintenance includes
actions performed to restore machines to operating condition after
failure, such as repairing the conveyer belt mechanism after a bag jams
the machine. TSA is responsible for EDS and ETD maintenance costs after
warranties on the machines expire.14
From June 2002 through March 2005, Boeing was the prime contractor
primarily for the installation and maintenance of EDS and ETD machines at
over 400 U.S. airports. TSA officials stated that the Boeing contract was
awarded at a time when TSA was a new agency with many demands and
extremely tight schedules for meeting numerous congressional mandates
related to passenger and checked baggage screening. The cost reimbursement
contract15 entered into with Boeing had been competitively bid and
contained renewable options through 2007. Boeing subcontracted for EDS
maintenance through firm-fixed-price contracts16 with the original EDS
manufacturers, GE InVision and L-3, which performed the maintenance on
their respective EDS. Boeing subcontracted for ETD maintenance through a
firm-fixed-price contract with Siemens. Consistent with language in the
fiscal year 2005 House Appropriations Committee report and due to TSA's
acknowledgment of Boeing's failure to control costs, TSA received DHS
authorization to negotiate new EDS and ETD maintenance contracts in
January 2005.
14A TSA official told us that typical EDS warranties are one year and ETD
warranties are for 2 years.
15 Cost-reimbursement contracts provide for payment of allowable incurred
costs, to the extent prescribed in the contract. These contracts establish
an estimate of total cost for the purpose of obligating funds and
establishing a ceiling that the contractor may not exceed (except at its
own risk) without the approval of the contracting officer.
16 Firm-fixed-price contracts provide for a price that is not subject to
any adjustment on the basis of the contractor's cost experience in
performing the contract. This contract type places upon the contractor
maximum risk and full responsibility for all costs and resulting profit
and loss. It provides maximum incentive for the contractor to control
costs and perform effectively and imposes a minimum administrative burden
upon the contracting parties.
In March 2005, TSA signed firm-fixed-price contracts for EDS and ETD
maintenance. TSA awarded a competitively bid contract to Siemens to
provide maintenance for ETD machines. According to TSA, it negotiated sole
source contracts with L-3 and GE InVision for maintaining their respective
EDS because they are the original equipment manufacturers and owners of
the intellectual property rights of their respective EDS. In September
2005, TSA awarded a competitively bid firm-fixed-price contract to Reveal
for both the procurement and maintenance of a reduced size EDS.
Results
TSA obligated almost $470 million from fiscal year 2002 through fiscal
year 2005 for EDS and ETD maintenance, according to TSA budget documents.
In fiscal year 2006, TSA estimates it will spend $199 million and has
projected it will spend $234 million in fiscal year 2007.17 According to
TSA officials, in fiscal year 2004, TSA requested and received approval to
reprogram about $32 million from another account to EDS/ETD maintenance
due to higher levels of maintenance costs than expected. Similarly, in
fiscal year 2005, TSA requested and received approval to reprogram $25
million to fund the L-3 contract and to close out the Boeing contract.18
TSA was not able to provide us with data on the maintenance cost per
machine before fiscal year 2005 because, according to TSA officials, TSA's
previous contract with Boeing to maintain EDS and ETD machines was not
structured to capture these data. Table 1 identifies the maintenance
costs19 by type of EDS and ETD machine for fiscal years 2005 and 2006.
17Amounts attributed to maintenance also include utility costs, such as
electricity, that generally amount to less than 10 percent of the overall
amount allocated for maintenance each fiscal year, according to TSA
officials. Further, TSA officials told us they could provide us with
amounts obligated for fiscal years 2005 and 2006, but could not provide us
with the amounts expended for this time period.
18As of April 2006, the Boeing contract had yet to be closed out,
according to TSA officials.
Table 1: Number of EDS and ETD Machines and Annual Per-Machine Maintenance
Cost, Fiscal Year 2005 and Fiscal Year 2006
FY 2005a FY 2006
Number of Cost per Number of Cost per
Type of machine machines unit machines unit
EDS
GE CTX 2500 140 $61,587 151 $ 63,590
GE CTX 5500 512 71,549 547 73,876
GE CTX 9000 172 93,286 231 96,320
L-3 ex6000 508 97,837 550 101,000
Reveal CT-80b n/a n/a 16 n/a
ETD
Smiths Ionscan 241 10,525 336 10,974
400A
Smiths Ionscan 5 10,525 6 10,974
400AE
Smiths Ionscan 3,038 8,580 3,035 8,946
400B
Thermo EGIS 3000 2 12,899 2 13,526
Thermo EGIS II 425 13,134 545 13,695
GE Iontrack 2,302 $ 7,727 2,322 $ 8,057
Itemiser-W
Source: TSA.
NOTE: Maintenance costs represent the negotiated prices in the maintenance
contracts for EDS and ETD machines.
aFiscal year 2005 per-machine maintenance costs were in effect from March
through September 2005. TSA could not provide per-machine maintenance
costs before March 2005.
bReveal's EDS machines were installed in fiscal year 2006 and were still
under the manufacturer's warranty.
TSA did not provide us with projections of EDS and ETD maintenance costs
beyond 2007. TSA officials told us that future costs will be influenced by
the number, type, quantity, and locations of machines necessary to support
system configurations at airports, such as the extent to which EDS are
integrated with airport baggage conveyor systems or are operated in
stand-alone modes. Further, TSA officials told us that future EDS and ETD
maintenance costs are dependent on decisions related to the deployment of
new technologies and the refurbishment of existing equipment, among other
things. The current contracts would have negotiated maintenance prices per
machine through March 2009, if TSA decides to exercise option years in the
contracts.
19Represents the negotiated prices for the maintenance of EDS and ETD
machines.
We identified different factors that have played a role in costs to date
and that will influence future maintenance costs for EDS and ETD machines.
According to a September 2004 DHS OIG report, TSA did not follow sound
contracting practices in administering the Boeing contract, which was
primarily for the installation and maintenance of EDS and ETD machines.20
According to DHS OIG officials, TSA's failure to control costs under the
Boeing contract, including the lack of sound contracting practices,
contributed to increases in maintenance costs. Among other things, the DHS
OIG report stated that TSA had paid provisional award fees totaling $44
million through December 2003 without any evaluation of Boeing's
performance.21 In response to the DHS OIG, TSA agreed to recover any
excessive award fees paid to Boeing, if TSA determined that such fees were
not warranted. In commenting on our draft report in July 2006, DHS stated
that TSA has conducted a contract reconciliation process to ensure that no
fees would be paid on costs that exceeded the target due to poor
contractor performance. Further, DHS stated that TSA and Boeing had
reached an agreement in principle on this matter and that the
documentation was in the approval process with closure anticipated in July
2006. In its report accompanying the DHS Appropriations Bill for fiscal
year 2007, the House Appropriations Committee stated its need for a report
from TSA on any actions it has taken to collect excessive award fees, how
much of the fees have been received to date, and specific plans to
obligate these collections and cited TSA's plans to use any cost
recoveries to purchase and install additional EDS. These actions were
based on the committee's long-standing concerns about the increasing costs
for EDS and ETD maintenance.22 In addition to matters related to the
Boeing contract, TSA officials stated that another factor contributing to
cost increases were the larger than expected number of machines that came
out of warranty and their related maintenance costs. According to TSA
officials, they were not able to determine the cost impact of these
additional machines because the Boeing contract was not structured to
provide maintenance costs for individual machines.
20 OIG-04-44.
21 GAO has identified similar instances of agencies' failure to properly
use incentives in making award fees. See GAO, Defense Acquisitions: DOD
Has Paid Billions in Award and Incentive Fees Regardless of Acquisition
Outcomes, GAO-06-66 (Washington D.C.: December 2005).
22 See H.R. Rep. No. 109-479, at 49-50.
With regard to future EDS and ETD maintenance costs under firm-fixed-price
contracts, maintenance costs per machine will increase primarily by an
annual escalation factor in the contracts that takes into account the
employment cost index and the consumer price index,23 if TSA decides to
exercise contract options. In addition, future maintenance costs may be
affected by a range of factors, including the number of machines deployed
and out of warranty, conditions under which machines operate, contractor
performance requirements, the emergence of new technologies or improved
equipment, and alternative screening strategies. Lastly, life-cycle cost
estimates were not developed for the Boeing, Siemens, L-3, and GE InVision
contracts before the maintenance contracts were executed, and, as a
result, TSA did not have a sound estimate of maintenance costs for all the
years the machines are expected to be in operation. In August 2005, TSA
hired a contractor to define parameters for a life-cycle cost model, among
other things. This contract states that TSA and the contractor will work
together to ensure that the full scope of work is planned, coordinated,
and executed according to approved schedules. In commenting on our draft
report in July 2006, DHS stated that the TSA contractor estimated
completing a prototype life-cycle cost model by September 2006. Further,
DHS stated that TSA's evaluation of the prototype would begin immediately
upon delivery and that full implementation of an EDS life-cycle cost model
would be completed within 12 months after the prototype had been approved.
According to a TSA official, the life-cycle cost model would be useful in
determining machine reliability and maintainability and to inform future
contract negotiations, such as when to replace a machine versus continuing
to repair it.
We identified several actions TSA has taken to control EDS and ETD
maintenance costs. First, TSA entered into firm-fixed-price contracts
starting in March 2005 with maintenance contractors, which offer TSA
certain advantages over cost reimbursement contracts because price
certainty is guaranteed for up to 5 years if TSA exercises options to
2009. Also, TSA included several performance requirements in the Siemens,
L-3, GE InVision, and Reveal contracts, including the collection of
metrics related to machine reliability, maintainability, and
availability24 and required specific cost data related to maintenance and
repair. TSA officials told us that these data will assist them in
monitoring the contractor performance as well as informing future contract
negotiations for equipment and maintenance. These contracts also stipulate
that maintenance contractors meet monthly with TSA to review all pertinent
technical schedules and cost aspects of contracts. TSA also incorporated
provisions in the L-3 and GE InVision contracts to specify that the agreed
price for maintaining EDS would be paid only if the contractor performs
within specified mean downtime (MDT) requirements.25 Contractors submit
monthly invoices for 95 percent of the negotiated contract price for the
month and then submit a MDT report to justify the additional 5 percent.
Consequently, if the contractor fails to fulfill the MDT requirements, it
is penalized 5 percent of the negotiated monthly maintenance price. As of
February 2006, neither GE InVision nor L-3 had been penalized for missing
their MDT requirements. The allowable MDT is lowered from 2005 to
subsequent renewable years in the contract, as shown in table 2.
23 For EDS contracts, future labor and material costs could not be
determined, so TSA negotiated an escalation factor to be used to determine
pricing for the contract option years. For the ETD contracts, TSA
determined after a review of cost data, that it would apply a 4 percent
escalation factor to prices in the contract option years. The employment
cost index is a measure of the change in the cost of labor, free from the
influence of employment shifts among occupations and industries. The
consumer price index is a measure of the average change in prices over
time of goods and services purchased by households.
Table 2: Mean Downtime Requirement for EDS Contractors, 2005 through 2009
2005 2006 2007 2008 2009
24 hours 20 hours 18 hours 14 hours 12 hours
Source: TSA.
24 Includes metrics such as mean time between failures (generally the
total time a machine is available to perform its required mission divided
by the number of failures over a given period of time) and operational
availability (generally the percentage of time, during operational hours,
that a machine is available to perform its required mission). Such
reliability, maintainability, and availability data are standard and
appropriate performance requirements for maintenance contracts.
25As noted in footnote 10, mean downtime is a performance requirement in
EDS and ETD maintenance contracts. Mean downtime is calculated by the
number of hours a machine is out of service in a month divided by the
number of times that machine is out of service per month. For example, if
a machine has a total downtime of 50 hours per month and is out of service
5 times in that month, the MDT would be equal to 50 divided by 5, which is
10 hours.
With regard to TSA's oversight of EDS and ETD contractor performance,
TSA's acquisition policies26 and GAO standards for internal controls27
call for documenting transactions and other significant events, such as
monitoring contractor activities. The failure of TSA to develop internal
controls and performance measures has been recognized by other GAO28 and
DHS OIG reviews.29 TSA has policies and procedures for monitoring its
contracts and has included contractor performance requirements in the
current EDS and ETD maintenance contracts. However, TSA officials provided
no evidence that they are reviewing maintenance cost data provided by the
contractor because they are not required to document such activities. For
example, even though TSA officials told us that they are reviewing
required contractor data, including actual maintenance costs related to
labor hours and costs associated with replacing and shipping machine
parts, they did not have any documentation to support this. TSA officials
told us that they have begun to capture these data to assist them in any
future contract negotiations.
Further, TSA officials provided no evidence that performance data for
corrective and preventative maintenance required under contracts are being
reviewed. TSA officials told us that they perform such reviews, but do not
document their activities since there are no TSA policies or procedures
requiring them to do so. Therefore, TSA could not provide assurance that
contractors are complying with contract performance requirements. For
example, although TSA documents monthly meetings with contractors to
discuss performance data, TSA officials did not provide evidence that they
independently determine the reliability and validity of data required by
the contracts, such as mean time between failures and mean time to repair,
which are important to making informed decisions about future purchases of
EDS and ETD equipment and their associated maintenance costs. Further, TSA
officials provided no evidence that they ensure that contractors are
performing scheduled preventative maintenance. TSA officials told us that
they review the contractor-submitted data to determine whether contractors
are fulfilling their contractual obligations, but do not document their
activities because there are no TSA policies or procedures to require such
documentation.
26 TSA uses the Federal Aviation Administration (FAA) Acquisition
Management System which, as adopted by TSA, requires contractors to act on
contractual quality assurance commitments and ensure that government
quality and reliability needs are met (FAA Acquisition Management Policy
3.10.4.2).
27 GAO/AIMD-00-21 .3.1.
28 GAO has identified contract surveillance issues in other agencies, such
as the Department of Defense. See GAO, Contract Management: Opportunities
to Improve Surveillance on Department of Defense Service Contracts,
GAO-05-274 (Washington, D.C.: March 2005).
29 OIG-04-44.
Additionally, for EDS contracts with possible financial penalties, TSA
officials told us that they review contractor-submitted mean downtime data
on a monthly basis to determine the reliability and validity of the data
and to determine whether contractors are meeting contract provisions or
should be penalized. However, TSA officials do not document these
activities because there are no TSA policies or procedures requiring them
to do so. As a result, without adequate documentation, there is no
assurance as to whether or not contractors are meeting contract provisions
or that TSA has ensured that it is making appropriate payments for
services provided.
Conclusions
The cost of maintaining checked baggage-screening equipment has increased
as more EDS and ETD machines have been deployed and warranties expire.
TSA's move in March 2005 to firm-fixed-price maintenance contracts for EDS
and ETD maintenance was advantageous to the government in that it helps
control present and future maintenance costs. Firm-fixed-price contracts
also help ensure price certainty and therefore are more predictable.
However, unresolved issues remain with the past contractor, specifically
fees awarded to former contractor Boeing that may have been excessive due
to a lack of timely evaluation of the contractor's performance. The House
Appropriations Committee has expressed concern about these unresolved
issues; specifically, what actions TSA has taken to recover these
excessive fees, and the extent to which any collections might impact
future TSA obligations. Closing out the Boeing contract is essential to
resolving these issues. In responding to our draft report, DHS stated that
the completion of an EDS life-cycle cost is over a year away. Absent such
a life-cycle cost model, TSA may not be identifying cost efficiencies and
making informed procurement decisions regarding the future purchase of EDS
and ETD machines and maintenance contracts. Further, TSA must provide
evidence of its reviews and analyses of contractor-submitted data and
perform analyses of contractor data to determine the reliability and
validity of the data and to provide assurance of compliance with contract
performance requirements and internal control standards. Without stronger
oversight, TSA will not have reasonable assurance that contractors are
performing as required and that full payment is justified based on meeting
mean downtime requirements.
Recommendations
To help improve TSA's management of EDS and ETD maintenance costs and
strengthen oversight of contract performance, we recommend that the
Secretary of Homeland Security instruct the Assistant Secretary,
Transportation Security Administration, to take the following three
actions:
o establish a timeline to complete its evaluation and close out
the Boeing contract and report to congressional appropriations
committees on its actions, including any necessary analysis, to
address the Department of Homeland Security Office of Inspector
General's recommendation to recover any excessive fees awarded to
Boeing Service Company;
o establish a timeline for completing life-cycle cost models for
EDS, which TSA recently began; and
o revise policies and procedures to require documentation of the
monitoring of EDS and ETD maintenance contracts to provide
reasonable assurance that
o contractor maintenance cost data and performance
data are recorded and reported in accordance with TSA
contractual requirements and
o self-reported contractor mean downtime data are
valid, reliable, and justify the full payment of the
contract amount.
Agency Comments and Our Evaluation
We provided a draft of this report to DHS for its review and comment. On
July 24, 2006, we received written comments on the draft report. DHS, in
its written comments, concurred with our findings and recommendations, and
agreed that efforts to implement these recommendations are essential to a
successful explosive detection systems program. DHS stated that it has
initiated efforts to improve TSA's management of EDS and ETD maintenance
costs and strengthen oversight of contract performance. Regarding our
recommendation that TSA establish a timeline to close out the Boeing
contact and report to congressional committees on its actions to recover
any excessive fees, DHS stated that TSA has conducted a contract
reconciliation process to ensure that no fees would be paid on costs that
exceeded the target due to poor contractor performance and that Boeing and
TSA have reached an agreement in principle on this matter and the
documentation is in the approval process with closure anticipated in July
2006. Regarding our recommendation to establish a timeline for completing
the EDS life-cycle cost model, DHS stated that TSA expects to complete its
prototype evaluation in September 2006 and that the EDS life-cycle cost
model will be completed 12 months after the prototype has been approved.
Regarding our recommendation to revise TSA policies and procedures to
require documentation of its monitoring of EDS and ETD maintenance
contracts, DHS stated that a TSA contractor is developing automated tools
to perform multiple analyses of contractor-submitted data that DHS said
would allow TSA to accurately and efficiently certify the contractors'
performance against their contractual requirements and would allow TSA to
independently validate and verify maintenance and cost data. The
department's comments are reprinted in appendix II.
We will send copies of this report to the Secretary of Homeland Security
and the Assistant Secretary, Transportation Security Administration, and
interested congressional committees. We will also make copies available to
others upon request. In addition, the report will be available at no
charge on GAO's Web site at http://www.gao.gov . If you or your staffs
have any questions or need additional information, please contact me at
(202) 512-8777 or [email protected] . Contact points for our Offices of
Congressional Relations and Public Affairs may be found on the last page
of this report. Key contributors to this report are acknowledged in
appendix III.
Cathleen A. Berrick Director, Homeland Security and Justice Issues
Appendix I: Information for Congressional Committees Appendix I:
Information for Congressional Committees
Appendix II: Agency Comments Appendix II: Agency Comments
Appendix III: GAOA Appendix III: GAO Contact and Staff Acknowledgements
GAO Contact
Cathleen A. Berrick (202) 512-8777
Staff Acknowledgements
In addition to the individual names above, Charles Bausell, R. Rochelle
Burns, Glenn Davis, Katherine Davis, Michele Fejfar, Richard Hung, Nancy
Kawahara, Dawn Locke, Thomas Lombardi, Robert Martin, and William Woods.
(440482)
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts newly
released reports, testimony, and correspondence on its Web site. To have
GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: [email protected]
Automated answering system: (800) 424-5454 or (202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202) 512-4800 U.S.
Government Accountability Office, 441 G Street NW, Room 7149 Washington,
D.C. 20548
www.gao.gov/cgi-bin/getrpt? GAO-06-795 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Cathleen A. Berrick at (202) 512-8777 or
[email protected].
Highlights of GAO-06-795 , a report to Congressional Committees
July 2006
TRANSPORTATION SECURITY ADMINISTRATION
Oversight of Explosive Detection Systems Maintenance Contracts Can Be
Strengthened
Mandated to screen all checked baggage by using explosive detection
systems at airports by December 31, 2003, the Transportation Security
Administration (TSA) has deployed two types of screening equipment:
explosive detection systems (EDS), which use computer-aided tomography
X-rays to recognize explosives, and explosive trace detection (ETD)
systems, which use chemical analysis to detect explosive residues. This
report discusses (1) EDS and ETD maintenance costs, (2) factors that
played a role in these costs, and (3) the extent to which TSA conducts
oversight of maintenance contracts. GAO reviewed TSA's contract files and
processes for reviewing contractor cost and performance data.
What GAO Recommends
GAO recommends that the Secretary of Homeland Security direct TSA to (1)
establish a timeline to close out the contract with Boeing Service Company
(Boeing) and report to congressional committees on actions to recover any
excessive fees awarded to Boeing, (2) establish a timeline to complete the
EDS life-cycle model, and (3) revise policies to require documentation for
monitoring EDS and ETD maintenance contracts. The Department of Homeland
Security concurred with GAO's recommendations and described actions TSA
had taken or planned to take to implement them.
TSA obligated almost $470 million from fiscal years 2002 through 2005 for
EDS and ETD maintenance, according to TSA budget documents. In fiscal year
2006, TSA estimates it will spend $199 million and has projected it will
spend $234 million in fiscal year 2007. TSA was not able to provide GAO
with data on the maintenance cost per machine before fiscal year 2005
because, according to TSA officials, its previous contract with Boeing to
install and maintain EDS and ETD machines was not structured to capture
these data.
Several factors have played a role in EDS and ETD maintenance costs.
According to a September 2004 Department of Homeland Security's Office of
Inspector General report, TSA did not follow sound contracting practices
in administering the contract with Boeing, and TSA paid provisional award
fees totaling $44 million through December 2003 without any evaluation of
Boeing's performance. TSA agreed to recover any excessive award fees paid
to Boeing if TSA determined that such fees were not warranted. In
responding to our draft report, DHS told us that TSA and Boeing had
reached an agreement in principle on this matter and that documentation
was in the approval process with closure anticipated in July 2006.
Moreover, TSA did not develop life-cycle cost models before any of the
maintenance contracts were executed and, as a result, TSA does not have a
sound estimate of maintenance costs for all the years the machines are
expected to be in operation. DHS also stated in its comments on our draft
report that a TSA contractor expected to complete a prototype life-cycle
cost model by September 2006 and that TSA anticipated that the EDS model
would be completed 12 months after the prototype was approved. Without
such an analysis, TSA may not be identifying cost efficiencies and making
informed procurement decisions on future purchases of EDS and ETD machines
and maintenance contracts. TSA has taken actions to control costs, such as
entering into firm-fixed-price contracts for maintenance starting in March
2005, which have advantages to the government because price certainty is
guaranteed. Further, TSA incorporated standard performance requirements in
the contracts including metrics related to machine reliability and monthly
performance reviews. For EDS contractors, TSA has specified that the full
agreed price would be paid only if mean downtime (i.e., the number of
hours a machine is out of service in a month divided by the number of
times that machine is out of service per month) requirements are met.
Although TSA has policies for monitoring contracts, TSA officials provided
no evidence that they are reviewing required contractor-submitted
performance data, such as mean downtime data. TSA officials told GAO that
they perform such reviews, but do not document their activities because
there are no TSA policies and procedures requiring them to do so. As a
result, without adequate documentation, TSA does not have reasonable
assurance that contractors are performing as required and that full
payment is justified based on meeting mean downtime requirements.
*** End of document. ***