Aviation Security: TSA's Change to Its Prohibited Items List Has
Not Resulted in Any Reported Security Incidents, but the Impact
of the Change on Screening Operations Is Inconclusive (25-APR-07,
GAO-07-623R).
The alleged August 2006 terrorist plot to detonate liquid
explosives onboard multiple commercial aircraft bound for the
United States from the United Kingdom has highlighted both the
continued importance of securing the civil aviation system and
the potential that improvised explosive devices (IED) may be
smuggled onboard passenger aircraft. The Transportation Security
Administration (TSA) has primary responsibility for ensuring the
security of civil aviation, which includes the safety of
passengers and flight crew. One measure TSA uses to protect the
aviation system is prohibiting individuals from carrying items
that it determines to be a threat to the aircraft and its
passengers into an airport sterile area or onboard an aircraft
either in their carry-on bag or on their person. To implement
this measure, TSA maintains a prohibited items list that informs
both the Transportation Security Officers (TSO) who conduct
passenger screening and the traveling public of items that will
not be allowed into an airport sterile area or onboard an
aircraft. In December 2005, TSA revised its prohibited items list
to allow passengers to carry: (1) metal scissors with pointed
tips and a blade 4 inches or less in length as measured from the
fulcrum; and (2) tools--such as pliers, screwdrivers, and
wrenches--7 inches or less in length (excluding crowbars, drills,
hammers, and saws). TSA considers any incident that threatens the
security or safety of an aircraft or its passengers and flight
crew to be a security incident. These could include a range of
activities onboard an aircraft such as disruptive passenger
behavior, violence against a passenger or crew member, hijacking
attempts, or the use of an improvised explosive device. By
examining the security impacts of the change to the prohibited
items list, this report considers the impacts that could result
from a passenger attempting to use scissors or tools to hijack an
aircraft or to commit other forms of violence onboard a flight.
Such actions fall within TSA's statutory responsibility to ensure
the safety and security of passengers and crew aboard aircraft.
In accordance with Conference Report 109-699, which accompanied
the fiscal year 2007 Department of Homeland Security (DHS)
appropriations act, this report addresses the following
questions: (1) What was TSA's basis for removing certain scissors
and tools from the prohibited items list and what are stakeholder
views on the change? (2) What have been the impacts, if any, of
the removal of certain scissors and tools from the prohibited
items list on the security of aircraft passengers and flight crew
and on the effectiveness of checkpoint screening operations?
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-07-623R
ACCNO: A68747
TITLE: Aviation Security: TSA's Change to Its Prohibited Items
List Has Not Resulted in Any Reported Security Incidents, but the
Impact of the Change on Screening Operations Is Inconclusive
DATE: 04/25/2007
SUBJECT: Agency missions
Airlines
Airport security
Airports
Aviation security
Commercial aviation
Data collection
Explosives
Flight crews
Homeland security
Passenger screening
Program evaluation
Search and seizure
Security threats
Terrorism
Weapons
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-07-623R
April 25, 2007
The Honorable Robert C. Byrd
Chairman
The Honorable Thad Cochran
Ranking Minority Member
Subcommittee on Homeland Security
Committee on Appropriations
United States Senate
The Honorable David E. Price
Chairman
The Honorable Harold Rogers
Ranking Minority Member
Subcommittee on Homeland Security
Committee on Appropriations
House of Representatives
The Honorable Hillary Rodham Clinton
United States Senate
Subject: Aviation Security: TSA's Change to Its Prohibited Items List Has
Not Resulted in Any Reported Security Incidents, but the Impact of the
Change on Screening Operations Is Inconclusive
The alleged August 2006 terrorist plot to detonate liquid explosives
onboard multiple commercial aircraft bound for the United States from the
United Kingdom has highlighted both the continued importance of securing
the civil aviation system and the potential that improvised explosive
devices (IED) may be smuggled onboard passenger aircraft. The
Transportation Security Administration (TSA) has primary responsibility
for ensuring the security of civil aviation, which includes the safety of
passengers and flight crew.^1
One measure TSA uses to protect the aviation system is prohibiting
individuals from carrying items that it determines to be a threat to the
aircraft and its passengers into an airport sterile area or onboard an
aircraft either in their carry-on bag or on their person.^2 To implement
this measure, TSA maintains a prohibited items list that informs both the
Transportation Security Officers (TSO) who conduct passenger screening and
the traveling public of items that will not be allowed into an airport
sterile area or onboard an aircraft. In December 2005, TSA revised its
prohibited items list to allow passengers to carry: (1) metal scissors
with pointed tips and a blade 4 inches or less in length as measured from
the fulcrum; and (2) tools--such as pliers, screwdrivers, and wrenches--7
inches or less in length (excluding crowbars, drills, hammers, and
saws).^3
1See 49 U.S.C. SS 114(d), 44903(b).
^2Sterile areas are located within the terminal where passengers are
provided access to boarding aircraft.
TSA considers any incident that threatens the security or safety of an
aircraft or its passengers and flight crew to be a security incident.
These could include a range of activities onboard an aircraft such as
disruptive passenger behavior, violence against a passenger or crew
member, hijacking attempts, or the use of an improvised explosive device.
By examining the security impacts of the change to the prohibited items
list, this report considers the impacts that could result from a passenger
attempting to use scissors or tools to hijack an aircraft or to commit
other forms of violence onboard a flight. Such actions fall within TSA's
statutory responsibility to ensure the safety and security of passengers
and crew aboard aircraft. In accordance with Conference Report 109-699,
which accompanied the fiscal year 2007 Department of Homeland Security
(DHS) appropriations act,^4 this report addresses the following questions:
(1) What was TSA's basis for removing certain scissors and tools from the
prohibited items list and what are stakeholder views on the change? (2)
What have been the impacts, if any, of the removal of certain scissors and
tools from the prohibited items list on the security of aircraft
passengers and flight crew and on the effectiveness of checkpoint
screening operations?
To address these objectives, we analyzed TSA documentation and data,
including TSA security incident reports, TSA written analyses related to
the prohibited items list change, results of Threat Image Projection (TIP)
testing,^5 and data on training hours completed by TSOs. Although the TIP
data we received had limitations, we believe that they are sufficiently
reliable for the purposes of this report and that the data on training
hours are sufficiently reliable as well. We also met with two Federal
Security Directors (FSD) to obtain their views on the impact of the
prohibited items list change on checkpoint screening operations.^6
However, information obtained from our interviews with these FSDs cannot
be generalized because we did not use random selection or representative
sampling when determining which FSDs should be interviewed. In addition,
we met with officials at the Federal Aviation Administration (FAA) and the
Federal Air Marshals Service (FAMS)--a component of TSA--to obtain their
views regarding the prohibited items list change. We also met with TSA
officials to obtain information on their rationale behind the change. We
spoke with 13 stakeholders within the aviation industry, including
representatives of 4 domestic aviation associations, the largest
association representing airline pilots in the United States, the largest
association representing flight attendants in the United States, an
association representing federal air marshals and other federal law
enforcement officers, an international aviation association, and 5
aviation security experts. We also met with a major aircraft manufacturer
to determine whether there are any major safety concerns related to the
change to the prohibited items list. Finally, we incorporated aspects of a
recently issued GAO report on passenger checkpoint screening procedures,
which included a review of the factors TSA considered in modifying the
prohibited items list and TSA's analysis supporting the December 2005
prohibited items list change.^7 We conducted our work from November 2006
through March 2007 in accordance with generally accepted government
auditing standards. More details about the scope and methodology of our
work are presented in enclosure I.
^370 Fed. Reg. 72,930 (Dec. 8, 2005).
^4See H.R. Conf. Rep. No. 109-699, at 139 (2006) (accompanying H.R. 5441,
enacted into law as the Department of Homeland Security Appropriations
Act, 2007, Pub. L. No. 109-295, 121 Stat 1355 (2006)).
^5The Threat Image Projection system is designed to test TSOs' detection
capabilities by projecting threat images, including guns, knives, and
explosives, onto carry-on bags as they are screened during actual
operations. TSOs are responsible for identifying the threat image and
calling for the bag to be searched. Once prompted, TIP identifies to the
TSO whether the threat is real and then records the TSO's performance in a
database that could be analyzed for performance trends.
^6FSDs are the ranking authorities responsible for leading and
coordinating security activities at U.S. commercial airports at which TSA
provides for or oversees the provision of screening activities.
Results in Brief
TSA's stated purpose in removing certain scissors and tools from the
prohibited items list was to shift TSO focus from items considered by TSA
to pose a low threat (including certain scissors and tools) to items
considered to pose a high threat, such as explosives. The change also was
intended to better allocate TSA resources to implement other security
measures that target explosives--a change supported by the majority of
aviation industry stakeholders that we interviewed. TSA's decision to
remove these items from the prohibited items list was based on the
professional judgment of TSA officials that these items do not pose a
significant threat to the security of the cockpit or to passengers and
flight crew as well as internal studies that sought to examine, among
other things, risks to flight security and considerations of customer
concerns and screening efficiencies. As part of these internal studies,
TSA collected data on the number and types of prohibited items surrendered
at checkpoints and the time it takes for TSOs to conduct carry-on bag
searches. In March 2007, we reported that TSA did not analyze these data
to determine the extent to which TSO resources would actually be freed up
to implement other security measures, nor did TSA analyze other relevant
factors such as the amount of time taken to search for small scissors and
tools and the number of TSOs conducting these searches. We recommended
that TSA develop sound evaluation methods, when possible, that can help
TSA determine whether proposed procedures would achieve their intended
purpose.^8 TSA concurred with the recommendation and stated that it plans
to make better use of generally accepted research design principles and
techniques when operationally testing proposed changes to screening
procedures. Based on our analysis of TSA data for the third and fourth
quarters of fiscal year 2005 (a 6-month period), we determined that TSOs
spent, on average, less than 1 percent of their time--about 1 minute per
day over the 6-month period--searching for the approximately 1.8 million
sharp objects, other than knives and box cutters, that were found at
passenger screening checkpoints between April 2005 and September 2005.
Therefore, it may not have been accurate for TSA to assume that no longer
requiring TSOs to search for small scissors and tools would significantly
contribute to TSA's efforts to free up TSO resources that could be used to
implement other security measures. TSA acknowledged that its data
collection and analysis effort may not have been methodologically rigorous
but stated that it did serve to provide insights regarding the type and
quantity of items collected at the passenger checkpoint. TSA officials
also stated that even if TSO resources were not freed up as intended, they
continue to view their decision to allow small scissors and tools onboard
aircraft as sound, particularly because their review of threat information
determined that small scissors and tools do not pose a significant threat
to aviation security. Additionally, 9 of the 13 aviation industry
stakeholders whom we interviewed supported the removal of small scissors
and tools from the prohibited items list because they believe small
scissors and tools do not pose a risk to the security of the aircraft and
stated that the change will increase TSA's focus on IEDs; the remainder
disagreed, citing potential increased security risks. TSA officials
acknowledged that small scissors and tools, as well as other items
permitted onboard commercial aircraft, may potentially be used as weapons
against passengers and flight crew, but stated that these items cannot be
used to hijack an aircraft given the other layers of security in place,
such as hardened cockpit doors that prevent unauthorized access to the
flight deck.
^7GAO, Aviation Security: Risk, Experience, and Customer Concerns Drive
Changes to Airline Passenger Screening Procedures, but Evaluation and
Documentation of Proposed Changes Could Be Improved, GAO-07-57SU
(Washington, D.C.: Mar. 7, 2007). The information in this report is
considered sensitive security information in accordance with 49 C.F.R.
part 1520 and is not available to the public. A public version of this
report ( [1]GAO-07-634 ) is expected to be issued in May 2007.
^8Ibid.
Based on our review of TSA security incident reports from the time period
following the prohibited items list change (December 2005 through February
2007), there have been no reported security incidents onboard an aircraft
involving the use of small scissors or tools. However, the impact of the
prohibited items list change on security is uncertain because the absence
of an event occurring involving the use of these items does not preclude
the possibility that future occurrences could happen. In addition, with
respect to the effectiveness of the change on checkpoint screening
operations, it is not possible to determine this because the available
data are inconclusive. As we reported in March 2007, TSA conducted
informal studies 30, 60, and 90 days following the change and concluded
that TSO time was freed up to focus on high-threat items, but our analysis
of TSA data does not support this conclusion.^9 TSA agrees that the agency
could have conducted a more methodologically sound evaluation of the
impact of the prohibited items list change, but continues to believe that
the change nevertheless significantly contributed to the agency's efforts
to free up TSO resources to focus on detecting high-threat items, such as
explosives. It also is not clear whether the change had any impact on
TSOs' ability to detect explosives--a key goal of the change. One way TSA
measures the effectiveness of the passenger screening system in detecting
threat items, such as explosives, is the results of threat image
projection testing.^10 However, TSA does not claim nor do the data
definitively support that TSA's change to the prohibited items list had
any impact on threat image projection results because TSA implemented
other changes to checkpoint screening operations at or around the same
time as the prohibited items list change. With regard to TSA's efforts to
increase training for identifying explosives as part of its overall effort
to become a more risk-based organization, TSA data between October 2004
and January 2007 show an increase in the average number of hours spent in
training per TSO, but this trend began before the change to the prohibited
items list and there are other factors that may have contributed to this
increase.
^9Ibid.
^10The results of TSA TIP testing are considered sensitive security
information and thus could not be included in this report.
Background
In accordance with applicable laws and regulations, TSA prohibits weapons,
explosives or incendiaries, and other items that TSA believes pose a
significant threat to civil aviation security onboard commercial
aircraft.^11 TSA has divided these prohibited types of items into seven
categories. Individuals are prohibited from carrying these items into an
airport sterile area or onboard an aircraft either in their carry-on bag
or on their person. Table 1 provides a description of the items included
in the seven categories.
Table 1: Categories and Descriptions of Prohibited Items
Category of prohibited Description of items included in the category
item
Guns and firearms BB guns; compressed air guns; firearms; flare
pistols; gun lighters; parts of guns and firearms;
pellet guns; realistic replicas of firearms; spear
guns; starter pistols; stun guns/cattle
prods/shocking devices.
Sharp objects Axes and hatchets; bows and arrows; ice axes/ice
picks; knives of any length, except rounded-blade
butter and plastic cutlery; meat cleavers;
razor-type blades, such as box cutters, utility
knives, and razor blades not in a cartridge, but
excluding safety razors; sabers; scissors, metal
with pointed tips and a blade length greater than
4 inches as measured from the fulcrum; swords;
throwing stars (martial arts).
Club-like items Baseball bats; billy clubs; blackjacks; brass
knuckles; cricket bats; golf clubs; hockey sticks;
lacrosse sticks; martial arts weapons, including
nunchucks, and kubatons; night sticks; pool cues;
ski poles.
All explosives Ammunition; blasting caps; dynamite; fireworks;
flares in any form; gunpowder; hand grenades;
plastic explosives; realistic replicas of
explosives.
Incendiaries Aerosol, any, except for personal care or
toiletries in limited quantities; fuels, including
cooking fuels and any flammable liquid fuel;
gasoline; gas torches, including microtorches and
torch lighters; lighter fluid; strike-anywhere
matches; turpentine and paint thinner; realistic
replicas of incendiaries; all lighters.
Disabling chemicals and Chlorine for pools and spas; compressed gas
other dangerous items cylinders (including fire extinguishers); liquid
bleach; mace; pepper spray; spillable batteries,
except those in wheelchairs; spray paint; tear
gas.
Tools Crowbars; drills and drill bits, including
cordless portable power drills; hammers; saws and
saw blades, including cordless portable power
saws; other tools greater than 7 inches in length,
including pliers, screwdrivers, and wrenches.
Source: TSA.
Passenger screening is a process by which personnel authorized by
TSA inspect individuals and property to deter and prevent the
carriage of any unauthorized explosive, incendiary, weapon, or
other items included on TSA's prohibited items list onboard an
aircraft or into a sterile area.^12 Passenger screening
personnel--TSOs--must inspect individuals for prohibited items at
designated screening locations.^13 As shown in figure 1, the
passenger screening functions are
o X-ray screening of property,
o walk-through metal detector screening of
individuals,
o hand-wand or pat-down screening of individuals,
o physical search of property and trace detection for
explosives, and
o behavioral observation.
^11See 49 U.S.C. S 44902; 49 C.F.R. SS 1540.111, 1544.201(d).
^12Access to sterile areas is controlled by TSOs (or by nonfederal
screeners at airports participating in the Screener Partnership Program)
at checkpoints where they conduct physical screening of individuals and
their carry-on baggage for weapons, explosives, and other prohibited
items.
^13TSOs must deny passage beyond the screening location to any individual
or property that has not been screened or inspected in accordance with
passenger screening standard operating procedures. If an individual
refuses to permit inspection of any item, that item must not be allowed
into the sterile area or onboard an aircraft.
Figure 1: Passenger Checkpoint Screening Functions
Notes: Explosive trace detection (ETD) works by detecting vapors and
residues of explosives. Human operators collect samples by rubbing swabs
along the interior and exterior of an object that TSOs determine to be
suspicious, and place the swabs in the ETD machine, which then chemically
analyzes the swab to identify any traces of explosive materials.
Bomb Appraisal Officers (BAO) are available to respond to unresolved
alarms at the checkpoint that involve possible explosive devices. The BAO
may contact appropriate law enforcement or bomb squad officials if review
indicates possible or imminent danger, in which case the BAO ensures that
the security checkpoint is cleared. The BAO approves reopening of security
lane(s) if no threat is posed.
aBDOs are TSOs specially trained to detect suspicious behavior in
individuals approaching the checkpoint. Should the BDO observe such
behavior, he or she may refer the individual for individual screening or
to a law enforcement officer. As of April 2007, BDOs are not present at
all airport checkpoints.
bThe hand-wand or pat-down is conducted if a passenger is identified or
randomly selected for additional screening because he or she met certain
criteria or alarmed the walk-through metal detector.
cManual or ETD searches of accessible property occur if the passenger is
identified or randomly selected for additional screening or if the
screener identified a potential prohibited item on X-ray.
Typically, passengers are only subjected to X-ray screening of
their carry-on items and screening by the walk-through metal
detector. Passengers whose carry-on baggage alarms the X-ray
machine, who alarm the walk-through metal detector, or who are
designated as selectees--that is, passengers selected by the
Computer-Assisted Passenger Prescreening System (CAPPS)^14 or
other TSA-approved processes to receive additional screening--are
screened by hand-wand or pat-down and have their carry-on items
screened for explosives traces or physically searched.
In addition to passenger checkpoint screening, other layers of aviation
security recognized by TSA include, among other things:
o Hardened cockpit doors to prevent unauthorized access or forced
entry to the flight deck.
o Deployment of federal air marshals on certain flights to provide
physical security should an incident occur.
o Federal Flight Deck Officers (FFDO) Program to train pilots on
commercial passenger and cargo aircraft on how to use lethal force
against an intruder on the flight deck.^15
o Security training for flight and cabin crews to handle potential
threats onboard aircraft. Flight and cabin crews are expected to
defend the flight deck in accordance with a TSA and FAA-developed
guidance manual known as the Common Strategy.^16
In addition, TSA considers the vigilance of able-bodied passengers to be
an important layer of aviation security. Able-bodied passengers are those
passengers who may engage in self-defense actions should an incident occur
onboard commercial aircraft.
^14CAPPS is a computer-assisted system that, based on information obtained
from airline reservation systems, identifies passengers that may pose a
high risk to aviation security. These high-risk passengers, along with
other individuals selected for secondary screening, and their carry-on
baggage are subject to additional and more thorough screening.
^15Administered by TSA, the FFDO Program deputizes volunteer pilots of
commercial passenger aircraft as armed federal law enforcement officers
for the purpose of defending the flight deck "against acts of criminal
violence or air piracy." Since the program was officially established on
February 25, 2003, TSA has deputized thousands of eligible flight crew
members as FFDOs.
^16The Common Strategy is a detailed guidance manual developed by TSA and
FAA for pilots and other crewmembers to identify their responsibilities
and the appropriate responses during in-flight security threats. In
January 2005, TSA and FAA issued a revised version of the Common Strategy.
The previous version, referred to as Common Strategy #1, was the strategy
in effect on September 11, 2001. Common Strategy #1 was developed jointly
by industry, FAA, and FBI, and presumed a hijacker whose motive might be
ransom, escape from the law, political asylum, or publicity. According to
the Common Strategy, the terrorist attacks of September 11, 2001,
demonstrated that Common Strategy #1 was not effective in dealing with a
new breed of hijacker whose motives are terrorism, mass murder, and
suicide.
TSA Changed the Prohibited Items List to Shift TSO Resources to
Higher-Threat Priorities and Most Aviation Industry Stakeholders
Interviewed Supported TSA's Change
TSA Conducted Various Studies to Determine Whether Changing the Prohibited
Items List Would Free Up TSO Resources, but Some Efforts Lacked
Methodological Rigor
As we reported in March 2007,^17 TSA changed the prohibited items list in
an effort to shift TSO resources to focus on higher threats, such as
explosives, and based on its determination that small scissors and tools
do not pose a risk to aviation security.^18 TSA's decision was informed by
the conclusions reached by an Explosives Detection Improvement Task Force
established in October 2005 by the TSA Assistant Secretary to respond to
the threat of IEDs being carried through the checkpoint. The goal of the
task force was to apply a risk-based approach to screening passengers and
their baggage in order to enhance TSA's ability to detect IEDs. As part of
its analysis, the task force considered a number of factors including
threat information, TSO effectiveness, and overall screening performance.
According to TSA officials, the task force also considered the results of
a Prohibited Items Working Group that was established in February 2005 by
the then-TSA Assistant Secretary to develop recommendations for modifying
the prohibited items list to better reflect the current aviation security
environment.
The Prohibited Items Working Group assessed each item on the prohibited
items list using four criteria: (1) risks to flight security (i.e., can
the item be used to take down an aircraft in flight); (2) legal
restrictions (i.e., hazardous and other materials that are prohibited from
the aircraft or from the flight cabin); (3) public concern and screener
effectiveness (i.e., would permitting the item onboard an aircraft cause
significant passenger and flight crew concern regarding their safety); and
(4) international standards (i.e., international protocols recommend that
the item be prohibited from the aircraft or the flight cabin). At the
conclusion of its analysis, the working group recommended that scissors
with pointed tips less than 6 centimeters (2.36 inches) and tools less
than 7 inches be removed from the prohibited items list because these
items were not considered to represent a risk to the aircraft or cockpit
security. Although the working group based its size restriction for
scissors on the size parameters recommended by the International Civil
Aviation Organization (ICAO)--which is to provide for the safe, orderly,
and efficient development of international civil aviation--the working
group deviated from ICAO's recommendation to prohibit all pointed/edged
scissors.^19 A TSA representative from the working group stated that this
change was recommended because the working group concluded that
pointed/edged scissors could not be used to gain access to the cockpit to
take down an aircraft in flight. In addition, the working group stated
that concentrating on such items diminished TSA's efforts to focus on
identifying objects that pose the greatest threat to aviation security,
such as IEDs.
^17GAO-07-57SU.
^18The change to the prohibited items list was one of several other
changes to TSA procedures intended to focus more TSA resources on higher
threats, such as explosives. The Explosive Detection Improvement Task
Force recommended seven proposed procedures that were ultimately
implemented by TSA. These procedures were considered by senior TSA
officials as especially important for enhancing the detection of
explosives and for deterring terrorists from attempting to carry out an
attack. According to TSA, some of the proposed procedures, such as the
prohibited items list change, could also free up TSOs so that they could
spend more time on procedures for detecting explosives and less time on
procedures associated with low security risks, such as identifying small
scissors in carry-on bags. The seven proposed procedures tested by the
task force reflect both new procedures and modifications to existing
procedures. The procedures are discussed in detail in GAO-07-57SU.
Subsequent to the analysis of the working group, TSA's Explosives
Detection Improvement Task Force collected information from several
sources to test its assumption that a disproportionate amount of TSO
resources was being spent searching for small scissors and tools. First,
TSA reviewed data maintained in TSA's Performance Management Information
System,^20 which showed that during the third and fourth quarters of
fiscal year 2005 (a 6-month period), TSOs collected a total of about 1.8
million sharp objects other than knives or box cutters and about 468,000
tools. The sharp objects constituted 19 percent of all prohibited items
surrendered at the checkpoint during this period and tools constituted 5
percent of the items. Second, based on information provided by FSDs, TSOs,
and other screening experts, TSA determined that scissors constituted a
large majority of the total number of sharp objects found at passenger
screening checkpoints. TSA also concluded that small screwdrivers,
wrenches, and pliers make up a large majority of the tools found at
checkpoints. Third, TSA headquarters officials searched through
surrendered items bins at four airports and found that most of the
scissors had blades less than 4 inches in length and a very large
percentage of the tools that were surrendered were 7 inches or smaller.
Based on these collective efforts, TSA's Explosive Detection Improvement
Task Force concluded that a significant number of items found at the
checkpoint were low-threat, easily identified items, such as small
scissors and tools, and that a disproportionate amount of time was spent
searching for these items--time that could have been spent searching for
high-threat items, such as explosives. The task force also concluded that
because TSOs can generally identify scissors and tools on X-ray images
easily, if small scissors and tools were no longer on the prohibited items
list, TSOs could avoid conducting time-consuming physical bag searches to
locate and remove these items. TSA ultimately concurred with the
recommendations provided by the Explosive Detection Improvement Task Force
and decided to remove scissors less than 4 inches and certain tools less
than 7 inches from the prohibited items list.
Although TSA's rationale for its December 2005 change to the prohibited
items list was to reduce focus on low-threat items in order to free up TSO
time, attention, and resources to implement screening practices that
better focus on high-threat items--such as Screening Passengers by
Observation Technique (SPOT) and Unpredictable Screening Process
(USP)^21--we reported in March 2007 that TSA had not conducted the
necessary analysis of the data collected to determine the extent to which
the removal of small scissors and tools from the prohibited items list
could free up TSO resources. Specifically, we found that TSA had not
analyzed the data on sharp objects surrendered at the checkpoint along
with other relevant factors, such as the amount of time taken to search
for scissors and the number of TSOs at the checkpoint conducting these
searches. Based on our analysis of TSA's data for the third and fourth
quarters of fiscal year 2005 (a 6-month period), where we considered these
other relevant factors, we determined that TSOs spent, on average, less
than 1 percent of their time--about 1 minute per day over the 6-month
period--searching for the approximately 1.8 million sharp objects, other
than knives and box cutters, that were found at passenger screening
checkpoints between April 2005 and September 2005. If the average amount
of time TSOs spent searching for sharp objects per day over a 6-month
period was less than 1 minute per TSO, and sharp objects constituted just
19 percent of all prohibited items surrendered at checkpoints over this
period, then it may not be accurate to assume that no longer requiring
TSOs to search for small scissors and tools would significantly contribute
to TSA's efforts to free up TSO resources that could be used to implement
other security measures. TSA stated that the decision to remove small
scissors and small tools from the prohibited items list was not only based
on an analysis of data but was also firmly rooted in its assessment of
risk, professional judgment, and experience. According to TSA, this
included interviews with FSDs who unanimously indicated the change would
free up TSO resources, as well as examinations of the prohibited items
surrendered at several airports and a study to determine the amount of
time taken to conduct bag searches.
^19Nations that are members to ICAO agree to cooperate with other member
states to meet standardized international aviation security measures. ICAO
recommends that pointed/edged scissors of any size should be prohibited
from the flight cabin, while rounded or blunt scissors less than 6 cm
should be permitted in the flight cabin. The TSA Prohibited Items Working
Group utilized the ICAO size parameters, but applied the parameters to
both rounded/blunt scissors as well as pointed/edged scissors, thus
deviating from the ICAO recommendation to ban all pointed/edged scissors.
^20TSA's Performance Management Information System is designed to collect,
analyze, and report passenger and baggage screening performance data, such
as wait times at selected airports, workload data, and the performance and
utilization of passenger and baggage screening equipment. TSA headquarters
uses the Performance Management Information System data to support
external reporting on performance and internal decision-making processes.
TSA acknowledged that this particular data collection and analysis effort
may not have been methodologically rigorous, but stated that it did serve
to provide insights regarding the type and quantity of items collected at
the passenger checkpoint and the analysis effort generally supported the
decision. Additionally, the TSA Assistant Secretary stated that even if
TSA determined that the proposed prohibited items list modification would
not free up existing TSO resources to conduct explosives detection
procedures, he would have implemented the change anyway considering that
such items no longer posed a significant security risk given the multiple
layers of aviation security.
In our March 2007 report, we recommended that TSA develop sound evaluation
methods, when possible, that can help TSA determine whether proposed
procedures that are operationally tested would achieve their intended
purpose, such as enhancing TSA's ability to detect prohibited items and
freeing up existing TSO resources that could be used to implement proposed
procedures.^22 TSA concurred with the recommendation and stated that it
plans to make better use of generally accepted research design principles
and techniques when operationally testing proposed changes to screening
procedures. For example, TSA agreed to consider using random selection,
representative sampling, and control groups in order to isolate the impact
of proposed changes to screening procedures from the impact of other
variables.
^21Screening Passengers by Observation Technique involves specially
trained TSOs observing the behavior of passengers and resolving any
suspicious behavior through casual conversation with passengers and
referring suspicious passengers to selectee screening. Unpredictable
Screening Process entails random selection of passengers across two
screening lanes to be subjected to a predetermined element of the selectee
screening process.
^22GAO-07-57SU.
Most Aviation Industry Stakeholders We Contacted Supported TSA's Changes
to the Prohibited Items List
The majority (9 of 13) of the aviation industry stakeholders that we
interviewed supported the removal of small scissors and tools from the
prohibited items list. In general, these stakeholders said that they
believe that the layers of aviation security reduce a passenger's ability
to access the cockpit with low-threat items, and further noted that
passengers may carry other items onboard an aircraft (such as glass
bottles, pens, and sharpened credit cards) that may also be used as
weapons. Stakeholders also stated that TSOs will be able to better focus
on detecting IEDs if low-threat items such as small scissors and tools are
removed from the prohibited items list. However, 4 out of 13 aviation
industry stakeholders that we interviewed were opposed to the prohibited
items list change, stating that permitting scissors increases the risk of
violence against passengers and flight crew onboard an aircraft. Some of
these stakeholders also stated that scissors also increase the risk that
hijackings could be successfully implemented because scissors have bladed
edges and pointed tips and therefore can be used as knives, and because
terrorists can train with scissors to perfect their use as weapons. These
stakeholders further stated that unlike other items that can be improvised
to create a cutting surface (such as broken glass bottles), terrorists
would not need to alter scissors onboard aircraft to use them as weapons.
This could also allow a passenger to use the cutting edge and/or the
sharpened tip of a scissor as a weapon without alerting other passengers
or flight crew, as compared with the attention that could be drawn to a
passenger that breaks a glass bottle.
TSA acknowledges that scissors and tools may be used as weapons against
passengers and flight crew. However, TSA stated that other items that are
permitted onboard commercial aircraft, such as pens and glass bottles, may
also be used as weapons against passengers and flight crew. TSA also
maintained that its focus is on detecting explosives or items that can be
used to breach the cockpit and potentially hijack the aircraft, which TSA
and the majority of the aviation industry stakeholders that we spoke with
view as a significant threat to aviation. TSA maintained that small
scissors and tools cannot be used to hijack an aircraft, particularly
given the other layers of security.
Although stakeholders who both supported and disagreed with TSA's change
stated that the layers of security implemented since September 11,
2001--particularly the hardened cockpit door--have decreased the
likelihood of a successful hijacking, stakeholders generally stated that
the risk of a hijacking is highest when the cockpit door is opened. In an
attempt to mitigate this potential vulnerability, and in accordance with
the air carrier's responsibility to ensure that no passenger can access
the flight deck when the cockpit door needs to be opened during flight,
air carriers will typically place a beverage cart between passengers and
the cockpit with a flight attendant standing behind the cart. The beverage
cart and the flight attendant serve as a "secondary barrier" between
passengers and the cockpit door. However, two aviation stakeholders--a
former law enforcement officer who provides self-defense training and a
representative from the association of flight attendants--stated that this
secondary barrier can be circumvented by a determined terrorist using a
scissor to attack the flight attendant who is manning the beverage cart,
which could allow the terrorist to negotiate around the beverage cart and
then access the open cockpit door. A senior TSA official stated that
flight crew protocols are sufficient to ensure passengers cannot breach
the cockpit and that mechanisms are in place to ensure that cockpit doors
are opened for brief periods of time.
No Security Incidents against Passengers or Crew Using Scissors or Tools
Have Been Reported to TSA Since the Change to the Prohibited Items List,
but the Impact of the Change on Screening Operations Is Inconclusive
No Onboard Incidents Involving Small Scissors or Tools Reported to TSA
Since Prohibited Items List Change and FAA Does Not Believe These Items
Pose a Risk to the Integrity of an Aircraft
Based on our review of TSA security incident reports from the time period
following the prohibited items list change (December 2005 through February
2007), there have been no reported security incidents onboard an aircraft
involving the use of small scissors or tools.^23 However, TSA and aviation
security stakeholders we spoke with acknowledged that the absence of an
onboard incident involving scissors or tools as weapons does not preclude
the possibility of such an incident in the future. In addition, based on
aircraft vulnerability and system safety and security analyses performed
to date by government and industry, neither FAA nor a major aircraft
manufacturer we interviewed perceive any meaningful increase in risk to
the integrity of an aircraft associated with TSA's decision to permit
small scissors and tools onboard aircraft.^24 FAA officials also stated
that aircraft are designed so that there are many layers of protection to
prevent damage to the integrity of an aircraft from within (e.g., hardened
cockpit doors and separate and redundant wiring for critical systems with
few internal access points). The aircraft manufacturer stated that while
it is possible that terrorists or others intending to do harm to the
aviation system could use these items in ways not currently foreseeable,
given current risk mitigation activities, the ability to inflict major
damage to an aircraft with them is extremely remote.
Impact of Prohibited Items List Change on Checkpoint Screening Operations
Is Inconclusive
TSA conducted informal studies 30, 60, and 90 days after the prohibited
items list changes went into effect to determine whether the change had
resulted in reductions in the percentage of carry-on bags that were
searched and overall screening time. However, in a prior report, we
identified limitations in TSA's methodology for conducting these studies
and concluded that it may not be accurate to assume that the prohibited
items list change freed up resources.^25 TSA agrees that the agency could
have conducted a more methodologically sound evaluation of the impact of
the prohibited items list change, but TSA continues to believe that the
change did significantly contribute to the agency's efforts to free up TSO
resources to focus on detection of high-threat items, such as explosives.
TSA officials stated that that they have not conducted or planned any
additional studies on the prohibited items list change to determine the
impact of the change on the effectiveness of screening operations.
Officials continue to view the change as sound based on their professional
judgment and assessment of risk, and state that the change allowed the
agency to shift focus from low risks to areas such as increased focus on
explosive devices and increased training.
^23Because TSA is the primary agency responsible for aviation security and
maintains records of aviation security incidents, TSA security incident
reports were our primary source of information for identifying incidents
involving small scissors or tools. These security incident reports
summarize transportation security incidents--including aviation--that are
reported to TSA and include descriptions of the incident. We used the
December 2005-February 2007 time period because it was after the effective
date of the prohibited items list change. Pursuant to TSA-issued Security
Directive 1544-04-15, all aircraft operators are required to immediately
report all threats that could affect the security of commercial aircraft
to TSA.
^24 The Federal Aviation Administration has primary responsibility for
ensuring the safety of civil aviation operations, including the operation
of air traffic control and regulating the manufacture, operation, and
maintenance of aircraft. See 49 U.S.C. S 44701.
In February 2007, a TSA official stated that some FSDs interviewed several
TSOs after the prohibited items list change went into effect, and these
TSOs reported that the change did save screening time. However, TSA could
not identify how many TSOs were interviewed, at which airports the TSOs
were located, and how the TSOs were selected for the interview; nor did
TSA document the results of these interviews. As TSA did not use random
selection or representative sampling when determining which TSOs should be
interviewed, the interview results cannot be generalized.
Most of the FSDs we interviewed in August 2006 as part of our passenger
screening procedures review stated that the prohibited items list change,
in addition to another change, did not collectively free up TSO resources
to perform screening activities focused on threats considered to pose a
high risk, such as explosives.^26 Specifically, 13 of 19 FSDs we
interviewed at airports that tested USP and SPOT said that TSO resources
were not freed up as a result of the prohibited items list change and
another change made by TSA during this time frame.^27 In addition, 9 of
the 19 FSDs said that in order to operationally test the procedures, TSOs
had to work overtime, switch from other functions (such as checked baggage
screening), or a screening lane had to be closed. Moreover, 13 of the 19
FSDs stated that TSOs did not experience more time to conduct explosives
training.^28
In addition to the lack of clarity about the impact of changes to the
prohibited items list on TSO's available time, it also is not clear
whether the change had any impact on TSOs' ability to detect IEDs--a key
goal of the change. The results of threat image projection (TIP) testing
are one way that TSA measures the effectiveness of the passenger screening
system in detection of threat items, such as explosives. The results of
TSA TIP testing are considered sensitive security information and thus
could not be included in this report. Nevertheless, it is not clear
whether TSA's change to the prohibited items list had any impact on TSOs'
ability to identify IEDs during TIP testing because multiple factors could
have accounted for the changes in TIP scores over time. For example, TSA
implemented other changes to checkpoint screening operations at or around
the same time as the prohibited items list change. These changes include
both new and revised procedures, such as: revising the USP to include
selected screening process elements like explosive trace detection of
footwear and accessible property; screening 100 percent of passengers'
footwear; banning liquids and gels; revising bulk-item pat downs to
include the waistline down to the ankles; targeting threat area searches
within baggage; revising the CAPPS rules; and implementing the new SPOT
procedure. In fact, FSDs we interviewed at two category X airports^29 in
February 2007 as well as other TSA officials stated that at this time it
is not possible to isolate the effect of the prohibited items list change
from these additional changes in order to determine its impact on
checkpoint screening operations and whether the prohibited items list
change freed up TSO resources.^30
^25The results of the informal follow-on studies, which were conducted at
6 to 9 airports, show that the percentage of carry-on bags searched
increased by about 0.1 percentage point at the time of the 30-day study,
then decreased by 2.3 and 0.7 percentage points at the time of the 60-day
and 90-day studies, respectively. However, the results of these informal
studies may not be reliable due to the limitations in the methodology TSA
used to conduct the studies. Specifically, TSA did not use a methodology
that would control for factors other than the prohibited items list change
that may influence the percentage of carry-on bags searched by TSOs. To do
this, TSA would have had to develop a formal, systematic methodology for
randomly selecting various times of day, location of checkpoints, number
of checkpoints, and so on for data collection. By not controlling for such
factors, TSA may not know the extent to which a reduction in the
percentage of carry-on bags searched is due to the prohibited items list
changes. See GAO-07-57SU.
^26See GAO-07-57SU. An additional measure intended to free up TSO
resources involved changes to CAPPS rules to reduce the number of
passengers selected for secondary screening. TSA's assumption is that
these changes could allow TSOs who were normally assigned to selectee
screening duties to be reassigned to new procedures, which may require new
screening positions.
^27Since we did not use random selection or representative sampling when
determining which FSDs should be interviewed, the interview results cannot
be generalized.
^28Of the remaining 6 FSDs, 5 said that TSO resources were freed up as a
result of the prohibited items list and CAPPS rules changes, and 1 was
uncertain whether TSO resources were actually freed up.
With regard to TSA's efforts to increase training for identifying IEDs as
part of its overall effort to become a more risk-based organization, TSA
data between October 2004 and January 2007 show an increase in the average
number of hours spent in training per TSO, but this trend began before the
change to the prohibited items list and there are other factors that may
have contributed to this increase. Our analysis of these data show an
increase of an average of 1.68 hours per TSO in monthly IED training over
the 29-month period, from an average of 0.42 hours per TSO in October 2004
to an average of 2.10 hours per TSO in February 2007. According to TSA's
TSO training officials, there are two primary explanations for the
increase: (1) in October 2005 TSA provided a 4-hour IED training course to
18,000 TSOs over a 3-week period, and, according to TSA, about 98 percent
of the 48,236 TSOs on board had received classroom, checkpoint, or
computer-based improvised explosive device recognition training as of
February 6, 2007; and (2) in May 2006 TSA instituted a new monthly
requirement of 4 hours of IED training per TSO. Thus, although a goal of
the prohibited items list change was to increase TSO training hours for
detecting IEDs, TSA program officials acknowledge, and we agree, that it
is not clear whether the change to the prohibited items list had any
impact on time spent in training.
Concluding Observations
TSA is faced with the challenge of addressing numerous threats to
commercial aviation security, as demonstrated by the alleged August 2006
terrorist plot to detonate liquid explosives onboard multiple commercial
aircraft bound for the United States from the United Kingdom. TSA's
December 2005 change to the prohibited items list is one of several
efforts TSA has made to focus its resources on addressing the threat posed
by explosives, which TSA considers to be the most significant threat to
commercial aviation security. While TSA's consideration of threat
information, the professional judgment of TSA personnel, data analysis,
and international standards all constitute reasonable inputs to making
informed decisions on how to best anticipate and address threats given its
available resources, the impact of the prohibited items list change on
security and screening effectiveness is inconclusive. Nevertheless, we are
encouraged that TSA recognized the limitations in its analysis of data
used to help inform the prohibited items list change and plans to improve
the methodological rigor for evaluating proposed changes to passenger
screening procedures in the future, as we recommended in our March 2007
report. This effort will be particularly important as additional changes
to passenger screening procedures--including future revisions to the
prohibited items list--are considered and implemented.
^29TSA classifies the over 400 commercial airports in the United States
into one of five categories--X, I, II, III, and IV. Generally, category X
airports have the largest number of passenger boardings and category IV
airports have the smallest number.
^30The views of the two FSDs we interviewed may not be representative of
the views of the population of FSDs.
Agency Comments
We provided a draft of the report to DHS for its review and comment. TSA
provided technical comments, which we incorporated as appropriate.
We will send copies of this report to the Secretary of the Department of
Homeland Security; the Assistant Secretary, TSA; and interested
congressional committees as appropriate. We will also make this report
available at no charge on GAO's Web site at http://www.gao.gov . If you
or your staff have any questions about this report, please contact me at
(202) 512-2757 or [email protected] . Contact points for our Offices
of Congressional Relations and Public Affairs may be found on the last
page of this report. Key contributors to this report are listed in
enclosure II.
Robert Goldenkoff
Acting Director, Homeland Security and Justice Issues
Enclosures
Enclosure I: Scope and Methodology
To describe TSA's basis for removing certain scissors and tools from the
prohibited items list and stakeholder views on the change, we obtained and
analyzed TSA documentation of the proposed prohibited items list change
considered by TSA's Explosives Detection Improvement Task Force, which was
the deliberating body for proposed TSA procedural changes that were
considered between October 2005 and December 2005. We also obtained and
analyzed a draft TSA Prohibited Items Working Group analysis, as well as
TSA public statements and testimonies regarding the rationale for the
prohibited items change. We also met with TSA officials to obtain
information on their rationale behind the change. In addition, we met with
officials at the Federal Aviation Administration (FAA) and the Federal Air
Marshals Service (FAMS)--a component of TSA--to obtain their views
regarding the prohibited items list change. We spoke with organizations
within the aviation community including four domestic aviation
associations, one international aviation association, a major aircraft
manufacturer, the largest association representing airline pilots in the
United States, the largest association representing flight attendants in
the United States, and an association representing federal air marshals
and other federal law enforcement officers.^31 In addition, we met with
five aviation security experts to obtain their views on TSA's change to
the prohibited items list.^32 We selected these experts based on their
depth of experience in the field of aviation, employment history, and
their recognition in the aviation security community. We also met with a
major aircraft manufacturer to determine whether there are any major
safety concerns related to the change to the prohibited items list.
Finally, we incorporated aspects from our recently issued report on
passenger checkpoint screening procedures,^33 which included a review of
the factors TSA considered in modifying the prohibited items list and
TSA's analysis supporting the December 2005 prohibited items list change.
To determine the impacts, if any, that the removal of certain scissors and
tools from the prohibited items list had on security and on the
effectiveness of screening operations, we obtained and reviewed TSA
documentation and data including the results of threat image projection
(TIP) testing and data on training hours completed by Transportation
Security Officers (TSO). We sent written questions about data quality
control and reporting procedures to TSA officials responsible for
collecting and analyzing these data, and received responses to these
questions. The TIP data TSA provided contained limitations. First, the
data contained only monthly averages for tests in which improvised
explosive devices (IED) images had been successfully identified by TSOs,
according to individual airports in each airport category; we did not
receive the raw numbers of image presentations from which the percentages
were derived. Therefore, to compute an average percentage of successful
TIP tests across all airports, we computed an average of averages.
Computing an average in this manner can provide a result that is slightly
different than if raw data had been used. For example, we could not adjust
our computations to account for differing numbers of image presentations
or the rate of image presentations by airport. Second, there were missing
values, or no test results, for some airports in certain months. Despite
these limitations, we believe the TIP data were sufficiently reliable to
provide an indication of TSOs' abilities to identify IED images. In
addition, we interviewed Federal Security Directors (FSD) from Boston
Logan Airport and Washington Dulles International Airport to obtain
anecdotal information about their views on the impact of the prohibited
items list change on checkpoint screening operations. However, the
perspectives of these two FSDs cannot be considered to be representative
of the views of FSDs nationwide or generalized because we did not use
random selection or representative sampling when determining which FSDs
should be interviewed.
^31Specifically, we met with the Air Transport Association (ATA), the
National Air Carrier Association (NACA), the Regional Airline Association
(RAA), the Air Carrier Association of America (ACAA), the International
Air Transport Association (IATA), the Air Line Pilots Association (ALPA),
the Association of Flight Attendants (AFA), and the Federal Law
Enforcement Officers Association (FLEOA).
^32The views of these five experts may not necessarily represent the
general view of other experts in the field of aviation security.
^33GAO-07-57SU.
To determine whether the change to the list of prohibited items had any
impact on TSO time spent in training, we also analyzed training data
provided by TSA on the average number of hours spent in training per TSO
for the period from October 2004 through February 2007. TSA uses a dynamic
system to capture training data called the Online Learning Center, and TSA
offers several reasons for the dynamic nature of this system. First, TSA
employees and contractors are continuously allowed to update training
history hours. As a result, data on training hours and attendance
extracted from the database at two different points in time may vary as
employees and contractors update their training history. Second, there can
be a delay in updating training data due to manual entry of student
results. TSA policy is that final reports are generated on the 10^th of
each month in order to permit time to collect and consolidate airport data
for manual data entry. The training hour data were sufficiently reliable
for our purpose in showing a general increase in IED training over time.
Our results are based on the data TSA provided to us on March 21, 2007.
To determine if any security incidents onboard an aircraft involving the
use of small scissors or tools were reported to TSA, we reviewed and
analyzed TSA security incident reports from the time period following the
prohibited items list change (December 22, 2005--the effective date of the
prohibited items list change--through February 28, 2007). Because TSA is
the agency with primary responsibility for aviation security and maintains
records of aviation security incidents, TSA security incident reports were
our primary source of information for identifying incidents involving
small scissors or tools. We followed a two-step process to identify
incidents appropriate to our review. During the first step, one analyst
reviewed all incidents in each daily TSA security incident report to
identify any incidents that he or she discerned involved small scissors or
tools based on key words or phrases in the incident title or description.
A log was created for each incident report reviewed. In the second step, a
random sample of 10 percent of the incident reports was selected, and
these reports and their accompanying logs were reviewed by a second
analyst to verify the accuracy of the first analyst's judgments. We
limited the scope of our TSA security incident report review to incidents
that occurred on commercial passenger aircraft in-flight. We defined
"in-flight" as the time between aircraft take-off and landing. Although it
is possible that there were some incidents involving small scissors or
tools that occurred during the time period of our review that were not
reported to TSA, and thus not recorded in the incident reports, we found
the incident reports sufficiently reliable for our purposes.
We conducted our work from November 2006 through March 2007 in accordance
with generally accepted government auditing standards.
Enclosure II
GAO Contact and Staff Acknowledgments
GAO Contact
Robert Goldenkoff (202) 512-2757 or [4][email protected] .
Staff Acknowledgments
Key contributors to this report were Maria Strudwick, Assistant Director;
David Alexander; Christopher Backley; Amy Bernstein; Tony Cheesebrough;
Adam Hoffman; Stanley Kostyla; Tom Lombardi; and Brian Sklar.
(440557)
References
Visible links
1. file:///home/webmaster/infomgt/d07623r.htm#http://www.gao.gov/cgi-bin/getrpt?GAO-07-634
*** End of document. ***