Aviation Security: Risk, Experience, and Customer Concerns Drive
Changes to Airline Passenger Screening Procedures, but Evaluation
and Documentation of Proposed Changes Could Be Improved
(16-APR-07, GAO-07-634).
The Transportation Security Administration's (TSA) most visible
layer of commercial aviation security is the screening of airline
passengers at airport checkpoints, where travelers and their
carry-on items are screened for explosives and other dangerous
items by transportation security officers (TSO). Several
revisions made to checkpoint screening procedures have been
scrutinized and questioned by the traveling public and Congress
in recent years. For this review, GAO evaluated (1) TSA's
decisions to modify passenger screening procedures between April
2005 and December 2005 and in response to the alleged August 2006
liquid explosives terrorist plot, and (2) how TSA monitored TSO
compliance with passenger screening procedures. To conduct this
work, GAO reviewed TSA documents, interviewed TSA officials and
aviation security experts, and visited 25 airports of varying
sizes and locations.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-07-634
ACCNO: A68225
TITLE: Aviation Security: Risk, Experience, and Customer
Concerns Drive Changes to Airline Passenger Screening Procedures,
but Evaluation and Documentation of Proposed Changes Could Be
Improved
DATE: 04/16/2007
SUBJECT: Airport security
Aviation security
Baggage screening
Commercial aviation
Documentation
Evaluation methods
Homeland security
Modifications
Passenger screening
Performance measures
Program evaluation
Risk assessment
Terrorism
Transportation policies
Policies and procedures
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-07-634
* [1]Results in Brief
* [2]Background
* [3]Passenger Checkpoint Screening System
* [4]Transportation Security Officers
* [5]Screening Technology
* [6]Standard Operating Procedures
* [7]TSA Considered Risk, Experience, and Customer Concerns when
* [8]TSA's Processes for Modifying SOPs Were Driven by Input from
* [9]Risk Factors, Day-to-Day Experiences, and Customer Concerns
* [10]TSA Balanced Security, Efficiency, and Customer Service when
* [11]TSA's Analysis of the Impact of Certain Proposed Screening C
* [12]Documentation of the Reasoning behind Proposed SOP Modificat
* [13]TSA Has Several Methods in Place to Monitor TSO Compliance w
* [14]A New Performance Accountability System Helps TSA Monitor TS
* [15]TSA Uses Local and National Covert Testing, in Part, to Asse
* [16]Conclusions
* [17]Recommendations for Executive Action
* [18]Agency Comments and Our Evaluation
* [19]GAO Contact
* [20]Acknowledgments
* [21]GAO's Mission
* [22]Obtaining Copies of GAO Reports and Testimony
* [23]Order by Mail or Phone
* [24]To Report Fraud, Waste, and Abuse in Federal Programs
* [25]Congressional Relations
* [26]Public Affairs
Report to Congressional Requesters
United States Government Accountability Office
GAO
April 2007
AVIATION SECURITY
Risk, Experience, and Customer Concerns Drive Changes to Airline Passenger
Screening Procedures, but Evaluation and Documentation of Proposed Changes
Could Be Improved
GAO-07-634
Contents
Letter 1
Results in Brief 4
Background 8
TSA Considered Risk, Experience, and Customer Concerns when Modifying
Passenger Screening Procedures, but Could Improve Its Evaluation and
Documentation of Proposed Procedures 15
TSA Has Several Methods in Place to Monitor TSO Compliance with Passenger
Checkpoint Screening SOPs 44
Conclusions 48
Recommendations for Executive Action 50
Agency Comments and Our Evaluation 50
Appendix I Objectives, Scope and Methodology 53
Appendix II Sources of SOP Changes 58
Appendix III Comments from the Department of Homeland Security 60
Appendix IV GAO Contact and Staff Acknowledgments 64
GAO Related Products 65
Tables
Table 1: Categories of Proposed and Implemented Passenger Checkpoint
Screening Changes Considered between April 2005 and December 2005 14
Table 2: Factors Considered by TSA When Deciding How to Modify Passenger
Checkpoint Screening SOPs in Response to the Alleged August 2006 Terrorist
Plot to Detonate Liquid Explosives on U.S.-Bound Aircraft 22
Table 3: Proposed Procedures Operationally Tested by the Explosives
Detection Improvement Task Force, October 2005-January 2006 31
Table 4: TSA Evaluation of Documentation of Agency Decisions Made between
August 7 and August 13, 2006, Regarding the Alleged Liquid Explosives
Terrorist Plot 43
Table 5: Modules Included in Recertification Knowledge and Skills
Assessment 46
Figures
Figure 1: Passenger Checkpoint Screening Functions 9
Figure 2: TSA Airport Screening Positions 11
Abbreviations
ATSA Aviation and Transportation Security Act
CAPPS computer-assisted passenger prescreening system
CBP Customs and Border Protection
COMPEX Compliance Examination
DHS Department of Homeland Security
ETD explosive trace detection
ETP explosives trace portal
FBI Federal Bureau of Investigation
FEMA Federal Emergency Management Agency
FFDO Federal Flight Deck Officer
FSD Federal Security Director
HHMD hand-held metal detector
IED improvised explosive device
PASS Performance Accountability and Standards System
PMIS Performance Management Information System
PWD person with disabilities
SOP standard operating procedure
SPOT Screening Passengers by Observation Technique
STEA Screener Training Exercises and Assessments
TIP Threat Image Projection
TSA Transportation Security Administration
TSO transportation security officer
USP Unpredictable Screening Process
WTMD walk-through metal detector
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office
Washington, DC 20548
April 16, 2007
The Honorable Bennie Thompson
Chairman, Committee on Homeland Security
House of Representatives
The Honorable John Mica
Ranking Republican Member
Committee on Transportation and Infrastructure
House of Representatives
The alleged August 2006 terrorist plot to detonate liquid explosives
onboard multiple commercial aircraft bound for the United States from the
United Kingdom has highlighted the continued importance of securing the
commercial aviation system. The Transportation Security Administration
(TSA) is responsible for, among other things, securing the nation's
commercial aviation system while also facilitating the movement of
passengers. To protect this system after the September 11, 2001, terrorist
attacks, TSA implemented a multilayered system of security--the most
publicly visible layer being the physical screening of passengers and
their carry-on bags at airport screening checkpoints, which all passengers
must pass through prior to entering an airport's sterile area, the area
within the terminal that provides passengers access to boarding aircraft.1
The passenger checkpoint screening system is composed of three elements:
(1) the people responsible for conducting the screening of airline
passengers and their carry-on items--Transportation Security Officers
(TSO) (formerly known as screeners), (2) the procedures TSOs are to follow
to conduct screening, and (3) the technology used in the screening
process. Collectively, these elements help to determine the effectiveness
and efficiency of passenger checkpoint screening. TSA has made efforts to
enhance each of the three elements of the passenger checkpoint screening
system.
Since its inception, TSA has issued 25 versions of the passenger
checkpoint screening standard operating procedures (SOP), to include new
screening procedures as well as changes to existing screening procedures.
Several of these revisions have been criticized and scrutinized by the
traveling public and Congress. For example, in September 2004, TSA
modified its passenger screening procedures in response to the August 2004
midair explosions of two Russian airliners, believed to have been caused
by Chechen women who concealed explosive devices under their clothing.
Specifically, the revision entailed a more invasive technique for patting
down the torso area of passengers. According to TSA officials, in response
to complaints raised by airline passengers and TSA's review of additional
threat information, TSA further modified the pat-down procedure in
December 2004 to entail a more targeted, less intrusive pat-down
procedure. In December 2005, TSA allowed passengers to carry small
scissors and small tools onto aircraft, resulting in concern by some
industry representatives that allowing sharp objects onto planes would put
flight crew at risk of attack.2 This procedural change also resulted in
the TSA Assistant Secretary being asked to testify before Congress on the
agency's rationale for allowing passengers to carry small scissors and
small tools onto planes and a legislative mandate for us to assess the
impact of the prohibited items list change on public safety and screening
operations.3
1In addition to passenger checkpoint screening, TSA's layers of aviation
security include, among other things, the screening of all checked baggage
for explosives and the deployment of Federal Air Marshals on designated
high-risk flights.
In light of the potential impact of changes to passenger checkpoint
screening procedures, you asked that we assess TSA's process for
determining whether and how screening procedures should be modified, to
include the creation of new screening procedures and changes to existing
screening procedures. Specifically, this report addresses the following
questions: (1) How and on what basis did TSA modify passenger screening
procedures and what factors guided the decisions to do so? (2) How does
TSA determine whether TSOs are complying with the standard procedures for
screening passengers and their carry-on items? In March 2007, we issued a
report that contained sensitive security information regarding TSA's
passenger checkpoint screening procedures, including the factors TSA
considered when modifying passenger screening procedures and TSA's efforts
to monitor TSO compliance with standard passenger screening procedures.
This report provides the results of our March 2007 report with sensitive
security information removed.
2Specifically, TSA modified the list of items prohibited and permitted on
aircraft by allowing metal scissors with pointed tips and a cutting edge
of 4 inches or less, as measured from the fulcrum, and small tools of 7
inches or less, including screwdrivers, wrenches, and pliers, to pass
through the passenger screening checkpoint. See 70 Fed. Reg. 72,930 (Dec.
8, 2005).
3We plan to issue a report on the impact of the prohibited items list
changes on public safety and screening operations later this year.
To obtain information on the process used to modify passenger checkpoint
screening procedures, we reviewed and analyzed available TSA documentation
on proposed procedure modifications that were considered between April
2005 and December 2005, as well as threat assessments and operational
studies that supported SOP modifications.4 We also reviewed and analyzed
similar documentation for proposed modifications considered between August
2006 and November 2006 in response to the alleged terrorist plot to
detonate liquid explosives onboard multiple aircraft en route from the
United Kingdom to the United States. We included modifications to
passenger checkpoint screening procedures related to this particular event
because they provided the most recent information available of TSA's
approach to modifying screening procedures in response to an immediate
perceived threat to civil aviation. To assess TSO compliance with standard
operating procedures, our work also involved a review of available
documentation, including guidance, checklists, and other evaluation tools
used by TSA. In addition, we met with TSA headquarters officials who were
involved in the process of determining whether proposed changes to
passenger checkpoint screening procedures should be implemented, and who
were responsible for overseeing efforts to monitor TSO compliance with
screening procedures. We also visited or conducted phone interviews with
staff at 25 airports, which we selected based on variation in size,
geographic location, and level of performance on compliance-related
assessments. At each airport, we interviewed Federal Security Directors
(FSD),5 members of their management teams, and TSOs with passenger
screening responsibilities. Information from these interviews cannot be
generalized to all airports nationwide. Two of the airports we visited
were also participants in TSA's Screening Partnership Program.6 We also
met with officials from the Department of Homeland Security (DHS) Science
and Technology Directorate as well as the Federal Bureau of Investigation
(FBI) to discuss the methodology and results of their liquid explosives
tests, which were used to support TSA's decisions to modify the SOP in
September 2006. We also interviewed five experts in the field of aviation
security to obtain their perspectives on TSA's approach for deciding
whether to implement proposed checkpoint screening procedures.7 We
compared TSA's approach for implementing and revising passenger checkpoint
screening procedures, and for monitoring TSO compliance, with the
Comptroller General's standards for internal control in the federal
government8 and with risk management guidance. We assessed the reliability
of the data we acquired from TSA regarding TSO compliance and found the
data to be sufficiently reliable for our purposes.
4We began our review period in April 2005 to coincide with TSA's
consideration of proposed SOP modifications related to the second major
revision of the passenger checkpoint screening SOP since TSA's inception.
5TSA security activities at airports are overseen by FSDs. Each FSD is
responsible for overseeing security activities, including passenger
screening, at one or more commercial airports. We visited or conducted
phone interviews with officials at 25 airports. However, we met with only
24 FSDs, as 1 FSD was responsible for 2 of the airports we visited.
We conducted our work from March 2005 through January 2007 in accordance
with generally accepted government auditing standards. More details about
the scope and methodology of our work are presented in appendix I.
Results in Brief
During our 9-month review period, proposed modifications to passenger
checkpoint screening procedures were made in various ways and for a
variety of reasons, and while a majority of the proposed modifications--
48 of 92--were ultimately implemented at airports, TSA's methods for
evaluating and documenting them could be improved. Some SOP modifications
were proposed based on the professional judgment of TSA senior-level
officials and program-level staff at headquarters and at airports
nationwide, while other modifications were proposed by members of a TSA
task force charged with enhancing TSA's ability to detect improvised
explosive devices at checkpoints. TSA officials proposed SOP modifications
based on risk information (threat and vulnerability information), daily
experiences of staff working at airports, and complaints and concerns
raised by the traveling public. In addition to these factors, TSA senior
leadership made efforts to balance the impact that proposed SOP
modifications--such as the changes to the liquids, gels, and aerosols
screening procedures--would have on security, efficiency, and customer
service when deciding whether proposed SOP modifications should be
implemented. In some cases, TSA first tested proposed modifications to
screening procedures at selected airports to help determine whether the
changes would achieve their intended purpose, such as to enhance detection
of prohibited items or to free up TSO resources to perform screening
activities focused on threats considered to pose a high risk, such as
explosives. However, TSA's data collection and analyses could be improved
to help TSA determine whether proposed procedures that are operationally
tested would achieve their intended purpose. Specifically, for the seven
tests of proposed screening procedures TSA conducted during our review
period, although TSA collected some data on the efficiency of and customer
response to the procedures at selected airports, the agency generally did
not collect the type of data or conduct the necessary analysis that would
yield information on whether proposed procedures would achieve their
intended purpose. TSA officials acknowledged that they could have made
some improvements in the various analyses they conducted related to the
decision to allow small scissors and tools onboard aircraft, but cited
several difficulties in doing so. Nevertheless, until TSA takes steps to
improve its ability to evaluate the potential impact of screening changes
on security and TSO resource availability, it may be difficult for TSA to
determine how best to allocate limited TSO resources, help ensure the
screeners' ability to detect explosives and other high-threat objects, and
evaluate whether proposed modifications to screening procedures would have
the intended effect. Finally, TSA's documentation on proposed
modifications to screening procedures was not always complete. TSA
documented the basis--that is, the information, experience, or event that
encouraged TSA officials to propose the modifications--for 72 of the 92
proposed modifications. In addition, TSA only documented the reasoning
behind its decisions for about half (26 of 44) of the proposed
modifications that were not implemented. Our standards for governmental
internal controls and associated guidance suggest that agencies should
document key decisions in a way that is complete and accurate. Without
such information, TSA cannot always justify significant SOP modifications
to Congress and the traveling public. TSA officials acknowledged that it
is beneficial to maintain documentation on the reasoning behind decisions
to implement or reject proposed SOP modifications deemed significant,
particularly given the organizational restructuring and staff turnover
within TSA.
6The Aviation and Transportation Security Act (ATSA), Pub. L. No. 107-71,
115 Stat. 597 (2001), established TSA and assigned TSA with the
responsibility of building a federal workforce to conduct screening of
airline passengers and their checked baggage. See 49 U.S.C. SS 114(a),
44901(a). ATSA also required that TSA allow commercial airports to apply
to TSA to transition from a federal to a private screener workforce. See
49 U.S.C S 44920. To support this effort, TSA created the Screening
Partnership Program to allow all commercial airports an opportunity to
apply to TSA for permission to use qualified private screening contractors
and private screeners. There are currently 6 airports participating in the
Screening Partnership Program, including Jackson Hole, Kansas City
International, Greater Rochester International, San Francisco
International, Sioux Falls Regional, and Tupelo Regional.
7We used the following criteria to identify aviation security experts:
present and past employment in aviation security, depth of experience in
aviation security, and recognition in the aviation industry.
8GAO, Internal Control: Standards for Internal Control in the Federal
Government, [27]GAO/AIMD-00-21.3.1 (Washington, D.C.: August 2001).
TSA monitors TSO compliance with passenger checkpoint screening SOPs
through its performance accountability and standards system and through
local and national covert testing.9 According to TSA officials, the agency
developed the performance accountability and standards system in response
to our 2003 report that recommended that TSA establish a performance
management system that makes meaningful distinctions in employee
performance,10 and in response to input from TSA airport staff on how to
improve passenger and checked baggage screening measures. This system is
used by TSA to measure TSO compliance with passenger checkpoint screening
procedures. Of the 24 FSDs we interviewed about compliance assessments, 9
cited difficulties in implementing the performance accountability and
standards system because of a lack of available staff to conduct
observations and administer SOP quizzes. When asked whether they planned
to address FSDs' concerns regarding a lack of available staff to evaluate
TSO compliance with SOPs, TSA headquarters officials said that they have
automated many of the data entry functions of the performance
accountability and standards system to relieve the field of the burden of
manually entering this information into the online system. Furthermore,
the TSA Assistant Secretary stated that FSDs were given the option of
delaying implementation of the performance accountability and standards
system if they were experiencing resource challenges. In addition to
implementing the performance accountability and standards system, TSA
conducts local and national covert tests to evaluate, in part, the extent
to which TSOs' noncompliance with the SOPs affects their ability to detect
simulated threat items hidden in accessible property or concealed on a
person.11 Even though all 24 FSDs said that they have conducted local
covert tests, 10 of these FSDs said that lack of available staff made it
difficult to conduct these tests. TSA officials told us that they are
considering resource alternatives for implementing these tests, but did
not provide us with specific details of these plans. Based on the results
of national covert tests conducted between September 2005 and July 2006,
which showed that some TSOs did not identify threat objects, in part
because they did not comply with SOPs, TSA's Office of Inspection
recommended, among other things, that the Office of Security Operations
ensure that TSOs adhere to the current passenger checkpoint screening
SOPs. However, until the resource limitations that have restricted TSA's
use of its compliance monitoring tools have been fully addressed, TSA may
not have assurance that TSOs are screening passengers according to
standard procedures.
9Covert testing involves TSA headquarters officials (national testing) or
TSA field staff and other federal employees (local testing) attempting to
carry simulated threat objects through the checkpoint without the objects
being detected by TSOs. The results of the national covert tests are
classified and therefore are not included in this report.)
10GAO, Transportation Security Administration: Actions and Plans to Build
a Results Oriented Culture, [28]GAO-03-190 (Washington, D.C.: January
2003).
To help improve TSA's ability to evaluate proposed SOP modifications and
to justify its decisions regarding whether proposed SOP modifications
should be implemented, in the March 2007 report that contained sensitive
security information, we recommended that the Secretary of the Department
of Homeland Security direct the Assistant Secretary of Homeland Security
for TSA to (1) develop sound evaluation methods, when possible, that can
help TSA determine whether proposed procedures that are operationally
tested would achieve their intended purpose, and (2) generate and maintain
complete documentation of proposed modifications deemed significant by
TSA. DHS generally concurred with our findings and recommendations and
outlined actions TSA plans to take to implement the recommendations. For
example, TSA intends to improve its methods for evaluating proposed SOP
modifications, which may entail randomly selecting the airports that will
participate in a study to better isolate the impact of proposed SOP
modifications on passenger screening. DHS also stated that TSA is in the
process of developing protocols that will require documentation of the
source and intent of proposed SOP modifications, as well as documentation
of TSA officials' reasoning for implementing or rejecting proposed
modifications. The full text of DHS's comments is included in appendix
III.
11The results of local covert testing are sensitive security information
and, therefore, are not included in this report.
Background
Passenger Checkpoint Screening System
Passenger screening is a process by which personnel authorized by TSA
inspect individuals and property to deter and prevent the carriage of any
unauthorized explosive, incendiary, weapon, or other dangerous item
onboard an aircraft or into a sterile area.12 Passenger screening
personnel must inspect individuals for prohibited items at designated
screening locations.13 As shown in figure 1, the four passenger screening
functions are
o X-ray screening of property,
o walk-through metal detector screening of individuals,
o hand-wand or pat-down screening of individuals, and
o physical search of property and trace detection for explosives.
Typically, passengers are only subjected to X-ray screening of their
carry-on items and screening by the walk-through metal detector.
Passengers whose carry-on baggage alarms the X-ray machine, who alarm the
walk-through metal detector, or who are designated as selectees--that is,
passengers selected by the Computer-Assisted Passenger Prescreening System
(CAPPS)14 or other TSA-approved processes to receive additional
screening--are screened by hand-wand or pat-down and have their carry-on
items screened for explosives traces or physically searched.
12Sterile areas are located within the terminal where passengers are
provided access to boarding aircraft. Access to these areas is controlled
by Transportation Security Officers (or by nonfederal screeners at
airports participating in the Screener Partnership Program) at checkpoints
where they conduct physical screening of individuals and their carry-on
baggage for weapons, explosives, and other prohibited items.
13Transportation Security Officers must deny passage beyond the screening
location to any individual or property that has not been screened or
inspected in accordance with passenger screening standard operating
procedures. If an individual refuses to permit inspection of any item,
that item must not be allowed into the sterile area or onboard an
aircraft.
14CAPPS is a computer-assisted system that, based on information obtained
from airline reservation systems, identifies passengers that may pose a
high risk to aviation security. These high-risk passengers and their
carry-on baggage are subject to additional and more thorough screening.
Figure 1: Passenger Checkpoint Screening Functions
Note: Explosive trace detection (ETD) works by detecting vapors and
residues of explosives. Human operators collect samples by rubbing swabs
along the interior and exterior of an object that TSOs determine to be
suspicious, and place the swabs in the ETD machine, which then chemically
analyzes the swab to identify any traces of explosive materials.
Bomb Appraisal Officers (BAO) are available to respond to unresolved
alarms at the checkpoint that involve possible explosive devices. The BAO
may contact appropriate law enforcement or bomb squad officials if review
indicates possible or imminent danger, in which case the BAO ensures that
the security checkpoint is cleared. The BAO approves reopening of security
lane(s) if no threat is posed.
aBDOs are TSOs specially trained to detect suspicious behavior in
individuals approaching the checkpoint. Should the BDO observe such
behavior, he or she may refer the individual for additional screening or
to a law enforcement officer.
bThe hand-wand or pat-down is conducted if a passenger is identified or
randomly selected for additional screening because he or she met certain
criteria or alarmed the walk-through metal detector.
cManual or ETD searches of accessible property occur if the passenger is
identified or randomly selected for additional screening or if the
screener identified a potential prohibited item on X-ray.
The passenger checkpoint screening system is composed of three elements:
the people responsible for conducting the screening of airline passengers
and their carry-on items--TSOs, the technology used during the screening
process, and the procedures TSOs are to follow to conduct screening.
Collectively, these elements help to determine the effectiveness and
efficiency of passenger checkpoint screening.
Transportation Security Officers
TSOs screen all passengers and their carry-on baggage prior to allowing
passengers access to their departure gates. There are several positions
within TSA that perform and directly supervise passenger screening
functions. Figure 2 provides a description of these positions.
Figure 2: TSA Airport Screening Positions
aNumber of annualized TSA screening positions for fiscal year 2006. These
positions do not include private screener positions at the six airports
that participated in the Screening Partnership Program during fiscal year
2006.
In May 2005, we reported on TSA's efforts to train TSOs and to measure and
enhance TSO performance.15 We found that TSA had initiated a number of
actions designed to enhance passenger TSO, checked baggage TSO, and
supervisory TSO training. However, at some airports TSOs encountered
difficulty accessing and completing recurrent (refresher) training because
of technological and staffing constraints. We also found that TSA lacked
adequate internal controls to provide reasonable assurance that TSOs were
receiving legislatively mandated basic and remedial training, and to
monitor the status of its recurrent training program. Further, we reported
that TSA had implemented and strengthened efforts to collect TSO
performance data as part of its overall effort to enhance TSO performance.
We recommended that TSA develop a plan for completing the deployment of
high-speed Internet/intranet connectivity to all TSA airport training
facilities, and establish appropriate responsibilities and other internal
controls for monitoring and documenting TSO compliance with training
requirements. DHS generally concurred with our recommendations and stated
that TSA has taken steps to implement them.
Screening Technology
There are typically four types of technology used to screen airline
passengers and their carry-on baggage at the checkpoint:
o walk-through metal detectors,
o X-ray machines,
o hand-held metal detectors, and
o explosive trace detection (ETD) equipment.
The President's fiscal year 2007 budget request noted that emerging
checkpoint technology will enhance the detection of prohibited items,
especially firearms and explosives, on passengers. As of December 2006,
TSA plans to conduct operational tests of three types of passenger
screening technologies within the next year. TSA has conducted other tests
in the past; for example, during fiscal year 2005, TSA operationally
tested document scanners, which use explosive trace detection technology
to detect explosives residue on passengers' boarding passes or
identification cards. TSA decided not to expand the use of the document
scanner, in part because of the extent to which explosives traces had to
be sampled manually. TSA also plans to begin operational tests of
technology that would screen bottles for liquid explosives. We are
currently evaluating the Department of Homeland Security's and TSA's
progress in planning for, managing, and deploying research and development
programs in support of airport checkpoint screening operations. We expect
to report our results in August 2007.
15GAO, Aviation Security: Screener Training and Performance Measurement
Strengthened, but More Work Remains, [29]GAO-05-457 (Washington, D.C.: May
2, 2005).
Standard Operating Procedures
TSA has developed checkpoint screening standard operating procedures,
which are the focus of this report, that establish the process and
standards by which TSOs are to screen passengers and their carry-on items
at screening checkpoints.16 Between April 2005 and December 2005, based on
available documentation, TSA deliberated 189 proposed changes to passenger
checkpoint screening SOPs, 92 of which were intended to modify the way in
which passengers and their carry-on items are screened.17 TSA issued six
versions of the passenger checkpoint screening SOPs during this period.18
TSA modified passenger checkpoint screening SOPs to enhance the traveling
public's perception of the screening process, improve the efficiency of
the screening process, and enhance the detection of prohibited items and
suspicious persons. As shown in table 1, 48 of the 92 proposed
modifications to passenger checkpoint screening SOPs were implemented, and
the types of modifications made or proposed generally fell into one of
three categories--customer satisfaction, screening efficiency, and
security.
16Private screeners conduct passenger and checked baggage screening at six
airports as part of TSA's Screening Partnership Program. TSA requires that
private screeners screen passengers using the same standard operating
procedures as TSOs.
17Between April 2005 and December 2005, TSA considered a total of 189
proposed modifications to passenger checkpoint screening SOPs. However, 97
of the proposed modifications were not intended to alter the way in which
passengers and their carry-on items are screened; rather, these
modifications were generally intended to correct, edit, or clarify SOP
language. For example, TSA modified SOP language to ensure that TSA field
staff were aware that tribal law enforcement officers should be granted
the same screening exemptions as other law enforcement officers. TSA also
amended the SOP to help ensure the occupational safety of TSOs. For
example, TSA headquarters officials proposed that procedures for reporting
potential radiation hazards regarding X-ray equipment be incorporated into
the SOP. The remaining 92 proposed SOP modifications were intended to
alter the way in which passengers and their carry-on items were screened,
and 48 of those proposed modifications were subsequently implemented.
18TSA issued six revised versions of the passenger checkpoint screening
SOP during the 9-month period under review: April 7, 2005; July 7, 2005;
August 26, 2005; September 12, 2005; October 25, 2005; and December 7,
2005. However, we did not include the April 2005 revised SOP in our review
since the changes incorporated in that revision were deliberated by TSA
officials outside of our 9-month period of review.
Table 1: Categories of Proposed and Implemented Passenger Checkpoint
Screening Changes Considered between April 2005 and December 2005
Category of Proposed SOP Implemented SOP
proposed changes Description of category changes changes
Customer Changes that will improve 42 22
satisfaction the traveling public's
perception of the screening
process or reduce or exempt
categories of authorized
individuals from certain
aspects of the screening
process.
Screening Changes that will improve 31 17
efficiency screening flow, clarify
screener duties, update
equipment procedures, or
enhance the working
environment of screening
locations.
Security Changes that will improve 19 9
TSA's ability to detect
prohibited items and
suspicious persons.
Total 92 48
Source: GAO analysis of TSA data.
TSA Considered Risk, Experience, and Customer Concerns when Modifying Passenger
Screening Procedures, but Could Improve Its Evaluation and Documentation of
Proposed Procedures
TSA used various processes between April 2005 and December 2005 to modify
passenger checkpoint screening SOPs, and a variety of factors guided TSA's
decisions to modify SOPs. TSA's processes for modifying SOPs generally
involved TSA staff recommending proposed modifications, reviewing and
commenting on proposed modifications, and TSA senior leadership making
final decisions as to whether proposed modifications should be
implemented. During our 9-month review period, TSA officials considered 92
proposed modifications to the way in which passengers and their carry-on
items were screened, and 48 were implemented.19 TSA officials proposed SOP
modifications based on risk factors (threat and vulnerability
information), day-to-day experiences of airport staff, and concerns and
complaints raised by passengers. TSA then made efforts to balance
security, efficiency, and customer service when deciding which proposed
SOP modifications to implement. Consistent with our prior work that has
shown the importance of data collection and analyses to support agency
decision making, TSA conducted data collection and analysis for certain
proposed SOP modifications that were tested before they were implemented
at all airports. Nevertheless, we found that TSA could improve its data
collection and analysis to assist the agency in determining whether the
proposed procedures would enhance detection or free up TSO resources, when
intended. In addition, TSA did not maintain complete documentation of
proposed SOP modifications; therefore, we could not fully assess the basis
for proposed SOP modifications or the reasons why certain proposed
modifications were not implemented. TSA officials acknowledged that it is
beneficial to maintain documentation on the reasoning behind decisions to
implement or reject SOP modifications deemed significant.
TSA's Processes for Modifying SOPs Were Driven by Input from TSA Field and
Headquarters Staff
Proposed SOP modifications were submitted and reviewed under two processes
during our 9-month review period, and for each process, TSA senior
leadership made the final decision as to whether the proposed
modifications would be implemented. One of the processes TSA used to
modify passenger checkpoint screening SOPs involved TSA field staff or
headquarters officials, and, to a lesser extent, TSA senior leadership,
suggesting ways in which passenger checkpoint screening SOPs could be
modified. These suggestions were submitted through various mechanisms,
including electronic mail and an SOP panel review conducted by TSA airport
personnel. (These methods are described in more detail in app. II.)
Eighty-two of the 92 proposed modifications were considered under this
process.
19Of the 48 proposed modifications that were implemented, TSA made the
decision to implement 16 of these modifications following our 9-month
review period. However, because much of TSA's deliberation of these 16
procedures occurred during our review period, we included these procedures
among those that were implemented.
If TSA officials determined, based on their professional judgment, that
the recommended SOP modifications--whether from headquarters or the
field--merited further consideration, or if a specific modification was
proposed by TSA senior leadership, the following chain of events occurred:
o First, the procedures branch of the Office of Security
Operations drafted SOP language for each of the proposed
modifications.20
o Second, the draft language for each proposed modification was
disseminated to representatives of various TSA divisions for
review, and the language was revised as needed.
o Third, TSA officials tested proposed modifications in the
airport operating environment if they found it necessary to:
o assess the security impact of the proposed
modification,
o evaluate the impact of the modification on the
amount of time taken for passengers to clear the
checkpoint,
o measure the impact of the proposed modification on
passengers and industry partners, or
o determine training needs created by the proposed
modification.
o Fourth, the revised SOP language for proposed modifications was
sent to the heads of several TSA divisions for comment.
o Fifth, considering the comments of the TSA division heads, the
head of the Office of Security Operations or other TSA senior
leadership made the final decision as to whether proposed
modifications would be implemented.
Another process for modifying passenger checkpoint screening SOPs during
our 9-month review period was carried out by TSA's Explosives Detection
Improvement Task Force. The task force was established in October 2005 by
the TSA Assistant Secretary to respond to the threat of improvised
explosive devices (IED) being carried through the checkpoint. The goal of
the task force was to apply a risk-based approach to screening passengers
and their baggage in order to enhance TSA's ability to detect IEDs.21 The
task force developed 13 of the 92 proposed SOP modifications that were
considered by TSA between April 2005 and December 2005.22 The task force
solicited and incorporated feedback from representatives of various TSA
divisions on these proposed modifications and presented them to TSA senior
leadership for review and approval. TSA senior leadership decided that 8
of the 13 proposed modifications should be operationally tested--that is,
temporarily implemented in the airport environment for the purposes of
data collection and evaluation--to better inform decisions regarding
whether the proposed modifications should be implemented. Following the
testing of these proposed modifications in the airport environment, TSA
senior leadership decided to implement 7 of the 8 operationally tested
changes.23 (The task force's approach to testing these procedures is
discussed in more detail below.) Following our 9-month period of review,
the changes that TSA made to its passenger checkpoint screening SOPs in
response to the alleged August 2006 liquid explosives terror plot were
decided upon by DHS and TSA senior leadership, with some input from TSA
field staff, aviation industry representatives, and officials from other
federal agencies.
20The Office of Security Operations is the TSA division responsible for
overseeing the implementation of passenger and property screening at
airport checkpoints.
Risk Factors, Day-to-Day Experiences, and Customer Concerns Were the Basis for
Proposed SOP Modifications
Based on available documentation,24 risk factors (i.e., threats to
commercial aviation and vulnerability to those threats), day-to-day
experiences of airport staff, and complaints and concerns raised by
passengers were the basis for TSA staff and officials proposing
modifications to passenger checkpoint screening SOPs.
Fourteen of the 92 procedure modifications recommended by TSA staff and
officials were based on reported or perceived threats to commercial
aviation, and existing vulnerabilities to those threats. For example, the
Explosives Detection Improvement Task Force proposed SOP modifications
based on threat reports developed by TSA's Intelligence and Analysis
division. Specifically, in an August 2005 civil aviation threat
assessment, the division reported that terrorists are likely to seek novel
ways to evade U.S. airport security screening.25 Subsequently, the task
force proposed that the pat-down procedure performed on passengers
selected for additional screening be revised to include not only the torso
area, which is what the previous pat-down procedure entailed, but
additional areas of the body such as the legs.26 The August 2005 threat
assessment also stated that terrorists may attempt to carry separate
components of an IED through the checkpoint, then assemble the components
while onboard the aircraft. To address this threat, the task force
proposed a new procedure to enhance TSOs' ability to search for components
of improvised explosive devices. According to TSA officials, threat
reports have also indicated that terrorists rely on the routine nature of
security measures in order to plan their attacks. To address this threat,
the task force proposed a procedure that incorporated unpredictability
into the screening process by requiring designated TSOs to randomly select
passengers to receive additional search procedures. Following our 9-month
review period, TSA continued to use threat information as the basis for
proposed modifications to passenger checkpoint screening SOPs. In August
2006, TSA proposed modifications to passenger checkpoint screening SOPs
after receiving threat information regarding an alleged terrorist plot to
detonate liquid explosives onboard multiple aircraft en route from the
United Kingdom to the United States. Regarding vulnerabilities to reported
threats, based on the results of TSA's own covert tests (undercover,
unannounced tests), TSA's Office of Inspection recommended27 SOP
modifications to enhance the detection of explosives at the passenger
screening checkpoint.28
21In order to achieve its goal of improving IED detection, in addition to
modifying passenger checkpoint screening SOPs, the task force established
several initiatives, including enhanced bomb detection training for TSOs
and increased use of explosives detection canine teams.
22Three of the 92 proposed SOP modifications were considered by TSA under
both processes.
23The number of airports at which any one proposed change was pilot tested
ranged from 3 to 14, and the duration of the pilot testing ranged from 5
days to several weeks.
24The number of proposed SOP modifications that fall under the various
"basis" categories (e.g., threat and vulnerability information) does not
total 92 because documentation was not available for all proposed
modifications and some of the proposed modifications had more than one
basis.
25We did not assess the quality of the intelligence information used by
TSA's Office of Intelligence and Analysis to generate its civil aviation
threat assessments.
26The pat-down procedure is performed for three purposes: (1) as a
substitute for walk-through metal detector screening, (2) to resolve
walk-through metal detector alarms, and (3) as a standard procedure for
screening passengers selected for additional screening. The details of the
pat-down procedures are sensitive security information and are not
discussed in this report.
TSA officials also proposed modifications to passenger checkpoint
screening SOPs based on their professional judgment regarding perceived
threats to aviation security. For example, an FSD recommended changes to
the screening of funeral urns based on a perceived threat. In some cases,
proposed SOP modifications appeared to reflect threat information analyzed
by TSA officials. For example, TSOs are provided with Threat in the
Spotlight, a weekly report that identifies new threats to commercial
aviation, examples of innovative ways in which passengers may conceal
prohibited items, and pictures of items that may not appear to be
prohibited items but actually are. TSOs are also provided relevant threat
information during briefings that take place before and after their
shifts. In addition, FSDs are provided classified intelligence summaries
on a daily and weekly basis, as well as monthly reports of suspicious
incidents that occurred at airports nationwide. TSA's consideration of
threat and vulnerability--through analysis of current documentation and by
exercising professional judgment--is consistent with a risk-based
decision-making approach.29 As we have reported previously, and DHS and
TSA have advocated, a risk-based approach, as applied in the homeland
security context, can help to more effectively and efficiently prepare
defenses against acts of terrorism and other threats.
TSA headquarters and field staff also based proposed SOP
modifications--specifically, 36 of the 92 proposed modifications--on
experience in the airport environment. For example, TSA headquarters
officials conduct reviews at airports to identify best practices and
deficiencies in the checkpoint screening process. During one of these
reviews, headquarters officials observed that TSOs were not fully
complying with the pat-down procedure. After discussions with TSOs, TSA
headquarters officials determined that the way in which TSOs were
conducting the procedure was more effective. In addition, TSA senior
leadership, after learning that small airports had staffing challenges
that precluded them from ensuring that passengers are patted down by TSOs
of the same gender, proposed that opposite-gender pat-down screening be
allowed at small airports.
27The recommendations made by the Office of Inspection are sensitive
security information or classified information. Therefore, they are not
discussed in this report.
28The DHS Office of Inspector General conducts similar covert tests, and
historically has recommended changes to the passenger checkpoint screening
SOP as a result of these tests. However, the Office of Inspector General
did not make any recommendations that resulted in procedural changes
between April 2005 and December 2005.
29A risk-based approach generally involves consideration of the following
when making decisions: threat--capability and intent of terrorists to
carry out an attack, vulnerability--weakness that may be exploited by
identified threats, and criticality or consequence--the impact of an
attack if it were to be carried out.
Passenger complaints and concerns shared with TSA also served as a basis
for proposed modifications during our 9-month review period. Specifically,
of the 92 proposed SOP modifications considered during this period, TSA
staff and officials recommended 29 modifications based on complaints and
concerns raised by passengers. For example, TSA headquarters staff
recommended allowing passengers to hold their hair while being screened by
the Explosives Trace Portal,30 after receiving complaints from passengers
about eye injuries from hair blowing in their eyes and hair being caught
in the doors of the portal.
TSA Balanced Security, Efficiency, and Customer Service when Deciding whether to
Implement Proposed SOP Modifications
When deciding whether to implement proposed SOP modifications, TSA
officials also made efforts to balance the impact of proposed
modifications on security, efficiency,31 and customer service. TSA's
consideration of these factors reflects the agency's mission to protect
transportation systems while also ensuring the free movement of people and
commerce. As previously discussed, TSA sought to improve the security of
the commercial aviation system by modifying the SOP for conducting the
pat-down search. (TSA identified the modified pat-down procedure as the
"bulk-item" pat-down.) When deciding whether to implement the proposed
modification, TSA officials considered not only the impact that the
bulk-item pat-down procedure would have on security, but also the impact
that the procedure would have on screening efficiency and customer
service. For example, TSA officials determined that the bulk-item pat-down
procedure would not significantly affect efficiency because it would only
add a few seconds to the screening process. Following our 9-month review
period, TSA continued to make efforts to balance security, efficiency, and
customer service when deciding whether to implement proposed SOP
modifications, as illustrated by TSA senior leadership's deliberation on
proposed SOP modifications in response to the alleged August 2006 liquid
explosives terrorist plot. TSA modified the passenger checkpoint screening
SOP four times between August 2006 and November 2006 in an effort to
defend against the threat of terrorists' use of liquid explosives onboard
commercial aircraft.32 While the basis for these modifications was to
mitigate risk, as shown in table 2, TSA senior leadership considered
several other factors when deciding whether to implement the
modifications.
30Explosives Trace Portal screening entails a passenger stepping into the
portal, after which puffs of air are emitted onto the passenger. The
portal then draws in any residue that was loosened as a result of the
puffs of air, and analyzes the residue to determine if there are explosive
traces.
31TSA defines SOP modifications related to efficiency as changes that will
improve screening flow, clarify TSO duties, update equipment procedures,
or enhance the working environment of screening locations.
32The SOP modifications made by TSA on August 10, 2006, August 12, 2006,
September 26, 2006, and November 21, 2006, were designed to address only
one particular hydrogen peroxide-based liquid explosives mixture, which,
according to TSA officials, was the same mixture that the alleged
terrorists had planned to detonate on U.S.-bound flights originating in
the United Kingdom. DHS and FBI have identified additional liquid
explosives mixtures that could pose a threat to commercial aviation. DHS
has ongoing evaluations of the additional mixtures to determine their
explosive potential and the extent of damage that detonation of these
mixtures could cause to an aircraft. DHS is also evaluating explosives
detection technology to determine the extent to which it can be used at
the checkpoint to defend against the liquid explosives threat. We are
currently evaluating DHS's and TSA's progress in planning for, managing,
and deploying research and development programs in support of airport
checkpoint screening operations. We expect to report on our results in
August 2007.
Table 2: Factors Considered by TSA When Deciding How to Modify Passenger
Checkpoint Screening SOPs in Response to the Alleged August 2006 Terrorist
Plot to Detonate Liquid Explosives on U.S.-Bound Aircraft
Procedures Impact on security
August 10, 2006: Total ban on liquids and Benefits
gels in accessible property or onboard
aircraft. Exceptions: o Terrorists less likely to
successfully carry liquid
o baby formula/milk if infant is explosives onto aircraft
traveling; using container.
o prescription medication with name o Terrorists less likely to
matching passenger's ticket; successfully carry liquid
o insulin and other essential explosives onto aircraft in
nonprescription medications; shoes (e.g., gel-based
o liquids and gels carried by passengers insoles).
with disabilities, after screening for
explosive materials, with Supervisory Drawbacks
TSO/Screening Manager concurrence;
o supplies brought into retail area by o None identified
approved vendors for restocking of
retail operations.
Passengers required to remove shoes at
checkpoints for X-ray screening.
August 12, 2006a: Aerosols prohibited. Benefits
Following additional items allowed past
checkpoints: o Terrorists less likely to
successfully carry liquid
o baby food in small containers, if explosives onto aircraft
baby/small child is traveling; using container.
o essential nonprescription medications o Terrorists less likely to
(e.g., contact lens saline solution, eye successfully carry liquid
care products), not to exceed 4 fluid explosives onto aircraft in
ounces per container; shoes (e.g., gel-based
o liquids and gels for diabetic insoles).
passengers, no greater than 8 fluid
ounces per container; Drawbacks
o gels, saline solutions, and other
liquids used to augment portions of body o None identified
for medical/cosmetic reasons;
o life support/life sustaining liquids
(e.g., bone marrow and blood products.
Impact on efficiency of Impact on customer
screening process service Other considerations
Benefits Benefits o Threat was a specific
type of liquid
o Requiring passengers o Exceptions explosive.
to remove footwear allow passengers o There was no
will speed the with legitimate checkpoint screening
screening process by medical and other technology available for
reducing the need to needs to bring deployment that could
ETD and physically essential liquids detect the specific
inspect footwear. onboard aircraft. liquid explosive.
Footwear now only o Passengers less
needs to be subjected confused about
to physical search if whether to remove
something suspicious shoes for X-ray
appears on the X-ray screening.
of the shoes.
Drawbacks
Drawbacks
o Inconvenient
o Total ban on liquids for passengers to
and gels may be not be able to
unsustainable for long carry toiletries
term because more and similar
passengers would check liquids and gels
their baggage rather onto planes.
than carry it on,
which would cause a
strain on the checked
baggage screening
system.
Benefits Benefits o Feedback from TSA
field staff and industry
o Requiring passengers o Clarified for representatives
to remove footwear TSOs types and regarding exemptions
will speed the amounts of associated with liquids,
screening process by liquids and gels gels, and aerosols
reducing the need to exempt from ban. restrictions and
ETD and physically o Created specific information on
inspect footwear. smoother process the quantities of
Footwear now only at checkpoint, certain types of
needs to be subjected minimizing impact liquids, gels, and
to physical search if upon travelers. aerosols that should be
something suspicious o Gave diabetic exempted from the
appears on the X-ray passengers access restrictions.
of the shoes. to essential o Additional information
liquids. was obtained about the
Drawbacks o Lifted alleged terrorist plot,
prohibition including information
o Total ban on liquids against critical from the United Kingdom
and gels may be life saving and U.S. intelligence
unsustainable for long fluids. communities and
term because more discussions with
passengers would check Drawbacks explosives experts.
their baggage rather
than carry it on, o None identified
which would cause a
strain to the checked
baggage screening
system.
Procedures Impact on security
September 26, 2006: Liquids, gels, Benefits
aerosols (not on prohibited items
list or considered hazardous o Plastic bags present deterrent
materials) permitted in accessible and operational complexities for
property in 3-fluid ounce bottles terrorists--attempts to combine
fit comfortably in one quart-size, liquids increase probability of
clear plastic, zip-top bag per detection.
passenger. Plastic bags screened by o Requirement to remove and submit
X-ray. Items purchased in sterile plastic bags for X-ray screening
area of airports permitted onboard serves as deterrent to terrorists,
aircraft. and provides TSOs opportunity to
view and examine all liquids,
Items allowed past checkpoints in gels, and aerosols.
amounts larger than 3 fluid ounces; o Plastic bags hinder terrorists
must be declared and cleared by TSO: from carrying large enough amounts
of liquid explosives that could
o baby formula/milk /food in potentially cause catastrophic
small containers if baby/small damage to an aircraft.
child is traveling, o Declaration process thought to
o medications (liquid, gel, deter terrorists from attempting
aerosol), to carry liquid explosives onboard
o liquids and gels for passengers aircraft.
indicating need to address o Random ETD sampling enables TSOs
diabetic or other medical to determine whether the small
condition. amounts of liquids and gels being
carried through the checkpoint
TSOs conducting declaration process are, in fact, explosives. This
positioned ahead of checkpoint to procedure may also deter
assess liquids, gels, and aerosols terrorists from attempting to
to determine reasonable quantity for carry liquid explosives onboard
passenger's itinerary, and to advise aircraft.
passengers on procedures related to
liquids, gels and aerosols that are Drawbacks
either prohibited (requiring
disposal or abandonment of items) or o Possibility that terrorists
permitted but outside of plastic bag could combine liquids in small
(TSO marks boarding pass or travel bottles to generate an amount
document to indicate items). large enough to potentially cause
catastrophic damage to an
Items newly permitted past aircraft.
checkpoints in any amount: o The additional drawbacks related
to the impact on security are
o liquid, gel, and aerosol sensitive security information.
cleaning supplies required by Therefore, we do not discuss those
airport employees servicing drawbacks in this report.
sterile area,
o gels and frozen liquids
required to cool any other items
permitted past checkpoints,
provided no unresolved suspicious
items or activities.
Random ETD sampling of plastic bags,
containers within plastic bags, and
other containers holding liquids,
gels, and aerosols.
Impact on efficiency of Impact on customer
screening process service Other considerations
Benefits Benefits o The results of liquid
explosives tests
o Enables TSOs to o Procedures easily conducted by DHS and the
focus resources on learned by public FBI. The results of
detecting and TSOs. these tests are
explosives, rather o Accommodates many sensitive security
than small amounts passengers with information and are not
of liquids and gels legitimate needs discussed in this
that do not for small report.
represent serious quantities of o TSA gathered data to
threat. liquids during test its assumption
o Checked baggage flights. regarding sustainability
screening expected of the total ban on
to return to Drawbacks liquids, gels, and
sustainable levels. aerosols and found that
o Requirement to o Possible negative following the total ban,
remove and submit public reaction to there was approximately
plastic bags for passengers having a 27 percent increase in
X-ray screening to provide their the number of bags
encourages own plastic bags. checked per passenger.
passengers to reduce
clutter in bags,
making it easier for
TSOs to screen for
prohibited and
threat items.
Drawbacks
o Increase in number
of items X-rayed per
passenger, which may
slow down screening
process.
Procedures Impact on security
November 21, 2006: Same as the procedures Benefits
implemented on September 26, 2006, with the
exception of the following: No additional security
benefits identified.
Liquids, gels, and aerosols allowed in
3.4-fluid-ounce (100-milliliter) "travel size" Drawbacks
bottles.
No additional drawbacks
Declaration process eliminated; TSA employee ahead to security identified.
of checkpoint offers public advisements and
assessments on procedures.
(Additional modifications were made to the
liquids, gels, and aerosols screening procedures.
However, these additional modifications are
sensitive security information. Therefore, we do
not discuss these modifications in this report.)
Source: GAO analysis of TSA documentation
aThe August 12, 2006, SOP change incorporates clarifications implemented
on August 16, 2006.
Impact on efficiency of Impact on customer
screening process service Other considerations
Benefits Benefits o The European Union
allowed passengers to
o Elimination of the o Allowing for carry liquids, gels,
declaration process risk-based and aerosols in travel
will reduce discretion on the sized containers up to
unnecessary redundancy part of Supervisory 100 milliliters,
in the examination of TSOs enhances approximately 3.4
exempted liquids and customer service fluid ounces.
gels, which previously for passengers who o The results of
occurred both prior to have legitimate liquid explosive
and following x-ray reasons for testing conducted by
screening. carrying liquids, FBI and DHS.
gels, or aerosols o TSA recognized that
(TSA identified onboard planes. no procedure could be
additional efficiency written to address
benefits of this Drawbacks every possible
modification to the scenario involving
liquids, gels, and No additional customer liquids, gels, and
aerosols screening service drawbacks aerosols. Therefore,
procedures. These identified. TSA enabled
additional benefits are Supervisory TSOs to
sensitive security use their discretion,
information. Therefore, while also considering
we do not discuss these security risks.
benefits in this report.)
Drawbacks
No additional efficiency
drawbacks identified.
As TSA senior leadership obtained more information about the particular
threat posed by the liquid explosives through tests conducted by DHS's
Science and Technology Directorate and FBI, TSA relaxed the restrictions
to allow passengers to carry liquids, gels, and aerosols onboard aircraft
in 3-fluid-ounce bottles--and as of November 2006, 3.4-fluid-ounce
bottles--that would easily fit in a quart-sized, clear plastic, zip-top
bag. TSA senior leadership identified both benefits and drawbacks to this
SOP modification, but determined that the balance of security, efficiency,
and customer service that would result from these SOP changes was
appropriate. As shown in table 2, TSA officials recognize that there are
security drawbacks--or vulnerabilities--associated with allowing
passengers to carry even small amounts of liquids and gels onboard
aircraft. For example, two or more terrorists could combine small amounts
of liquid explosives after they pass through the checkpoint to generate an
amount large enough to possibly cause catastrophic damage to an aircraft.
However, TSA officials stated that doing so would be logistically
challenging given the physical harm that the specific explosives could
cause to the person handling them, and that suspicion among travelers, law
enforcement officials, and airport employees would likely be raised if an
individual was seen combining the liquid contents of small containers
stored in two or more quart-sized plastic bags. TSA officials stated that
at the time of the modifications to the liquid, gels, and aerosols
screening procedures, there was consensus among explosives detection
experts, both domestically and abroad, regarding TSA's assumptions about
how the explosives could be used and the damage they could cause to an
aircraft.33 TSA officials also stated that after reviewing the
intelligence information related to the alleged August 2006 London terror
plot--particularly with regard to the capability and intent of the
terrorists--TSA determined that allowing small amounts of liquids, gels,
and aerosols onboard aircraft posed an acceptable level of risk to the
commercial aviation system.34 Moreover, TSA officials acknowledged that
there are vulnerabilities with allowing passengers to carry liquids that
are exempted from the 3.4-fluid-ounce limit--such as baby formula and
medication--onboard aircraft.
33In February 2007, DHS Science and Technology directorate conducted
aircraft vulnerability tests to determine the extent of damage the liquid
explosives that were to be used in the alleged August 2006 London terror
plot would cause to an aircraft. The results of these tests, however, are
sensitive security information and are not included in this report.
34The intelligence information regarding the August 2006 London terror
plot is classified and, therefore, is not included in this report.
TSA officials stated that the enhancements TSA is making to the various
other layers of aviation security will help address the security
vulnerabilities identified above. For example, TSA has increased
explosives detection canine patrols, deployed Federal Air Marshals on
additional international flights, increased random screening of passengers
at boarding gates, and increased random screening of airport and TSA
employees who pass through the checkpoint. TSA also plans to expand
implementation of its Screening Passengers by Observation Technique (SPOT)
to additional airports. SPOT involves specially trained TSOs observing the
behavior of passengers and resolving any suspicious behavior through
casual conversation with passengers and referring suspicious passengers to
selectee screening.35 TSA intends for SPOT to provide a flexible,
adaptable, risk-based layer of security that can be deployed to detect
potentially high-risk passengers based on certain behavioral cues.
TSA's Analysis of the Impact of Certain Proposed Screening Changes on Security
and TSO Resources Could Be Strengthened
While professional judgment regarding risk factors, experience in the
operating environment, and customer feedback have guided many of the
decisions TSA leadership made about which screening procedures to
implement, TSA also sought to use empirical data as a basis for evaluating
the impact some screening changes could have on security and TSO
resources. The TSA Assistant Secretary stated in December 2005 that TSA
sought to make decisions about screening changes based on data and
metrics--a practice he said TSA would continue. The use of data and
metrics to inform TSA's decision making regarding implementing proposed
screening procedures is consistent with our prior work that has shown the
importance of data collection and analyses to support agency decision
making. Between October 2005 and January 2006, TSA's Explosives Detection
Improvement Task Force sought to collect data as part of an effort to test
the impact of seven proposed procedures at selected airports, as noted
earlier.36 These seven proposed procedures were selected because officials
believed they would have a significant impact on how TSOs perform daily
screening functions, TSO training, and customer acceptability. According
to TSA's chief of security operations, the purpose of testing these
procedures in the airport environment was to ensure that TSA was "on the
right path" in implementing them. These particular procedures were
considered by senior TSA officials as especially important for enhancing
the detection of explosives and for deterring terrorists from attempting
to carry out an attack. According to TSA, some of the proposed procedures
could also free up TSOs so that they could spend more time on procedures
for detecting explosives and less time on procedures associated with low
security risks, such as identifying small scissors in carry-on bags. The
seven proposed procedures tested by the task force reflect both new
procedures and modifications to existing procedures, as shown in table 3.
35In the event that TSOs cannot determine the reason for a passenger's
suspicious behavior, the TSO refers the passenger to law enforcement
officials. TSA officials responsible for SPOT told us that in designing
the implementation of SPOT, they worked closely with FBI staff, Secret
Service staff, Israeli security experts, and state police with experience
in recognizing suspicious behaviors.
36Another SOP change was operationally tested and subsequently rejected.
TSA did not provide documentation or other information on the reason it
was rejected.
Table 3: Proposed Procedures Operationally Tested by the Explosives
Detection Improvement Task Force, October 2005-January 2006
New or
Title of proposed revised
procedure procedure Previous procedure Proposed procedure
Screening New N/A Designated TSOs will
Passengers by observe the
Observation behavioral patterns
Techniquea of passengers, and
based on their
observations, TSOs
will conduct casual
conversations, refer
suspicious
passengers to
secondary screening,
and in some cases
refer some
individuals to law
enforcement officers
Unpredictable Revised Selectee, or additional, Random selectee
Screening Process screening of passengers screening is to be
(USP) must be conducted replaced by the USP,
continuously. If the which entails random
number of individuals selection of
that alarm the passengers across
walk-through metal two screening lanes
detector or if the number to be subjected to a
of bags that alarm is not predetermined
enough to ensure element of the
continual additional selectee screening
screening, individuals process. The
and bags must be randomly specific elements
selected to meet this are sensitive
requirement. security information
and are not
discussed in this
report.
Bulk-item Revised The pat-down procedure The pat-down is to
pat-down search included only the torso include not only the
area of the body. torso, but also from
the waistline down.
IED components New N/A TSOs are to
search implement additional
measures if they
find an IED
component, such as a
battery, when
screening.
Selectee Revised There was a rigid set of More flexibility is
screening changes procedures for resolving to be provided for
alarms set off by resolving alarms set
selectees. off by selectees.
Threat area Revised For bags that appear to For bags that appear
search pose a security threat, to pose a security
various searches were threat, the searches
conducted, where some of that are conducted
the searches were not are intended to
directly focused on the focus more on the
reason for suspicion. reason for
suspicion.
Prohibited items Revised Scissors (metal with Allow scissors with
list changes pointed tips, except pointed tips and
ostomy scissors with blades less than 4
pointed tips with an inches and tools
overall length, including less than 7 inches
blades and handle, of 4 in length onto
inches or less, when aircraft.
accompanied by an
ostomate supply kit
containing related
supplies, such as
collection pouches,
wafers, positioning
plates, tubing, or
adhesives) and tools
(including, but not
limited to, wrenches and
pliers) were not
permitted on aircraft.
Source: TSA
Note: N/A stands for "not applicable," meaning that no previous procedure
existed prior to the new procedure.
aImplementation of SPOT did not involve a revision to the passenger
checkpoint screening SOP; rather, TSA developed a separate set of standard
operating procedures for SPOT. However, we included SPOT in our review
because it modifies the way in which TSOs screen passengers and their
carry-on items at the checkpoint.
Our analysis of TSA's data collection and data analysis for the seven
procedures that were operationally tested identified several problems that
affected TSA's ability to determine whether these procedures, as designed
and implemented by TSA, would have the intended effect--to enhance the
detection of explosives during the passenger screening process or to free
up resources so that explosives detection procedures could be implemented.
Although the deterrence of persons intending to do harm is also an
intended effect of some proposed SOP modifications, TSA officials said
that it is difficult to assess the extent to which implementation of
proposed procedures would deter terrorists. The Office of Management and
Budget has also acknowledged the difficulty in measuring deterrence,
particularly for procedures intended to prevent acts of terrorism. While
we agree that measuring deterrence is difficult, opportunities exist for
TSA to strengthen its analyses to help provide information on whether the
proposed procedures would enhance detection or free up TSO resources, when
intended.
Screening Passengers by Observation Technique. TSA officials stated that
SPOT is intended to both deter terrorists and identify suspicious persons
who intend to cause harm while on an aircraft. While we recognize that it
is difficult to assess the extent to which terrorists are deterred by the
presence of designated TSOs conducting behavioral observations at the
checkpoint, we believe that there is an opportunity to assess whether SPOT
contributes to enhancing TSA's ability to detect suspicious persons that
may intend to cause harm on an aircraft. One factor that may serve as an
indicator that a person intends to do harm on an aircraft is whether that
individual is carrying a prohibited item. TSA collected and assessed data
at 14 airports for various time periods on the number of prohibited items
found on passengers who were targeted under SPOT and referred to secondary
screening or law enforcement officials.37 However, these data collection
efforts, alone, did not enable TSA to determine whether the detection of
prohibited items would be enhanced if SPOT were implemented because TSA
had no means of comparing whether persons targeted by SPOT were more
likely to carry prohibited items than persons not targeted by SPOT. To
obtain this information, the task force would have had to collect data on
the number of passengers not targeted by SPOT that had prohibited items on
them. This information could be used to determine whether a greater
percentage of passengers targeted under SPOT are found to have prohibited
items than those passengers who are not targeted by SPOT, which could
serve as one indicator of the extent to which SPOT would contribute to the
detection of passengers intending to cause harm on an aircraft.
37SPOT was operationally tested at 1 airport beginning in December 2003,
at 2 additional airports beginning in October 2004, and at 2 other
airports beginning in October 2005. The remaining 9 airports began
participating in the operational testing of SPOT in December 2005.
Although it has not yet done so, it may be possible for TSA to evaluate
the impact of SPOT on identifying passengers carrying prohibited items.
There is precedent in other federal agencies for evaluating the security
benefits of similar procedures. For instance, U.S. Customs and Border
Protection (CBP) within DHS developed the Compliance Examination (COMPEX)
system to evaluate the effectiveness of its procedures for selecting
international airline passengers for secondary screening. Specifically,
COMPEX compares the percentage of targeted passengers on which prohibited
items are found to the percentage of randomly selected passengers on which
prohibited items are found. The premise is that targeting is considered to
be effective if a greater percentage of targeted passengers are found to
possess prohibited items than the percentage of randomly selected
passengers, and the difference between the two percentages is
statistically significant.38 CBP officials told us in May 2006 that they
continue to use COMPEX to assess the effectiveness of their targeting of
international airline passengers.39 When asked about using a method such
as COMPEX to assess SPOT, TSA officials stated that CBP and TSA are
seeking to identify different types of threats through their targeting
programs. CBP, through its targeting efforts, is attempting to identify
passengers with contraband and unauthorized aliens, whereas TSA, through
SPOT, is attempting to identify potential high-risk passengers.
Additionally, in commenting on a draft of this report, DHS stated that,
according to TSA, the possession of a prohibited item is not a good
measure of SPOT effectiveness because an individual may not intend to use
a prohibited item to cause harm or hijack an aircraft. While it may be
possible for a terrorist to cause harm or hijack an aircraft without using
a prohibited item, as in the case of the September 11 terrorist attacks,40
other terrorist incidents and threat information identify that terrorists
who carried out or planned to carry out an attack on a commercial aircraft
intended to do so by using prohibited items, including explosives and
weapons. Therefore, we continue to believe that comparing the percentage
of individuals targeted and not targeted under SPOT on which prohibited
items are found could be one of several potential indicators of the
effectiveness of SPOT. Such a measure may be most useful with regard to
the prohibited items that could be used to bring down or hijack an
aircraft. TSA officials stated that the agency agrees in principle that
measuring SPOT effectiveness, if possible, may provide valuable insights.
38Statistically significant means that it is highly unlikely to obtain a
difference of a given size or more by chance, assuming that there is
actually no difference in the probability of finding prohibited items
between targeted and randomly selected passengers.
39CBP officials could not comment on whether a similar methodology could
be used by TSA, since they were not familiar with the SPOT procedure.
40Following the September 11 terrorist attacks, the items terrorists
reportedly used to carry out the attacks--box cutters--were subsequently
prohibited onboard aircraft.
Unpredictable Screening Process, Bulk-Item Pat-Down Search, and IED
Component Search. We found that the task force also could have
strengthened its efforts to evaluate the security impact of other proposed
procedures--specifically, USP, the bulk-item pat-down search, and the IED
component search. For all three of these procedures, the task force did
not collect any data during the operational testing that would help
determine whether they would enhance detection capability. TSA officials
told us that they did not collect these data because they had a limited
amount of time to test the procedures because they had to make SOP
modifications quickly as part of the agency's efforts to focus on higher
threats, such as explosives, and the TSA Assistant Secretary's goal of
implementing the SOP modifications before the 2005 Thanksgiving holiday
travel season. Nevertheless, TSA officials acknowledged the importance of
evaluating whether proposed screening procedures, including USP and the
bulk-item pat-down, would enhance detection capability. TSA officials
stated that covert testing has been used to assess TSOs' ability to detect
prohibited items, but covert testing was not implemented during
operational testing of proposed procedures. Office of Inspection officials
questioned whether covert testing could be used to test, exclusively, the
security benefit of proposed procedures, because TSO proficiency and the
capability of screening technology also factor into whether threat objects
are detected during covert tests. Four of the five aviation security
experts we interviewed acknowledged this limitation but stated that covert
testing is the best way to assess the effectiveness of passenger
checkpoint screening.41 In commenting on a draft of this report, DHS
stated that, according to TSA, USP is intended to disrupt terrorists'
planning of an attack by introducing unpredictability into the passenger
checkpoint screening process, and tools such as covert testing could not
be used to measure the effectiveness of USP to this end. While we agree
that covert testing may not be a useful tool to assess the impact USP has
on disrupting terrorists' plans and deterring terrorists from attempting
to carry out an attack, we continue to believe that covert testing could
have been used to assess whether USP would have helped to enhance
detection capability during the passenger screening process, which TSA
officials stated was another intended result of USP.
41The fifth expert we interviewed said that he was uncertain how to assess
the effectiveness of passenger checkpoint screening procedures.
Although TSA did not collect data on the security impact of the USP and
bulk-item pat-down procedures, the task force did collect data on the
impact these procedures had on screening efficiency--the time required to
perform procedures--and on the reaction of TSOs, FSDs, and passengers to
the proposed procedures. These data indicated that the USP procedure took
less time, on average, for TSOs to conduct than the procedure it replaced
(the random continuous selectee screening process); the revised pat-down
procedure took TSOs about 25 seconds to conduct; and that passengers
generally did not complain about the way in which both procedures were
conducted.
With respect to operational testing of the IED component search procedure,
TSA was unable to collect any data during the testing period because no
IEDs were detected by TSOs at the airports where the testing took place.
As with the USP and bulk-item pat-down procedures, TSA could have
conducted covert tests during the operational testing period to gather
simulated data for the IED search procedure, in the absence of actual
data.
Selectee Screening Changes and Threat Area Search. Recognizing that some
of the proposed procedures intended to enhance detection would require
additional TSO resources, TSA implemented several measures aimed
collectively at freeing up TSOs' time so that they could focus on
conducting more procedures associated with higher threats--identifying
explosives and suspicious persons. For example, TSA modified the selectee
screening procedure and the procedure for searching carry-on items--the
threat area search--in order to reduce screening time. During an informal
pilot of these proposed procedures at 3 airports in November 2005, TSA
determined that the proposed selectee screening procedure would reduce
search time of each selectee passenger, on average, by about 1.17 minutes
at these airports. TSA also determined through this study that the
proposed threat area search, on average, took 1.83 minutes to conduct at
the participating airports, as compared to the existing target object
search that took, on average, 1.89 minutes, and the existing whole bag
search that took, on average, 2.37 minutes.
Prohibited Items List Changes. Another measure TSA implemented to free up
TSO resources to focus on higher threats involved changes to the list of
items prohibited onboard aircraft. According to TSA, TSOs were spending a
disproportionate amount of TSA's limited screening resources searching for
small scissors and small tools, even though, based on threat information
and TSA officials' professional judgment, such items no longer posed a
significant security risk given the multiple layers of aviation security.
TSA officials surmised that by not having to spend time and resources
physically searching passengers' bags for low-threat items, such as small
scissors and tools, TSOs could focus their efforts on implementing more
effective and robust screening procedures that can be targeted at
screening for explosives.
To test its assumption that a disproportionate amount of TSO resources was
being spent searching for small scissors and tools, TSA collected
information from several sources. First, TSA reviewed data maintained in
TSA's Performance Management Information System (PMIS),42 which showed
that during the third and fourth quarters of fiscal year 2005 (a 6-month
period), TSOs confiscated a total of about 1.8 million sharp objects other
than knives or box cutters. These sharp objects constituted 19 percent of
all prohibited items confiscated at the checkpoint. Second, based on
information provided by FSDs, TSOs, and other screening experts, TSA
determined that scissors constituted a large majority of the total number
of sharp objects found at passenger screening checkpoints. Third, TSA
headquarters officials searched through confiscated items bins at 4
airports and found that most of the scissors that were confiscated had
blades less than 4 inches in length. Based on these collective efforts,
TSA concluded that a significant number of items found at the checkpoint
were low-threat, easily identified items, such as small scissors and
tools, and that a disproportionate amount of time was spent searching for
these items--time that could have been spent searching for high-threat
items, such as explosives. TSA also concluded that because TSOs can
generally easily identify scissors, if small scissors were no longer on
the prohibited items list, TSOs could avoid conducting time-consuming
physical bag searches to locate and remove these items.
42TSA's Performance Management Information System is designed to collect,
analyze, and report passenger and baggage screening performance data, such
as wait times at selected airports, workload data, and the performance and
utilization of passenger and baggage screening equipment. TSA headquarters
uses PMIS data to support external reporting on performance and internal
decision-making processes.
While we commend TSA's efforts to supplement professional judgment with
data and metrics in its decision to modify passenger checkpoint screening
procedures, TSA did not conduct the necessary analysis of the data
collected to determine the extent to which the removal of small scissors
and tools from the prohibited items list could free up TSO resources.
Specifically, TSA did not analyze the data on sharp objects confiscated at
the checkpoint along with other relevant factors, such as the amount of
time taken to search for scissors and the number of TSOs at the checkpoint
conducting these searches, to determine the extent to which TSO resources
could actually be freed up. Based on our analysis of TSA's data for the
6-month period, where we considered these other relevant factors, we
determined that TSOs spent, on average, less than 1 percent of their
time--about 1 minute per day over the 6-month period--searching for the
approximately 1.8 million sharp objects, other than knives and box
cutters, that were found at passenger screening checkpoints between April
2005 and September 2005.43 If the average amount of time TSOs spent
searching for sharp objects per day over a 6-month period was less than 1
minute per TSO, and sharp objects constituted just 19 percent of all
prohibited items confiscated at checkpoints over this period, then it may
not be accurate to assume that no longer requiring TSOs to search for
small scissors and tools would significantly contribute to TSA's efforts
to free up TSO resources that could be used to implement other security
measures.
43To conduct our analysis we used TSA data that showed (1) it takes, on
average, about 1.89 minutes to conduct a bag search that was initiated
because a TSO identified a prohibited item (such as a pair of scissors) in
the X-ray image of a carry-on bag--this average search time was derived
from an informal TSA property search time study conducted at 9
airports--and (2) there were 28,785 actual full-time-equivalent (FTE)
passenger screening TSOs during fiscal year 2005. One FTE is equal to 1
work year or 2,080 nonovertime hours. To determine the number of minutes
per day, on average, each TSO spent searching for sharp objects found
during the 6-month period, we took the following steps. First, we
calculated the total amount of time (in minutes) taken to conduct the
searches by multiplying the number of sharp objects found (1,762,571) by
the average time to conduct targeted searches (1.89 minutes), assuming
that one item was found per search. This totaled 55,521 hours. Next, we
calculated the amount of time, on average, each TSO spent searching for
the sharp objects found by dividing 55,521 hours by 28,785 TSO FTEs. The
result was 1.93 hours per TSO. Finally, we converted average hours per TSO
to minutes and divided by 130 days--the number of days worked by a TSO for
26 weeks over a 6-month period (assuming 5 work days per week at 8 hours
per day). The result was an average of 0.89 minutes per day per TSO over
the 6-month period.
To further support its assertion that significant TSO resources would be
freed up as a result of removing small scissors and tools from the list of
prohibited items, TSA officials cited the results of an informal study
conducted in October 2005--which was intended to provide a general idea of
the types of prohibited items TSOs were finding as a result of their
searches and how long various types of searches were taking TSOs to
conduct. Specifically, according to the study conducted at 9 airports over
a 14-day period, TSA determined that 24 percent of items found during
carry-on bag searches were scissors. However, based on data regarding the
number of bags searched, removing scissors may not significantly
contribute to TSA's efforts to free up TSO resources.44
TSA conducted additional informal studies 30, 60, and 90 days after the
prohibited items list change went into effect to determine whether the
change had resulted in reductions in the percentage of carry-on bags that
were searched and overall screening time. However, we identified
limitations in TSA's methodology for conducting these studies.45 In
February 2007, a TSA official stated that some FSDs interviewed several
TSOs after the prohibited items list change went into effect, and these
TSOs reported that the change did save screening time. However, TSA could
not identify how many TSOs were interviewed, at which airports the TSOs
were located, and how the TSOs were selected for the interview; nor did
TSA document the results of these interviews. TSA also did not use random
selection or representative sampling when determining which TSOs should be
interviewed. Therefore, the interview results cannot be generalized.
TSA officials acknowledged that they could have made some improvements in
the various analyses they conducted on the prohibited items list change.
However, they stated that they had to make SOP modifications quickly as
part of the agency's efforts to focus on higher threats, such as
explosives, and the TSA Assistant Secretary's goal of implementing the SOP
modifications before the 2005 Thanksgiving holiday travel season.
Additionally, officials stated that they continue to view their decision
to remove small scissors and tools from the prohibited items list as
sound, particularly because they believe small scissors and tools do not
pose a significant threat to aviation security. TSA officials also stated
that they believe the prohibited items list change would free up resources
based on various sources of information, including the professional
judgment of TSA airport staff, and their analysis of PMIS data on
prohibited items confiscated at checkpoints. The TSA Assistant Secretary
told us that even if TSA determined that the proposed SOP modifications
would not free up existing TSO resources to conduct explosives detection
procedures, he would have implemented the modifications anyway considering
the added security benefit of the explosives detection procedures.
Additionally, a TSA headquarters official responsible for airport security
operations stated that to help strengthen the agency's analysis of future
proposed SOP changes, the agency plans to provide the Explosives Detection
Improvement Task Force with the necessary resources to help improve its
data collection and analysis.
44The number of bags searched is sensitive security information.
45The results of the informal follow-on studies, which were conducted at 6
to 9 airports, show that the percentage of carry-on bags searched
increased slightly at the time of the 30-day study, then decreased
slightly at the time of the 60-day and 90-day studies, respectively.
However, the results of these informal studies may not be reliable due to
the limitations in the methodology TSA used to conduct the studies.
Specifically, TSA did not use a methodology that would control for factors
other than the prohibited items list change that may influence the
percentage of carry-on bags searched by TSOs. To do this, TSA would have
had to develop a formal, systematic methodology for randomly selecting
various times of day, location of checkpoints, number of checkpoints, and
so on for data collection. By not controlling for such factors, TSA may
not know the extent to which a reduction in the percentage of carry-on
bags searched is due to the prohibited items list changes.
An additional measure intended to free up TSO resources46 involved changes
to CAPPS rules.47 TSA's assumption is that these changes could allow TSOs
who were normally assigned to selectee screening duties to be reassigned
to new procedures, such as USP, which may require new screening positions.
(Both USP and SPOT require TSO positions: USP requires one screening
position for every two screening lanes, while SPOT typically uses more
than one screening position per ticket checker at the checkpoint.48)
46TSA officials told us that TSA's Office of Intelligence assessed the
potential impact each of these CAPPS changes would have on security and,
based in its analysis, determined that none of the CAPPS changes would
compromise security.
47Passengers can be selected for secondary screening through CAPPS or
other TSA-approved processes, such as the Selectee List. CAPPS rules are
sensitive security information and, therefore, are not discussed in this
report.
According to FSDs we interviewed, the changes made to the prohibited items
list and the CAPPS rules had not freed up existing TSO resources, as
intended. Specifically, as of August 2006, 13 of 19 FSDs we interviewed at
airports that tested USP or SPOT said that TSO resources were not freed up
as a result of these changes. In addition, 9 of the 19 FSDs said that in
order to operationally test USP or SPOT, TSOs had to work overtime, switch
from other functions (such as checked baggage screening), or a screening
lane had to be closed. TSA's Explosives Detection Improvement Task Force
reported that nearly all of the FSDs at airports participating in
operational testing of USP believed that the procedure had security
value,49 though the task force also reported that 1 FSD dropped out of the
operational testing program for USP due to insufficient staffing resources
and another could only implement the procedure during off-peak travel
periods. Additionally, most of the FSDs we interviewed stated that the
changes to the prohibited items list and CAPPS rules did not free up TSOs,
as intended, to better enable TSOs to take required explosives detection
training. Specifically, as of August 2006, of the 19 FSDs we interviewed
at airports that implemented USP and SPOT, 13 said that they did not
experience more time to conduct explosives training as a result of changes
to the prohibited items list and CAPPS rules.50 Three of the 13 FSDs said
that they used overtime to enable TSOs to take the explosives training. As
previously stated, the TSA Assistant Secretary stated that even if
existing TSO resources are not freed up to conduct explosives detection
procedures, longer lines and wait times at airport checkpoints are an
acceptable consequence, considering their added security benefit. With
regard to explosives training, he stated that it is acceptable for FSDs to
use overtime or other methods to ensure that all TSOs participated in the
required explosives detection training. He further noted that even if one
screening change does not free up TSO resources, all of the changes
intended to accomplish this--when taken together--should ultimately help
to redirect TSO resources to where they are most needed.
48Ticket checkers are aircraft operator or TSA employees who are
positioned before the screening checkpoint to perform identification check
and sterile area access responsibilities as required by TSA. For
passengers, ticket checkers verify travel documents and make sure the
identifying information on the travel document is consistent with the
information on the individual's personal identification documents (e.g.,
licenses, passport, etc.). Ticket checkers are also responsible for
directing passengers designated as selectees to the appropriate screening
lane. For nonpassengers, ticket checkers verify required identification
before allowing access to the sterile area.
49The task force reported that 1 FSD was unsure of the security benefits
provided by USP, though this FSD did support the concept of introducing
unpredictability into the screening process.
50Of the remaining 6 FSDs, 5 said that TSO resources were freed up as a
result of the prohibited items list and CAPPS rules changes, and 1 was
uncertain whether TSO resources were actually freed up.
TSA's efforts to add data and metrics to its tool kit for evaluating the
impact of screener changes are a good way to supplement the use of
professional judgment and input from other experts and sources in making
decisions about modifying screening procedures. However, TSA's methods for
data collection and analysis could be improved. We recognize the
challenges TSA faces in evaluating the effectiveness of proposed
procedures, particularly when faced with time pressures to implement
procedures. However, by attempting to evaluate the potential impact of
screening changes on security and resource availability, TSA could help
support its decision making on how best to allocate limited TSO resources
and ensure that the ability to detect explosives and other high-threat
objects during the passenger screening process is enhanced.
Documentation of the Reasoning behind Proposed SOP Modifications Was Incomplete
While we were able to assess TSA's reasoning behind certain proposed SOP
modifications considered during our review period, our analysis was
limited because TSA did not maintain complete documentation of proposed
SOP modifications. Documentation of the reasoning behind decisions to
implement or reject proposed modifications was maintained in various
formats, including spreadsheets developed by TSA officials, internal
electronic mail discussions among TSA officials, internal memorandums,
briefing slides, and reports generated based on the results of operational
testing. TSA did improve its documentation of the proposed SOP
modifications that were considered during the latter part of our 9-month
review period. Specifically, the documentation for the SOP modifications
proposed under the Explosives Detection Improvement Task Force provided
more details regarding the basis of the proposed modifications and the
reasoning behind decisions to implement or reject the proposed
modifications.
Of the 92 proposed SOP modifications considered during our 9-month review
period that TSA documented, TSA provided the basis for 72. More
specifically, TSA documented the basis--that is, the information,
experience, or event that encouraged TSA officials to propose an SOP
modification--for 35 of the 48 that were implemented and for 37 of the 44
that were not implemented. However, TSA only documented the reasoning
behind TSA senior leadership's decisions to implement or not implement
proposed SOP modifications for 43 of 92 proposed modifications. According
to TSA officials, documentation that explains the basis for recommending
proposed modifications can also be used to explain TSA's reasoning behind
its decisions to implement proposed modifications. However, the basis on
which an SOP modification was proposed cannot always be used to explain
TSA senior leadership's decisions not to implement a proposed
modification. In these cases, additional documentation would be needed to
understand TSA's decision making. However, TSA only documented the
reasoning behind its decisions for about half (26 of 44) of the proposed
modifications that were not implemented. TSA officials told us that they
did not intend to document all SOP modifications that were proposed during
our review period. Officials stated that, in some cases, the reasoning
behind TSA's decision to implement or not implement a proposed SOP
modification is obvious and documentation is not needed. TSA officials
acknowledged that it is beneficial to maintain documentation on the
reasoning behind decisions to implement or reject proposed SOP
modifications deemed significant, particularly given the organizational
restructuring and staff turnover within TSA.51 However, TSA officials
could not identify which of the 92 proposed SOP modifications they
consider to be significant because they do not categorize proposed
modifications in this way.
Our standards for governmental internal controls and associated guidance
suggest that agencies should document key decisions in a way that is
complete and accurate, and that allows decisions to be traced from
initiation, through processing, to after completion.52 These standards
further state that documentation of key decisions should be readily
available for review. Without documenting this type of information, TSA
cannot always justify significant modifications to passenger checkpoint
screening procedures to internal or external stakeholders, including
Congress and the traveling public. In addition, considering the ongoing
personnel changes, without sufficient documentation, future decision
makers in TSA may not know on what basis the agency historically made
decisions to develop new or revise existing screening procedures.
51Since its inception in November 2001, TSA has had multiple Assistant
Secretaries (originally titled Under Secretaries of Transportation for
Security). In addition, between January 2005 and August 2006, TSA issued
seven press releases regarding senior-level personnel changes within the
agency.
52GAO, Internal Control: Internal Control Management and Evaluation Tool,
[30]GAO-01-1008G (Washington, D.C.: August 2001).
Following our 9-month review period, TSA continued to make efforts to
improve documentation of agency decision making, as evidenced by decisions
regarding the August 2006 and September 2006 SOP modifications related to
the screening of liquids and gels. For example, TSA senior leadership
evaluated the actions taken by the agency between August 7 and August 13,
2006, in response to the alleged liquid explosives terrorist plot, in
order to identify lessons learned and improve the agency's reaction to
future security incidents. As a result of this evaluation, as shown in
table 4, TSA made several observations and recommendations for improving
documentation of agency decision making when considering modifications to
screening procedures.
Table 4: TSA Evaluation of Documentation of Agency Decisions Made between
August 7 and August 13, 2006, Regarding the Alleged Liquid Explosives
Terrorist Plot
Observations Recommendations for improvement
There was no tracking of the overall Track and record key issues
timing and progress of deliberations of raised and the timing of
the various decision options. deliberations.
There was no formal tracking of the Formally document options
decision options that were discussed or discussed, decisions made, and
the rationale that was used when the rationale behind the
selecting among the various decision decisions.
options.
There were no formal requirements for For each decision that is made,
the type of information that needed to standardize the type of
be documented or the format used to information that should be
document the information on agency documented and develop an
decisions. appropriate mechanism to store
the information.
The documentation that was used to For each document used to support
support agency decisions did not contain agency decisions, identify the
basic audit trail information, such as origin of the document and how
the origin of the document and how the the document was used by decision
document was used. This may prevent makers.
decision makers from understanding the
relevancy of the documentation to agency
decisions.
Source: TSA.
Documentation of TSA's decisions regarding the September 26, 2006,
modifications to the liquid screening procedures showed that TSA had begun
implementing the recommendations in table 4. TSA's documentation
identified the various proposed liquid screening procedures that were
considered by TSA, the benefits and drawbacks of each proposal, and the
rationale behind TSA's final decision regarding which proposal to
implement. The documentation also tracked the timing of TSA's
deliberations of each of the proposed liquid screening procedures.
However, the documentation of TSA's decisions was not always presented in
a standard format, nor was the origin and use of supporting documentation
always identified. TSA officials acknowledged that documentation of the
September 2006 SOP modifications could have been improved and stated that
efforts to improve documentation, through implementation of the
recommendations in table 4, will continue to be a high priority.
TSA Has Several Methods in Place to Monitor TSO Compliance with Passenger
Checkpoint Screening SOPs
A New Performance Accountability System Helps TSA Monitor TSO Compliance with
SOPs
TSA implemented a performance accountability system in part to strengthen
its monitoring of TSO compliance with passenger checkpoint screening SOPs.
Specifically, in April 2006, TSA implemented the Performance
Accountability and Standards System (PASS) to assess the performance of
all TSA employees, including TSOs.53 According to TSA officials, PASS was
developed in response to our 2003 report that recommended that TSA
establish a performance management system that makes meaningful
distinctions in employee performance,54 and in response to input from TSA
airport staff on how to improve passenger and checked baggage screening
measures. With regard to TSOs, PASS is not intended solely to measure TSO
compliance with SOPs. Rather, PASS will be used by TSA to assess agency
personnel at all levels on various competencies, including training and
development, readiness for duty, management skills, and technical
proficiency.
53In July 2005, prior to the implementation of PASS, TSA required all FSDs
to implement an audit program of screening checkpoint operations,
primarily focused on assessing TSO compliance with checkpoint screening
SOPs. Specifically, each airport is to have an audit program that
evaluates TSOs' ability to detect threat objects taken through the
checkpoint, as well as TSOs' compliance with SOPs for screening passengers
and their accessible property. The audit program is also intended to
evaluate screening supervisors' and lead TSOs' compliance with the SOP.
54 [31]GAO-03-190.
There are three elements of the TSO technical proficiency component of
PASS that are intended to measure TSO compliance with passenger checkpoint
screening procedures: (1) quarterly observations conducted by FSD
management staff of TSOs' ability to perform particular screening
functions in the operational environment, such as pat-down searches and
use of the hand-held metal detector, to ensure they are complying with
checkpoint screening SOPs; (2) quarterly quizzes given to TSOs to assess
their knowledge of the SOPs; and (3) an annual, multipart knowledge and
skills assessment. While the first two elements are newly developed, the
third element--the knowledge and skills assessment--is part of the annual
TSO recertification program that is required by the Aviation and
Transportation Security Act (ATSA) and has been in place since October
2003.55 Collectively, these three elements of PASS are intended to provide
a systematic method for monitoring whether TSOs are screening passengers
and their carry-on items according to SOPs. TSA's implementation of PASS
is consistent with our internal control standards, which state that
agencies should ensure that policies and procedures are applied
properly.56
The first component of PASS (quarterly observations) is conducted by
screening supervisors or screening managers, using a standard checklist
developed by TSA headquarters, with input from TSA airport staff. There is
one checklist used for each screening function, and TSOs are evaluated on
one screening function per quarter. For example, the hand-held metal
detector skills observation checklist includes 37 tasks to be observed,
such as whether the TSO conducted a pat-down search to resolve any suspect
areas. The second component of PASS (quarterly quizzes) consists of
multiple-choice questions on the standard operating procedures. For
example, one of the questions on the PASS quiz is "What is the correct
place to start an HHMD outline [a hand-held metal detector search] on an
individual: (a) top of the head, (b) top of the feet, or (c) top of the
shoulder?"
55ATSA requires that each TSO receive an annual proficiency review to
ensure he or she continues to meet all qualifications and standards
required to perform screening functions. See 49 U.S.C. S 44935(f)(5).
56 [32]GAO-01-1008G.
The third component of PASS is the annual knowledge and skills assessment,
a component of the annual recertification program that evaluates the
technical proficiency of TSOs. This assessment is composed of three
modules: (1) knowledge of standard operating procedures, (2) recognition
of threat objects on an X-ray image, and (3) demonstration of screening
functions. According to TSA officials, while recertification testing is
not a direct measure of operational compliance with passenger checkpoint
screening SOPs, recertification testing, particularly module 1 and module
3, is an indicator of whether TSOs are capable of complying with SOPs. TSA
officials stated that if a TSO does not have knowledge of SOPs and if the
TSO cannot demonstrate basic screening functions as outlined in the SOPs,
then the TSO will likely not be able to comply with SOPs when performing
in the operating environment. Table 5 provides a summary of each of these
modules.
Table 5: Modules Included in Recertification Knowledge and Skills
Assessment
Testing module Description
Knowledge of standard Computerized 50-question multiple-choice test.
operating procedures It is either passenger- or baggage-specific.
Image recognition Computerized test that consists of 100 images
and is used to evaluate a TSO's skill and
ability in detecting threat or prohibited
objects within X-ray images.
Practical demonstration of Hands-on simulated work sample to evaluate a
skills TSO's knowledge, skills, and ability when
performing specific screening tasks along with
ability to provide customer service.
Source: TSA.
FSDs we interviewed reported that they have faced resource challenges in
implementing PASS. Specifically, as of July 2006, 9 of 24 FSDs we
interviewed said they experienced difficulties in implementing PASS due to
lack of available staff to conduct the compliance-related evaluations. TSA
officials stated that they have automated many of the data-entry functions
of PASS to relieve the field of the burden of manually entering this
information into the PASS online system. For example, all scores related
to the quarterly quiz and skill observation components are automatically
uploaded, and PASS is linked to TSA's online learning center database to
eliminate the need to manually enter TSOs' learning history. In addition,
the TSA Assistant Secretary said that FSDs were given the option of
delaying implementation of PASS if they were experiencing resource
challenges.
TSA Uses Local and National Covert Testing, in Part, to Assess TSO Compliance
with SOPs
TSA also conducts local and national covert tests, which are used to
evaluate, in part, the extent to which noncompliance with the SOPs affects
TSOs' ability to detect simulated threat items hidden in accessible
property or concealed on a person. TSA first issued guidance on its local
covert testing program--known as Screener Training Exercises and
Assessments (STEA)--in February 2004. STEA testing is conducted by FSD
staff at airports, who determine the frequency at which STEA tests are
conducted as well as which type of STEA tests are conducted. According to
the STEA results reported by TSA between March 2004 and February 2006,57
TSOs' noncompliance with the SOP accounted for some of the STEA test
failures.58 TSOs' lack of proficiency in skills or procedures, which may
affect TSOs' ability to comply with procedures, was also cited as the
reason for some of the STEA test failures. TSOs who fail STEA tests are
required to take remedial training to help them address the reasons for
their failure.
FSDs we interviewed reported that they have faced resource challenges in
conducting STEA tests. Specifically, even though all 24 FSDs we
interviewed as of July 2006 said that they have conducted STEA tests, 10
of these FSDs said that the lack of available staff made it difficult to
conduct these tests. When asked how they planned to address FSDs' concerns
regarding a lack of available staff to complete STEA tests, TSA
headquarters officials told us that they are considering resource
alternatives for implementing the STEA program, but could not provide us
with the specific details of these plans.59 Until the resource limitations
that have restricted TSA's use of its compliance monitoring tools have
been fully addressed, TSA may not have assurance that TSOs are screening
passengers according to the SOP.
57As of February 2006, STEA test results had been recorded for a total of
417 airports.
58The results of STEA testing are sensitive security information and,
therefore, are not included in this report.
59As of December 2006, TSA was in the process of modifying STEA into a
performance measurement program. TSA plans to implement the new STEA
program during the second quarter of fiscal year 2007.
As previously discussed, TSA's Office of Inspection initiated its national
covert testing program in September 2002. National covert tests are
conducted by TSA headquarters-based inspectors who carry simulated threat
objects hidden in accessible property or concealed on their person through
airport checkpoints, and in cases where TSOs fail to detect threat
objects, the inspectors identify the reasons for failure. During September
2005, TSA implemented a revised covert testing program to focus more on
catastrophic threats--threats that can bring down or destroy an aircraft.
According to Office of Inspection officials, TSOs may fail to detect
threat objects during covert testing for various reasons, including
limitations in screening technology, lack of training, limitations in the
procedures TSOs must follow to conduct passenger and bag searches, and
TSOs' noncompliance with screening checkpoint SOPs. Office of Inspection
officials also said that one test could be failed due to multiple factors,
and that it is difficult to determine the extent to which any one factor
contributed to the failure. TSOs who fail national covert tests, like
those who fail STEA tests, are also required to take remedial training to
help them address the reasons for failure.60
Conclusions
The alleged August 2006 terrorist plot to detonate liquid explosives
onboard multiple U.S.-bound aircraft highlighted the need for TSA to
continuously reassess and revise, when deemed appropriate, existing
passenger checkpoint screening procedures to address threats against the
commercial aviation system. In doing so, TSA faces the challenge of
securing the aviation system while facilitating the free movement of
people. Passenger screening procedures are only one element that affects
the effectiveness and efficiency of the passenger checkpoint screening
system. Securing the passenger checkpoint screening system also involves
the TSOs who are responsible for conducting the screening of airline
passengers and their carry-on items, and the technology used to screen
passengers and their carry-on items.
We believe that TSA has implemented a reasonable approach to modifying
passenger checkpoint screening procedures through its consideration of
risk factors (threat and vulnerability information), day-to-day experience
of TSA airport staff, and complaints and concerns raised by passengers and
by making efforts to balance security, efficiency, and customer service.
We are also encouraged by TSA's efforts to conduct operational testing and
use data and metrics to support its decisions to modify screening
procedures. We acknowledge the difficulties in assessing the impact of
proposed screening procedures, particularly with regard to the extent to
which proposed procedures would deter terrorists from attempting to carry
out an attack onboard a commercial aircraft. However, there are existing
methods, such as covert testing and CBP's COMPEX--a method that evaluates
the effectiveness of CBP's procedures for selecting international airline
passengers for secondary screening--that could be used by TSA to assess
whether proposed screening procedures enhance detection capability. It is
also important for TSA to fully assess available data to determine the
extent to which TSO resources would be freed up to perform higher-priority
procedures, when this is the intended effect. Without collecting the
necessary data or conducting the necessary analysis that would enable the
agency to assess whether proposed SOP modifications would have the
intended effect, it may be difficult for TSA to determine how best to
improve TSOs' ability to detect explosives and other high-threat items and
to allocate limited TSO resources. With such data and analysis, TSA would
be in a better position to justify its SOP modifications and to have a
better understanding of how the changes affect TSO resources.
Additionally, because TSA did not always document the basis on which SOP
modifications were proposed or the reasoning behind decisions to implement
or not implement proposed modifications, TSA may not be able to justify
SOP modifications to Congress and the traveling public. While we are
encouraged that TSA's documentation of its decisions regarding the SOP
modifications made in response to the alleged August 2006 liquid
explosives terrorist plot was improved compared to earlier documentation,
it is important for TSA to continue to work to strengthen its
documentation efforts. Such improvements would enable TSA officials
responsible for making SOP decisions in the future to understand how
significant SOP decisions were made historically-- a particular concern
considering the restructuring and staff turnover experienced by TSA.
60The covert testing results, including the reasons for failure, and the
recommendations made by the Office of Inspection are classified and cannot
be discussed in this report.
As shown by TSA's covert testing results, the effectiveness of passenger
checkpoint screening relies, in part, on TSOs' compliance with screening
procedures. We are, therefore, encouraged by TSA's efforts to strengthen
its monitoring of TSO compliance with passenger screening procedures. We
believe that TSA has implemented a reasonable process for monitoring TSO
compliance and that this effort should assist TSA in providing reasonable
assurance that TSOs are screening passengers and their carry-on items
according to screening procedures. Given the resource challenges FSDs
identified in implementing the various methods for monitoring TSO
compliance, it will be important for TSA to take steps, such as automating
PASS data entry functions, to address such challenges.
Recommendations for Executive Action
To help strengthen TSA's evaluation of proposed modifications to passenger
checkpoint screening SOPs and TSA's ability to justify its decisions to
implement or not implement proposed SOP modifications, in the March 2007
report that contained sensitive security information, we recommended that
the Secretary of Homeland Security direct the Assistant Secretary of
Homeland Security for TSA to take the following two actions:
o when operationally testing proposed SOP modifications, develop
sound evaluation methods, when possible, that can be used to
assist TSA in determining whether proposed procedures would
achieve their intended result, such as enhancing TSA's ability to
detect prohibited items and suspicious persons and freeing up
existing TSO resources that could be used to implement proposed
procedures, and
o for future proposed SOP modifications that TSA senior leadership
determines are significant, generate and maintain documentation to
include, at minimum, the source, intended purpose, and reasoning
behind decisions to implement or not implement proposed
modifications.
Agency Comments and Our Evaluation
On March 6, 2007, we received written comments on the draft report, which
are reproduced in full in appendix III. DHS generally concurred with our
recommendations and outlined actions TSA plans to take to implement the
recommendations.
DHS stated that it appreciates GAO's conclusion that TSA has implemented a
reasonable approach to modifying passenger checkpoint screening procedures
through its assessment of risk factors, the expertise of TSA employees,
and input from the traveling public and other stakeholders, as well as
TSA's efforts to balance security, operational efficiency, and customer
service while evaluating proposed changes.
With regard to our recommendation to develop sound evaluation methods,
when possible, to help determine whether proposed SOP modifications would
achieve their intended result, DHS stated that TSA plans to make better
use of generally accepted research design principles and techniques when
operationally testing proposed SOP modifications. For example, TSA will
consider using random selection, representative sampling, and control
groups in order to isolate the impact of proposed SOP modifications from
the impact of other variables. DHS also stated that TSA's Office of
Security Operations is working with subject matter experts to ensure that
operational tests are well designed and executed, and produce results that
are scientifically valid and reliable. As discussed in this report,
employing sound evaluation methods for operationally testing proposed SOP
modifications will enable TSA to have better assurance that new passenger
checkpoint screening procedures will achieve their intended purpose, which
may include improved allocation of limited TSO resources and enhancing
detection of explosives and other high-threat objects during the passenger
screening process. However, DHS stated, and we agree, that the need to
make immediate SOP modifications in response to imminent terrorist threats
may preclude operational testing of some proposed modifications.
Concerning our recommendation regarding improved documentation of proposed
SOP modifications, DHS stated that TSA intends to document the source,
intent, and reasoning behind decisions to implement or reject proposed SOP
modifications that TSA senior leadership deems significant. Documenting
this type of information will enable TSA to justify significant
modifications to passenger checkpoint screening procedures to internal and
external stakeholders, including Congress and the traveling public. In
addition, considering the ongoing personnel changes TSA has experienced,
such documentation should enable future decision makers in TSA to
understand on what basis the agency historically made decisions to develop
new or revise existing screening procedures.
In addition to commenting on our recommendations, DHS provided comments on
some of our findings, which we considered and incorporated in the report
where appropriate. One of DHS's comments pertained to TSA's evaluation of
the prohibited items list change. Specifically, while TSA agrees that the
agency could have conducted a more methodologically sound evaluation of
the impact of the prohibited items list change, TSA disagrees with our
assessment that the prohibited items list change may not have
significantly contributed to TSA's efforts to free up TSO resources to
focus on detection of high-threat items, such as explosives. As we
identified in this report, based on interviews with FSDs, airport visits
to determine the types of items confiscated at checkpoints, and a study to
determine the amount of time taken to conduct bag searches and the number
of sharp objects collected as a result of these searches, TSA concluded
that the prohibited items list change would free up TSO resources. DHS
also stated that interviews with TSOs following the prohibited items list
change confirmed that the change had freed up TSO resources. However,
based on our analysis of the data TSA collected both prior to and
following the prohibited items list change, we continue to believe that
TSA did not conduct the necessary analysis to determine the extent to
which the removal of small scissors and tools from the prohibited items
list would free up TSA resources.
As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 21 days from
the date of this report. At that time, we will send copies of the report
to the Secretary of the Department of Homeland Security, the TSA Assistant
Secretary, and interested congressional committees as appropriate. We will
also make copies available to others on request.
If you or your staff have any questions about this report, please contact
me at (202) 512-3404 or [email protected] . Contact points for our
Offices of Congressional Relations and Public Affairs may be found on the
last page of this report. GAO staff that made major contributions to this
report are listed in appendix IV.
Cathleen A. Berrick
Director, Homeland Security and Justice Issues
Appendix I: Objectives, Scope and Methodology
To assess the Transportation Security Administration's (TSA) process for
modifying passenger checkpoint screening procedures and how TSA monitors
compliance with these procedures, we addressed the following questions:
(1) How and on what basis did TSA modify passenger screening procedures
and what factors guided the decisions to do so? (2) How does TSA determine
whether TSOs are complying with the standard procedures for screening
passengers and their carry-on items?
To address how TSA modified passenger screening procedures and what
factors guided the decisions to do so, we obtained and analyzed
documentation of proposed standard operating procedures (SOP) changes
considered between April 2005 and September 2005, as well as threat
assessments and operational studies that supported SOP modifications.1 The
documentation included a list of proposed changes considered, as well as
the source, the intended purpose, and in some cases the basis for
recommending the SOP modification--that is, the information, experience,
or event that encouraged TSA officials to propose the modifications--and
the reasoning behind decisions to implement or reject proposed SOP
modifications. We also obtained documentation of the proposed SOP changes
considered by TSA's Explosives Detection Improvement Task Force, which was
the deliberating body for proposed changes that were considered between
October 2005 and December 2005. We also reviewed and analyzed similar
documentation for proposed SOP modifications considered between August
2006 and November 2006 in response to the alleged terrorist plot to
detonate liquid explosives onboard multiple aircraft en route from the
United Kingdom to the United States. We included modifications to
passenger checkpoint screening procedures related to this particular event
because they provided the most recent information available of TSA's
approach to modifying screening procedures in response to an immediate
perceived threat to civil aviation. The documentation included notes from
internal meetings, slides for internal and external briefings on proposed
SOP modifications, data on customer complaints and screening efficiency,
and the results of liquid explosives testing conducted by the Department
of Homeland Security (DHS) Science and Technology Directorate and the
Federal Bureau of Investigation (FBI). We also obtained each revision of
the passenger checkpoint screening SOP that was generated between April
2005 and December 2005 and August 2006 and November 2006,2 as well as
accompanying documentation that highlighted all of the changes made in
each revision. In addition, we met with TSA headquarters officials who
were involved in the process for determining whether proposed passenger
checkpoint screening procedures should be implemented. We also met with
officials in the DHS Science and Technology Directorate as well as the FBI
to discuss the methodology and results of their liquid explosives tests,
which were used to support TSA's decisions to modify the SOP in September
2006. We also met with TSA Office of Inspection and DHS Office of
Inspector General staff to discuss their covert testing at passenger
checkpoints and the recommended changes to the passenger checkpoint
screening SOP that were generated based on testing results. We also
obtained and analyzed data and information collected by TSA on the
proposed procedures that were evaluated in the operational environment. In
addition, we met or conducted phone interviews with Federal Security
Directors (FSD) and their management staff, including Assistant FSDs and
Screening Managers, and Transportation Security Officers (TSO) with
passenger screening responsibilities, at 25 commercial airports to gain
their perspectives on TSA's approach to revising the passenger checkpoint
screening SOP. We also met with officials from four aviation
associations--the American Association of Airport Executives, Airports
Council International, the Air Transport Association, and the Regional
Airline Association--to gain their perspectives on this objective.
Finally, we met with five aviation security experts to obtain their views
on methods for assessing the impact of proposed passenger checkpoint
screening procedures. We selected these experts based on their depth of
experience in the field of aviation security, employment history, and
their recognition in the aviation security community. However, the views
of these experts may not necessarily represent the general view of other
experts in the field of aviation security. We compared TSA's approach to
revising its passenger checkpoint screening SOP with the Comptroller
General's standards for internal control in the federal government3 and
risk management guidance.
1We began our review period in April 2005 to coincide with TSA's
consideration of proposed SOP modifications related to the second major
revision of the passenger checkpoint screening SOP since TSA's inception.
To address how TSA determines whether TSOs are complying with the standard
procedures for screening passengers and their carry-on items, we obtained
documentation of compliance-related initiatives, including guidance,
checklists, and SOP quizzes used to assess TSO compliance under the
Performance Accountability and Standards System (PASS), and guidance
provided to FSDs for developing local compliance audit programs. We also
obtained the fiscal year 2005 recertification and Screener Training
Exercises and Assessments (STEA) test results, which were used, in part,
to assess TSO compliance with and knowledge of the passenger checkpoint
screening SOP. In addition, we reviewed the results of covert testing
conducted by TSA's Office of Inspection, which were also used, in part, to
assess TSO compliance with the passenger checkpoint screening SOP. We
assessed the reliability of the compliance-related data we received from
TSA, and found the data to be sufficiently reliable for our purposes. In
addition, we interviewed TSA headquarters officials who were responsible
for overseeing efforts to monitor TSO compliance with standard operating
procedures. This included officials in the Office of Security Operations,
Office of Human Capital, and the Office of Operational Process and
Technology. Our audit work also included visits to or phone conferences
with 25 airports, where we interviewed FSDs, members of their management
teams, and Transportation Security Officers with passenger screening
responsibilities.4 However, the perspectives of these FSDs and their staff
cannot be generalized across all airports. In July 2006, we submitted two
sets of follow-up questions to FSD staff, related to their experiences
with implementing PASS and STEA tests. We also obtained documentation of
local compliance audit programs from the FSD staff at several of these
airports. We compared TSA's approach for monitoring TSO compliance with
the Comptroller General's standards for internal control in the federal
government.5
2We did not assess all of the proposed SOP modifications associated with
the SOP revisions issued between August 2006 and November 2006; rather, we
only reviewed the proposed modifications associated with screening for
liquids, gels, and aerosols.
3 [34]GAO/AIMD-00-21 .3.1.
As previously mentioned, we conducted site visits and/or phone interviews
at 25 airports6 (8 category X airports, 7 category I airports, 4 category
II airports, 4 category III airports, and 2 category IV airports) to
discuss issues related to TSA's approach to revising the passenger
checkpoint screening SOP, and the agency's approach to monitoring TSO
compliance with the SOP.7 We visited 7 of these airports during the design
phase of our study. These airports were selected based on variations in
size and geographic location, and whether they were operationally testing
any proposed passenger checkpoint screening procedures or passenger
screening technology. We also selected 2 airports that participated in the
Screening Partnership Program.8
4We visited 25 airports. However, we met with only 24 FSDs, as 1 FSD was
responsible for 2 of the airports we visited.
5 [35]GAO/AIMD-00-21 .3.1.
6The list of airports we visited is sensitive security information.
Therefore, we do not identify those airports in this report.
After visiting the 7 airports during the design phase of our review, we
selected an additional 15 airports to visit based on variations in size,
geographic distribution, and performance on compliance-related
assessments. Specifically, we obtained and analyzed fiscal year 2005
Screener Training Exercise and Assessments results and fiscal year 2005
recertification testing results to identify airports across a range of
STEA and recertification scores. Additionally, we visited 3 additional
airports that operationally tested the proposed Unpredictable Screening
Process (USP) and the Screening Passengers by Observation Technique (SPOT)
procedure.
7TSA classifies the more than 400 commercial airports in the United States
into one of five categories--X, I, II, III, and IV. Generally, category X
airports have the largest number of passenger boardings and category IV
airports have the smallest number.
8The Aviation and Transportation Security Act (ATSA) required that TSA
begin allowing commercial airports to apply to TSA to transition from a
federal to a private screener workforce. See 49 U.S.C. S 44920. To support
this effort, TSA created the Screening Partnership Program to allow all
commercial airports an opportunity to apply to TSA for permission to use
qualified private screening contractors and private screeners. There are
currently 6 airports participating in the Screening Partnership Program,
including Jackson Hole, Kansas City International, Greater Rochester
International, San Francisco International, Sioux Falls Regional, and
Tupelo Regional.
In July 2006, we received from 19 FSDs answers to follow-up questions on
their experiences with implementing pilot testing of SPOT or USP. This
included 14 FSDs that were not part of our initial rounds of interviews.
Nine of these 14 FSDs were from airports that participated in SPOT pilots.
The remaining 5 of 14 FSDs that were not part of our initial rounds of
interviews were from airports that were participants in USP pilots.
We conducted our work from March 2005 through March 2007 in accordance
with generally accepted government auditing standards.
Appendix II: Sources of SOP Changes
Of the 92 proposed screening changes considered by TSA between April 2005
and December 2005, 63 were submitted by TSA field staff, including Federal
Security Directors and Transportation Security Officers.1 Thirty proposed
screening changes were submitted by TSA headquarters officials. Last, TSA
senior leadership, such as the TSA Assistant Secretary, recommended 5 of
the 92 proposed screening changes considered during this time period. One
SOP modification was also proposed through a congressional inquiry. TSA's
solicitation of input from both field and headquarters officials regarding
changes to the passenger checkpoint screening SOP was consistent with
internal control standards,2 which suggest that there be mechanisms in
place for employees to recommend improvements in operations.
The FSDs with whom we met most frequently identified periodic conference
calls with the Assistant Secretary, the SOP Question and Answer mailbox,
or electronic mail to Security Operations officials as the mechanisms by
which they recommended changes to the SOP. The TSOs with whom we met
identified their chain of command and the SOP Question and Answer mailbox
as the primary mechanisms by which they submitted suggestions for new or
revised procedures. According to TSA officials, the SOP mailbox entails
FSDs and their staff, including TSOs, submitting suggestions, questions,
or comments to TSA's Security Operations division via electronic mail,
either directly or through their supervisors. Submissions are then
compiled and reviewed by a single Security Operations official, who
generates responses to the questions that have clear answers. However, for
submissions for which the appropriate response is not obvious or for
submissions that include a suggestion to revise the SOP, this official
forwards the submissions to other Security Operations officials for
further deliberation. SOP mailbox responses are provided to all TSA
airport officials. If TSA headquarters revised a screening procedure based
on a mailbox submission, the revision is noted in the mailbox response.
Thirty of the screening changes considered by TSA between April 2005 and
December 2005 were proposed by TSA headquarters officials, including
Security Operations officials, who are responsible for overseeing
implementation of checkpoint screening. According to Security Operations
officials, they recommended changes to checkpoint screening procedures
based on communications with TSA field officials and airport optimization
reviews. Security Operations officials conduct optimization reviews to
identify best practices and deficiencies in the checkpoint screening and
checked baggage screening processes. As part of these reviews, Security
Operations officials may also assess screening efficiency and whether TSOs
are implementing screening procedures correctly.
1There were 10 SOP modifications that were proposed both by multiple
sources. We attributed 9 of these proposed modifications to each of the
relevant sources. TSA did not identify the sources for the 1 remaining
modification that was proposed by multiple sources.
2 [36]GAO/AIMD-00-21 .3.1.
Other TSA headquarters divisions also suggested changes to passenger
checkpoint screening procedures. For example, the Office of Law
Enforcement recommended that there be an alternative screening procedure
for law enforcement officials who are escorting prisoners or protectees.
Previously, all armed law enforcement officers were required to sign a
logbook at the screening checkpoint, prior to entering the sterile area of
the airport. The officials in the Office of Passengers with Disabilities
also recommended changes to checkpoint screening procedures. For example,
in the interest of disabled passengers, they suggested that TSOs be
required to refasten all wheelchair straps and buckles undone during the
screening process.
Last, TSA senior leadership suggested 5 of the 92 procedural changes
considered by TSA between April 2005 and December 2005. TSA senior
leadership also proposed a procedure that would allow TSOs to conduct the
pat-down procedure on passengers of the opposite gender at airports with a
disproportionate ratio of male and female TSOs.
Appendix III: Comments from the Department of Homeland Security
Appendix IV: GAO Contact and Staff Acknowledgments
GAO Contact
Cathleen A. Berrick, (202) 512-3404 or [email protected]
Acknowledgments
In addition to the person named above, Maria Strudwick, Assistant
Director; David Alexander; Christopher W. Backley; Amy Bernstein; Kristy
Brown; Yvette Gutierrez-Thomas; Katherine N. Haeberle; Robert D. Herring;
Richard Hung; Christopher Jones, Stanley Kostyla; and Laina Poon made key
contributions to this report.
GAO Related Products
Aviation Security: TSA's Staffing Allocation Model Is Useful for
Allocating Staff among Airports, but Its Assumptions Should Be
Systematically Reassessed. [37]GAO-07-299 . Washington, D.C.: February 28,
2007
Aviation Security: Progress Made in Systematic Planning to Guide Key
Investment Decisions, but More Work Remains. [38]GAO-07-448T . Washington,
D.C.: February 13, 2007.
Homeland Security: Progress Has Been Made to Address the Vulnerabilities
Exposed by 9/11, but Continued Federal Action Is Needed to Further
Mitigate Security Risks. [39]GAO-07-375 . Washington, D.C.: January 24,
2007.
Aviation Security: TSA Oversight of Checked Baggage Screening Procedures
Could Be Strengthened [40]GAO-06-869 . Washington, D.C.: July 28, 2006.
Aviation Security: TSA Has Strengthened Efforts to Plan for the Optimal
Deployment of Checked Baggage Screening Systems, but Funding Uncertainties
Remain [41]GAO-06-875T . Washington, D.C.: June 29, 2006.
Aviation Security: Management Challenges Remain for the Transportation
Security Administration's Secure Flight Program. [42]GAO-06-864T .
Washington, D.C.: June 14, 2006.
Aviation Security: Further Study of Safety and Effectiveness and Better
Management Controls Needed if Air Carriers Resume Interest in Deploying
Less-than-Lethal Weapons. [43]GAO-06-475. Washington, D.C.: May 26, 2006.
Aviation Security: Enhancements Made in Passenger and Checked Baggage
Screening, but Challenges Remain. [44]GAO-06-371T . Washington, D.C.:
April 4, 2006.
Aviation Security: Transportation Security Administration Has Made
Progress in Managing a Federal Security Workforce and Ensuring Security at
U.S. Airports, but Challenges Remain. [45]GAO-06-597T . Washington, D.C.:
April 4, 2006.
Aviation Security: Progress Made to Set Up Program Using Private-Sector
Airport Screeners, but More Work Remains. [46]GAO-06-166 . Washington,
D.C.: March 31, 2006.
Aviation Security: Significant Management Challenges May Adversely Affect
Implementation of the Transportation Security Administration's Secure
Flight Program. [47]GAO-06-374T . Washington, D.C.: February 9, 2006.
Aviation Security: Federal Air Marshal Service Could Benefit from Improved
Planning and Controls. [48]GAO-06-203 . Washington, D.C.: November 28,
2005.
Aviation Security: Federal Action Needed to Strengthen Domestic Air Cargo
Security. [49]GAO-06-76 . Washington, D.C.: October 17, 2005.
Transportation Security Administration: More Clarity on the Authority of
Federal Security Directors Is Needed. [50]GAO-05-935 . Washington, D.C.:
September 23, 2005.
Aviation Security: Flight and Cabin Crew Member Security Training
Strengthened, but Better Planning and Internal Controls Needed.
[51]GAO-05-781 . Washington, D.C.: September 6, 2005.
Aviation Security: Transportation Security Administration Did Not Fully
Disclose Uses of Personal Information during Secure Flight Program Testing
in Initial Privacy Notes, but Has Recently Taken Steps to More Fully
Inform the Public. [52]GAO-05-864R . Washington, D.C.: July 22, 2005.
Aviation Security: Better Planning Needed to Optimize Deployment of
Checked Baggage Screening Systems. [53]GAO-05-896T . Washington, D.C.:
July 13, 2005.
Aviation Security: Screener Training and Performance Measurement
Strengthened, but More Work Remains. [54]GAO-05-457 . Washington, D.C.:
May 2, 2005.
Aviation Security: Secure Flight Development and Testing Under Way, but
Risks Should Be Managed as System Is Further Developed. [55]GAO-05-356 .
Washington, D.C.: March 28, 2005.
Aviation Security: Systematic Planning Needed to Optimize the Deployment
of Checked Baggage Screening Systems. [56]GAO-05-365 . Washington, D.C.:
March 15, 2005.
Aviation Security: Measures for Testing the Effect of Using Commercial
Data for the Secure Flight Program. [57]GAO-05-324 . Washington, D.C.:
February 23, 2005.
Transportation Security: Systematic Planning Needed to Optimize Resources.
[58]GAO-05-357T . Washington, D.C.: February 15, 2005.
Aviation Security: Preliminary Observations on TSA's Progress to Allow
Airports to Use Private Passenger and Baggage Screening Services.
[59]GAO-05-126 . Washington, D.C.: November 19, 2004.
General Aviation Security: Increased Federal Oversight Is Needed, but
Continued Partnership with the Private Sector Is Critical to Long-Term
Success. [60]GAO-05-144 . Washington, D.C.: November 10, 2004.
Aviation Security: Further Steps Needed to Strengthen the Security of
Commercial Airport Perimeters and Access Controls. [61]GAO-04-728 .
Washington, D.C.: June 4, 2004.
Transportation Security Administration: High-Level Attention Needed to
Strengthen Acquisition Function. [62]GAO-04-544 . Washington, D.C.: May
28, 2004.
Aviation Security: Challenges in Using Biometric Technologies.
[63]GAO-04-785T . Washington, D.C.: May 19, 2004.
Nonproliferation: Further Improvements Needed in U.S. Efforts to Counter
Threats from Man-Portable Air Defense Systems. [64]GAO-04-519 .
Washington, D.C.: May 13, 2004.
Aviation Security: Private Screening Contractors Have Little Flexibility
to Implement Innovative Approaches. [65]GAO-04-505T . Washington, D.C.:
April 22, 2004.
Aviation Security: Improvement Still Needed in Federal Aviation Security
Efforts. [66]GAO-04-592T . Washington, D.C.: March 30, 2004.
Aviation Security: Challenges Delay Implementation of Computer-Assisted
Passenger Prescreening System. [67]GAO-04-504T . Washington, D.C.: March
17, 2004.
Aviation Security: Factors Could Limit the Effectiveness of the
Transportation Security Administration's Efforts to Secure Aerial
Advertising Operations. [68]GAO-04-499R . Washington, D.C.: March 5, 2004.
Aviation Security: Computer-Assisted Passenger Prescreening System Faces
Significant Implementation Challenges. [69]GAO-04-385 . Washington, D.C.:
February 13, 2004.
Aviation Security: Challenges Exist in Stabilizing and Enhancing Passenger
and Baggage Screening Operations. [70]GAO-04-440T . Washington, D.C.:
February 12, 2004.
The Department of Homeland Security Needs to Fully Adopt a Knowledge-based
Approach to Its Counter-MANPADS Development Program. [71]GAO-04-341R .
Washington, D.C.: January 30, 2004.
Aviation Security: Efforts to Measure Effectiveness and Strengthen
Security Programs. [72]GAO-04-285T . Washington, D.C.: November 20, 2003.
Aviation Security: Federal Air Marshal Service Is Addressing Challenges of
Its Expanded Mission and Workforce, but Additional Actions Needed.
[73]GAO-04-242 . Washington, D.C.: November 19, 2003.
Aviation Security: Efforts to Measure Effectiveness and Address
Challenges. [74]GAO-04-232T . Washington, D.C.: November 5, 2003.
Airport Passenger Screening: Preliminary Observations on Progress Made and
Challenges Remaining. [75]GAO-03-1173 . Washington, D.C.: September 24,
2003.
Aviation Security: Progress since September 11, 2001, and the Challenges
Ahead. [76]GAO-03-1150T . Washington, D.C.: September 9, 2003.
Transportation Security: Federal Action Needed to Enhance Security
Efforts. [77]GAO-03-1154T . Washington, D.C.: September 9, 2003.
Transportation Security: Federal Action Needed to Help Address Security
Challenges. [78]GAO-03-843 . Washington, D.C.: June 30, 2003.
Federal Aviation Administration: Reauthorization Provides Opportunities to
Address Key Agency Challenges. [79]GAO-03-653T . Washington, D.C.: April
10, 2003.
Transportation Security: Post-September 11th Initiatives and Long-Term
Challenges. [80]GAO-03-616T . Washington, D.C.: April 1, 2003.
Airport Finance: Past Funding Levels May Not Be Sufficient to Cover
Airports' Planned Capital Development. [81]GAO-03-497T . Washington, D.C.:
February 25, 2003.
Transportation Security Administration: Actions and Plans to Build a
Results-Oriented Culture. [82]GAO-03-190 . Washington, D.C.: January 17,
2003.
Aviation Safety: Undeclared Air Shipments of Dangerous Goods and DOT's
Enforcement Approach. [83]GAO-03-22 . Washington, D.C.: January 10, 2003.
Aviation Security: Vulnerabilities and Potential Improvements for the Air
Cargo System. [84]GAO-03-344 . Washington, D.C.: December 20, 2002.
Aviation Security: Registered Traveler Program Policy and Implementation
Issues. [85]GAO-03-253 . Washington, D.C.: November 22, 2002.
Airport Finance: Using Airport Grant Funds for Security Projects Has
Affected Some Development Projects. [86]GAO-03-27. Washington, D.C.:
October 15, 2002.
Commercial Aviation: Financial Condition and Industry Responses Affect
Competition. [87]GAO-03-171T . Washington, D.C.: October 2, 2002.
Aviation Security: Transportation Security Administration Faces Immediate
and Long-Term Challenges. [88]GAO-02-971T . Washington, D.C.: July 25,
2002.
Aviation Security: Information Concerning the Arming of Commercial Pilots.
[89]GAO-02-822R. Washington, D.C.: June 28, 2002.
Aviation Security: Vulnerabilities in, and Alternatives for, Preboard
Screening Security Operations. [90]GAO-01-1171T . Washington, D.C.:
September 25, 2001.
Aviation Security: Weaknesses in Airport Security and Options for
Assigning Screening Responsibilities. [91]GAO-01-1165T . Washington, D.C.:
September 21, 2001.
Homeland Security: A Framework for Addressing the Nation's Efforts.
[92]GAO-01-1158T . Washington, D.C.: September 21, 2001.
Aviation Security: Terrorist Acts Demonstrate Urgent Need to Improve
Security at the Nation's Airports. [93]GAO-01-1162T . Washington, D.C.:
September 20, 2001.
Aviation Security: Terrorist Acts Illustrate Severe Weaknesses in Aviation
Security. [94]GAO-01-1166T . Washington, D.C.: September 20, 2001.
(440600)
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
[email protected] Automated answering system: (800) 424-5454 or (202)
512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
www.gao.gov/cgi-bin/getrpt?GAO-07-634 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Cathleen Berrick at (202) 512-3404 or
[email protected].
Highlights of [102]GAO-07-634 , a report to congressional requesters
April2007
AVIATION SECURITY
Risk, Experience, and Customer Concerns Drive Changes to Airline Passenger
Screening Procedures, but Evaluation and Documentation of Proposed Changes
Could Be Improved
[103]transparent illustrator graphic
The Transportation Security Administration's (TSA) most visible layer of
commercial aviation security is the screening of airline passengers at
airport checkpoints, where travelers and their carry-on items are screened
for explosives and other dangerous items by transportation security
officers (TSO). Several revisions made to checkpoint screening procedures
have been scrutinized and questioned by the traveling public and Congress
in recent years.
For this review, GAO evaluated (1) TSA's decisions to modify passenger
screening procedures between April 2005 and December 2005 and in response
to the alleged August 2006 liquid explosives terrorist plot, and (2) how
TSA monitored TSO compliance with passenger screening procedures. To
conduct this work, GAO reviewed TSA documents, interviewed TSA officials
and aviation security experts, and visited 25 airports of varying sizes
and locations.
[104]What GAO Recommends
In the March 2007 report that contained sensitive security information,
GAO recommended, and the Department of Homeland Security concurred, that
TSA develop sound methods to assess whether proposed screening changes
would achieve their intended purpose and generate complete documentation
on proposed screening changes that are deemed significant.
Between April 2005 and December 2005, proposed modifications to passenger
checkpoint screening standard operating procedures (SOP) were made for a
variety of reasons, and while a majority of the proposed modifications--48
of 92--were ultimately implemented at airports, TSA's methods for
evaluating and documenting them could be improved. SOP modifications were
proposed based on the professional judgment of TSA senior-level officials
and program-level staff. TSA considered the daily experiences of airport
staff, complaints and concerns raised by the traveling public, and
analysis of risks to the aviation system when proposing SOP modifications.
TSA also made efforts to balance the impact on security, efficiency, and
customer service when deciding which proposed modifications to implement,
as in the case of the SOP changes made in response to the alleged August
2006 liquid explosives terrorist plot. In some cases, TSA tested proposed
modifications at selected airports to help determine whether the changes
would achieve their intended purpose. However, TSA's data collection and
analyses could be improved to help TSA determine whether proposed
procedures that are operationally tested would achieve their intended
purpose. For example, TSA officials decided to allow passengers to carry
small scissors and tools onto aircraft based on their review of threat
information, which indicated that these items do not pose a high risk to
the aviation system. However, TSA did not conduct the necessary analysis
of data it collected to assess whether this screening change would free up
TSOs to focus on screening for high-risk threats, as intended. TSA
officials acknowledged the importance of evaluating whether proposed
screening procedures would achieve their intended purpose, but cited
difficulties in doing so, including time pressures to implement needed
security measures quickly. Finally, TSA's documentation on proposed
modifications to screening procedures was not complete. TSA documented the
basis--that is, the information, experience, or event that encouraged TSA
officials to propose the modifications--for 72 of the 92 proposed
modifications. In addition, TSA documented the reasoning behind its
decisions for half (26 of 44) of the proposed modifications that were not
implemented. Without more complete documentation, TSA may not be able to
justify key modifications to passenger screening procedures to Congress
and the traveling public.
TSA monitors TSO compliance with passenger checkpoint screening procedures
through its performance accountability and standards system and through
covert testing. Compliance assessments include quarterly observations of
TSOs' ability to perform particular screening functions in the operating
environment, quarterly quizzes to assess TSOs' knowledge of procedures,
and an annual knowledge and skills assessment. TSA uses covert tests to
evaluate, in part, the extent to which TSOs' noncompliance with procedures
affects their ability to detect simulated threat items hidden in
accessible property or concealed on a person. TSA airport officials have
experienced resource challenges in implementing these compliance
monitoring methods. TSA headquarters officials stated that they are taking
steps to address these challenges.
References
Visible links
27. http://www.gao.gov/cgi-bin/getrpt?GAO/AIMD-00-21.3.1
28. http://www.gao.gov/cgi-bin/getrpt?GAO-03-190
29. http://www.gao.gov/cgi-bin/getrpt?GAO-05-457
30. http://www.gao.gov/cgi-bin/getrpt?GAO-01-1008G
31. http://www.gao.gov/cgi-bin/getrpt?GAO-03-190
32. http://www.gao.gov/cgi-bin/getrpt?GAO-01-1008G
34. http://www.gao.gov/cgi-bin/getrpt?GAO/AIMD-00-21
35. http://www.gao.gov/cgi-bin/getrpt?GAO/AIMD-00-21
36. http://www.gao.gov/cgi-bin/getrpt?GAO/AIMD-00-21
37. http://www.gao.gov/cgi-bin/getrpt?GAO-07-299
38. http://www.gao.gov/cgi-bin/getrpt?GAO-07-448T
39. http://www.gao.gov/cgi-bin/getrpt?GAO-07-375
40. http://www.gao.gov/cgi-bin/getrpt?GAO-06-869
41. http://www.gao.gov/cgi-bin/getrpt?GAO-06-875T
42. http://www.gao.gov/cgi-bin/getrpt?GAO-06-864T
43. http://www.gao.gov/cgi-bin/getrpt?GAO-06-475.
44. http://www.gao.gov/cgi-bin/getrpt?GAO-06-371T
45. http://www.gao.gov/cgi-bin/getrpt?GAO-06-597T
46. http://www.gao.gov/cgi-bin/getrpt?GAO-06-166
47. http://www.gao.gov/cgi-bin/getrpt?GAO-06-374T
48. http://www.gao.gov/cgi-bin/getrpt?GAO-06-203
49. http://www.gao.gov/cgi-bin/getrpt?GAO-06-76
50. http://www.gao.gov/cgi-bin/getrpt?GAO-05-935
51. http://www.gao.gov/cgi-bin/getrpt?GAO-05-781
52. http://www.gao.gov/cgi-bin/getrpt?GAO-05-864R
53. http://www.gao.gov/cgi-bin/getrpt?GAO-05-896T
54. http://www.gao.gov/cgi-bin/getrpt?GAO-05-457
55. http://www.gao.gov/cgi-bin/getrpt?GAO-05-356
56. http://www.gao.gov/cgi-bin/getrpt?GAO-05-365
57. http://www.gao.gov/cgi-bin/getrpt?GAO-05-324
58. http://www.gao.gov/cgi-bin/getrpt?GAO-05-357T
59. http://www.gao.gov/cgi-bin/getrpt?GAO-05-126
60. http://www.gao.gov/cgi-bin/getrpt?GAO-05-144
61. http://www.gao.gov/cgi-bin/getrpt?GAO-04-728
62. http://www.gao.gov/cgi-bin/getrpt?GAO-04-544
63. http://www.gao.gov/cgi-bin/getrpt?GAO-04-785T
64. http://www.gao.gov/cgi-bin/getrpt?GAO-04-519
65. http://www.gao.gov/cgi-bin/getrpt?GAO-04-505T
66. http://www.gao.gov/cgi-bin/getrpt?GAO-04-592T
67. http://www.gao.gov/cgi-bin/getrpt?GAO-04-504T
68. http://www.gao.gov/cgi-bin/getrpt?GAO-04-499R
69. http://www.gao.gov/cgi-bin/getrpt?GAO-04-385
70. http://www.gao.gov/cgi-bin/getrpt?GAO-04-440T
71. http://www.gao.gov/cgi-bin/getrpt?GAO-04-341R
72. http://www.gao.gov/cgi-bin/getrpt?GAO-04-285T
73. http://www.gao.gov/cgi-bin/getrpt?GAO-04-242
74. http://www.gao.gov/cgi-bin/getrpt?GAO-04-232T
75. http://www.gao.gov/cgi-bin/getrpt?GAO-03-1173
76. http://www.gao.gov/cgi-bin/getrpt?GAO-03-1150T
77. http://www.gao.gov/cgi-bin/getrpt?GAO-03-1154T
78. http://www.gao.gov/cgi-bin/getrpt?GAO-03-843
79. http://www.gao.gov/cgi-bin/getrpt?GAO-03-653T
80. http://www.gao.gov/cgi-bin/getrpt?GAO-03-616T
81. http://www.gao.gov/cgi-bin/getrpt?GAO-03-497T
82. http://www.gao.gov/cgi-bin/getrpt?GAO-03-190
83. http://www.gao.gov/cgi-bin/getrpt?GAO-03-22
84. http://www.gao.gov/cgi-bin/getrpt?GAO-03-344
85. http://www.gao.gov/cgi-bin/getrpt?GAO-03-253
86. http://www.gao.gov/cgi-bin/getrpt?GAO-03-27.
87. http://www.gao.gov/cgi-bin/getrpt?GAO-03-171T
88. http://www.gao.gov/cgi-bin/getrpt?GAO-02-971T
89. http://www.gao.gov/cgi-bin/getrpt?GAO-02-822R.
90. http://www.gao.gov/cgi-bin/getrpt?GAO-01-1171T
91. http://www.gao.gov/cgi-bin/getrpt?GAO-01-1165T
92. http://www.gao.gov/cgi-bin/getrpt?GAO-01-1158T
93. http://www.gao.gov/cgi-bin/getrpt?GAO-01-1162T
94. http://www.gao.gov/cgi-bin/getrpt?GAO-01-1166T
102. http://www.gao.gov/cgi-bin/getrpt?GAO-07-634
*** End of document. ***