Aviation Security: Screener Training and Performance Measurement
Strengthened, but More Work Remains (02-MAY-05, GAO-05-457).
The screening of airport passengers and their checked baggage is
a critical component in securing our nation's commercial aviation
system. Since May 2003, GAO has issued six products related to
screener training and performance. This report updates the
information presented in the prior products and incorporates
results from GAO's survey of 155 Federal Security Directors--the
ranking Transportation Security Administration (TSA) authority
responsible for the leadership and coordination of TSA security
activities at the nation's commercial airports. Specifically,
this report addresses (1) actions TSA has taken to enhance
training for passenger and checked baggage screeners and
screening supervisors, (2) how TSA ensures that screeners
complete required training, and (3) actions TSA has taken to
measure and enhance screener performance in detecting threat
objects.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-05-457
ACCNO: A23117
TITLE: Aviation Security: Screener Training and Performance
Measurement Strengthened, but More Work Remains
DATE: 05/02/2005
SUBJECT: Airport security
Checked baggage screening
Employee training
Homeland security
Internal controls
Passengers
Performance measures
Search and seizure
Training utilization
Policies and procedures
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-05-457
United States Government Accountability Office
GAO Report to the Chairman, Subcommittee on Aviation, Committee on
Transportation and Infrastructure, House of Representatives
May 2005
AVIATION SECURITY
Screener Training and Performance Measurement Strengthened, but More Work
Remains
GAO-05-457
[IMG]
May 2005
AVIATION SECURITY
Screener Training and Performance Measurement Strengthened, but More Work
Remains
What GAO Found
TSA has initiated a number of actions designed to enhance screener
training, such as updating the basic screener training course. TSA also
established a recurrent training requirement and introduced the Online
Learning Center, which makes self-guided training courses available over
TSA's intranet and the Internet. Even with these efforts, Federal Security
Directors reported that insufficient screener staffing and a lack of
high-speed Internet/intranet connectivity at some training facilities have
made it difficult to fully utilize training programs and to meet the
recurrent training requirement of 3 hours per week, averaged over a
quarter year, within regular duty hours. TSA acknowledged that challenges
exist in recurrent training delivery and is taking steps to address these
challenges, including factoring training into workforce planning efforts
and distributing training through written materials and CD-ROMs. However,
TSA has not established a plan prioritizing the deployment of high-speed
Internet/intranet connectivity to all airport training facilities to
facilitate screener access to training materials.
TSA lacks adequate internal controls to provide reasonable assurance that
screeners receive legislatively mandated basic and remedial training, and
to monitor its recurrent training program. Specifically, TSA policy does
not clearly specify the responsibility for ensuring that screeners have
completed all required training. In addition, TSA officials have no formal
policies or methods for monitoring the completion of required training and
were unable to provide documentation identifying the completion of
remedial training.
TSA has implemented and strengthened efforts to measure and enhance
screener performance. For example, TSA has increased the number of covert
tests it conducts at airports, which test screeners' ability to detect
threat objects on passengers, in their carry-on baggage, and in checked
baggage. These tests identified that overall, weaknesses and
vulnerabilities continue to exist in passenger and checked baggage
screening systems at airports of all sizes, at airports with federal
screeners, and at airports with private-sector screeners. While these test
results are an indicator of performance, they cannot solely be used as a
comprehensive measure of any airport's screening performance or any
individual screener's performance. We also found that TSA's efforts to
measure and enhance screener performance have primarily focused on
passenger screening, not checked baggage screening. For example, TSA only
uses threat image software on passenger screening X-ray machines, and the
recertification testing program does not include an image recognition
module for checked baggage screeners. TSA is taking steps to address the
overall imbalance in passenger and checked baggage screening performance
data. TSA also established performance indexes for the passenger and
checked baggage screening systems, to identify an overall desired level of
performance. However, TSA has not established performance targets for each
of the component indicators that make up the performance indexes,
including performance targets for covert testing. TSA plans to finalize
these targets by the end of fiscal year 2005.
United States Government Accountability Office
Contents
Letter
Results in Brief
Background
TSA Has Enhanced and Expanded Training, but Some Screeners
Have Encountered Difficulty Accessing and Completing Recurrent Training
TSA Lacks Adequate Controls to Provide Reasonable Assurance That Screeners
Receive Required Training
Progress Has Been Made in Implementing Tools to Measure and Enhance
Screener Performance, but Key Performance Targets Have Not Been Finalized
Conclusions
Recommendations for Executive Action
Agency Comments and Our Evaluation
1
3 7
14
26
30 45 46 47
Appendix I �Summary of Previous Findings Related to Screener
Training and Performance
Appendix II Objectives, Scope, and Methodology
Appendix III �TSA Screener Training Tools Designed to Help Improve
Screener Performance
Appendix IV �Summary of TSA's Short-Term Action Items for
Strengthening Passenger Screener Performance
Appendix V �Comments from the Department of Homeland Security 58
Appendix VI GAO Contacts and Acknowledgments 60
Tables
Table 1: Categories of Required Training Provided to TSA Screeners
Table 2: Training Needed to Enhance Screener Supervision, as Reported by
FSDs
Table 3: Checkpoint and Checked Baggage Tests Conducted by OIAPR,
September 10, 2002-September 30, 2004
Table 4: Modules Included in Recertification Knowledge and Skills
Assessment
Table 5: Screener Recertification Module Testing Percentage Pass Rates,
October 2003-March 2004
Table 6: Recertification Testing Modules by Screening Function
Table 7: Components of TSA's Performance Indexes 13
20
33
38
39 40 44
Figures
Figure 1: Commercial Airports by Airport Security Category as of December
2003 8 Figure 2: Passenger Checkpoint and Checked Baggage Screening
Operations 11 Figure 3: Description of Screening-Related Positions 12
Figure 4: Newly Hired Screeners Trained as Dual-Function
Screeners between April 1, 2004, and September 1, 2004 16 Figure 5:
Percentage of Airports Reported to Have High-Speed Connectivity for
Training Purposes as of October 2004 25 Figure 6: Screener Recertification
Results, October 2003- March 2004 39
Abbreviations
ATSA Aviation and Transportation Security Act
DHS Department of Homeland Security
DOT Department of Transportation
EDS explosive detection system
ETD electronic trace detection
FAA Federal Aviation Administration
FSD Federal Security Director
IED improvised explosive device
MBS modular bomb set
MTAT Mobile Training Assist Team
OIAPR Office of Internal Affairs and Program Review
OIG Office of the Inspector General
SIDA security identification display area
TIP Threat Image Projection
TSA Transportation Security Administration
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office Washington, DC 20548
May 2, 2005
The Honorable John Mica
Chairman, Subcommittee on Aviation
Committee on Transportation and Infrastructure
House of Representatives
Dear Mr. Chairman:
The screening of airport passengers and their checked baggage is a
critical
component in securing our nation's commercial aviation system. In an
effort to strengthen the security of commercial aviation, the President
signed the Aviation and Transportation Security Act on
November 19, 2001. The act created the Transportation Security
Administration (TSA) and mandated actions designed to strengthen
aviation security, including requiring that TSA assume responsibility for
conducting passenger and checked baggage screening at over 450
commercial airports in the United States by November 19, 2002. It has
been over 2 years since TSA assumed this responsibility, and the agency
has spent billions of dollars and implemented a wide range of initiatives
to
enhance its passenger and checked baggage screening operations. Despite
the attention to passenger and checked baggage screening operations,
however, concerns about the effectiveness of the screening system
remain. For example, covert testing conducted by TSA's Office of Internal
Affairs and Program Review and the Department of Homeland Security's
(DHS) Office of Inspector General identified weaknesses in the ability of
screeners to detect threat objects. (The results of our analysis of TSA's
covert testing data and test program are included in a separate classified
GAO report.)
To determine the progress TSA has made in strengthening its passenger
and checked baggage screening operations, the Subcommittee on
Aviation, House Committee on Transportation and Infrastructure,
requested that we examine TSA efforts to train screeners and to measure
and enhance screener performance. Since we began our work in May 2003,
we have issued six products that address issues related to screener
training and performance, including four to this Subcommittee (see app.
I). This report updates some of the information presented in our prior
products. In addition, it incorporates results from our surveys of 155
Federal Security Directors (FSD).1 The surveys were designed to obtain
information related to, among other issues, TSA's efforts to train
screeners and supervisors and assess screener performance in detecting
threat objects. Specifically, this report addresses the following
questions: (1) What actions has TSA taken to enhance training for
passenger and checked baggage screeners and screening supervisors? (2) How
does TSA ensure that screeners complete required training? (3) What
actions has TSA taken to measure and enhance screener performance in
detecting threat objects?
In conducting our work, we reviewed TSA documentation related to screener
training requirements and performance testing. We also analyzed data from
our survey responses from all 155 FSDs about screener training,
supervision and performance. We also visited 29 airports of various sizes
and geographic locations to obtain a cross-section of all airports,
including the 5 airports with private-sector screeners.2 To gain a better
understanding of training and performance issues, during these visits, we
interviewed FSDs, members of their management teams, passenger and checked
baggage screeners, and airport officials. However, information obtained
during these visits cannot be generalized to all airports across the
nation. Additionally, we interviewed officials at TSA headquarters and
TSA's transportation security laboratory about their experiences with
training and screener performance. We compared TSA practices and
1The Federal Security Director is the ranking TSA authority responsible
for the leadership and coordination of TSA security activities at the
nation's commercial airports. We sent two surveys-a general survey and an
airport-specific survey-to all 155 Federal Security Directors on March 23,
2004. In the general survey, we asked each Federal Security Director to
answer security-related questions that pertain to all of the airports for
which he/she is responsible. In the airport-specific survey, we asked
Federal Security Directors a number of airport-specific questions about
screening and other security concerns for one or two airports, depending
on the number of airports they were responsible for. By early May 2004, we
had received responses from 100 percent of the Federal Security Directors
for both surveys.
2Pursuant to section 108 of the Aviation and Transportation Security Act
(ATSA), TSA conducted a 2-year private screening pilot program at five
airports-one in each airport security category (Pub. L. No. 107-71). The
mission of the pilot, as defined by TSA, was to test the effectiveness of
using private screening contractors in a post-September 11 environment.
The pilot concluded on November 18, 2004. On November 19, 2004, consistent
with ATSA, TSA began allowing airports to apply to opt out of using
federal screeners in favor of private contractors. For additional
information on TSA's progress in developing the opt-out program, see GAO,
Aviation Security: Preliminary Observations on TSA's Progress to Allow
Airports to Use Private Passenger and Baggage Screening Services,
GAO-05-126 (Washington, D.C.: Nov. 19, 2004).
procedures for monitoring completion of training with the Comptroller
General's Standards for Internal Control in the Federal Government. We
assessed the reliability of the data we acquired from TSA regarding
screener testing and training completion, and found the data to be
sufficiently reliable for our purposes. A more detailed description of our
scope and methodology is contained in appendix II.
We conducted our work from May 2003 through April 2005 in accordance with
generally accepted government auditing standards.
Results in Brief
TSA has initiated a number of actions designed to enhance passenger
screener, checked baggage screener, and screener supervisor training.
However, at some airports screeners encountered difficulty accessing and
completing recurrent (refresher) training because of technological and
staffing constraints. Among the actions TSA has taken to enhance training
are changes and updates to the basic training program. For example, TSA
added a modular dual-function course during basic training that covers
passenger and checked baggage screening functions and allows newly hired
screeners to perform either function upon completion of the training. TSA
also established a requirement for recurrent screener training and
developed and introduced the Online Learning Center, which makes
self-guided training courses available to employees over TSA's intranet
and the Internet. In addition, TSA has provided Federal Security Directors
with hands-on training tools to use for local recurrent training and
testing. Finally, TSA has taken steps to provide leadership and technical
training to Screening Supervisors. Despite these improvements, some
Federal Security Directors, in response to open-ended survey questions,
identified a desire for more training in specific areas, including
leadership, communication, and supervision. Further in survey responses,
Federal Security Directors reported that largely because of insufficient
screener staffing, screeners were not always able to meet the recurrent
training requirement within regular duty hours. Federal Security Directors
at some airports also reported ongoing problems with a lack of high-speed
Internet/intranet connectivity, which severely limited screener access to
the Online Learning Center and diminished its value as a learning tool. As
of October 2004, nearly half of the screener workforce did not have
highspeed access to the Online Learning Center at their training facility.
TSA has acknowledged that challenges exist in recurrent screener training
delivery and is taking steps to address these challenges, including
factoring training requirements into workforce planning efforts and
distributing training through written materials and CD-ROMs until full
Internet/intranet connectivity is achieved. However, TSA does not have a
plan for prioritizing and scheduling the deployment of high-speed
connectivity to all airport training facilities once funding is available.
The absence of such a plan limits TSA's ability to make prudent decisions
about how to move forward with deploying connectivity to all airports to
provide screeners access to online training.
TSA lacks adequate internal controls to provide reasonable assurance that
screeners are receiving legislatively mandated basic and remedial
training, and to monitor the status of its recurrent training program. The
Comptroller General's Standards for Internal Control call for (1) areas of
authority and responsibility to be clearly defined and appropriate lines
of reporting established, (2) transactions and other significant events to
be documented clearly and documentation to be readily available for
examination, and (3) controls generally to be designed to ensure that
ongoing monitoring occurs in the course of normal operations.3 In
addition, the standards also require that information be communicated
within an organization to enable individuals to carry out their internal
control responsibilities. However, our review of TSA's training program
noted weaknesses in each of these control areas. First, TSA policy does
not clearly define the responsibility for ensuring that screeners have
completed all required training. Second, TSA officials were unable to
produce documentation of remedial training completion for our examination.
Third, TSA has no formal policies for monitoring the completion of
required training. Moreover, because of the lack of highspeed
Internet/intranet connectivity at some airports, staff had to manually
input training data that would otherwise be recorded automatically by the
Online Learning Center, making it challenging for some airports to keep
accurate and up-to-date training records. TSA headquarters officials
acknowledged that it can be difficult for airports to keep the Online
Learning Center up-to-date with the most recent training records without
high-speed connectivity.
TSA has improved its efforts to measure and enhance screener performance.
In September 2003, we first reported on the need for TSA to strengthen its
efforts to measure and enhance screener performance. At that time, TSA had
collected limited data on screener performance. Specifically, limited
covert testing-unannounced undercover tests in which TSA agents attempt to
pass threat objects through screening checkpoints and in checked
baggage-had been performed, the Threat
3GAO, Standards for Internal Control in the Federal Government,
GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999).
Image Projection system was not fully operational,4 and TSA had not fully
implemented the annual screener recertification program.5 Since then, TSA
has implemented and strengthened efforts to collect screener performance
data as part of its overall effort to enhance screener performance. For
example, TSA has increased the number of covert tests it conducts at
airports. These tests have identified that overall, weaknesses and
vulnerabilities continue to exist in the passenger and checked baggage
screening systems at airports of all sizes, at airports with federal
screeners, and at airports with private-sector screeners. While these test
results are an indicator of screener performance, they cannot solely be
used as a comprehensive measure of any airport's screening performance or
any individual screener's performance. TSA has also implemented other
efforts to measure and enhance screener performance. However, these
efforts have primarily focused on passenger screening, not checked baggage
screening. Specifically, the Threat Image Projection system is only
available for passenger screening; the recertification testing program
does not include an image recognition module for checked baggage
screeners; and the screener performance improvement study focused solely
on passenger screeners. TSA is taking steps to address the overall
imbalance in passenger and baggage screening performance data, including
working toward implementing the Threat Image Projection system for checked
baggage screening and developing an image recognition module for checked
baggage screener recertification. TSA has also implemented a number of
other improvements, which it identified in a passenger screener
performance improvement study and incorporated into a screener performance
improvement plan. Furthermore, TSA has established two performance indexes
for the screening systems-one for passenger and one for checked baggage
screening.6 These indexes measure overall performance through a composite
of component indicators. However, TSA has not established performance
targets for each of the component indicators-such as covert testing-which
would
4The Threat Image Projection system is designed to test screeners'
detection capabilities by projecting threat images, including images of
guns and explosives, into bags as they are screened. Screeners are
responsible for positively identifying the threat image and calling for
the bag to be searched.
5ATSA requires that TSA collect performance information on all passengers
and baggage screeners by conducting an annual proficiency evaluation to
ensure each screener continues to meet all qualifications and standards
related to the functions he or she performs.
6TSA's performance indexes show how well screening systems are functioning
on a scale of 1 to 5, using a weighted average of the values of four
distinct performance indicators.
allow it to draw more meaningful conclusions about its performance and
most effectively direct its improvement efforts. Although TSA has not yet
established performance targets for each of the component indicators, TSA
plans to finalize performance targets for the indicators by the end of
fiscal year 2005.
Certain information we obtained and analyzed regarding screener training
and performance is classified or is considered by TSA to be sensitive
security information. Accordingly, the results of our review of this
information are not included in this report.7
To help ensure access to and completion of required training, we are
making recommendations to the Secretary of the Department of Homeland
Security to direct the Assistant Secretary, Transportation Security
Administration, to (1) develop a plan for completing the deployment of
high-speed Internet/intranet connectivity to all TSA's airport training
facilities and (2) establish appropriate responsibilities and other
internal controls for monitoring and documenting screener compliance with
training requirements.
We provided a draft copy of this report to DHS for review. DHS, in its
written comments, generally concurred with the findings and
recommendations in the report and agreed that efforts to implement our
recommendations are critical to successful passenger and checked baggage
screening training and performance. DHS described some actions TSA had
taken or planned to take to implement these recommendations. DHS also
stated that TSA had already developed a plan for prioritizing and
scheduling the deployment of high-speed Internet/intranet connectivity to
all TSA airport training facilities. However, although we requested a copy
of the plan several times during our review and after receiving written
comments from DHS, TSA did not provide us with a copy of this plan.
Therefore, we cannot assess the extent to which the plan DHS referenced in
its written comments fulfills our recommendation. Additionally, DHS stated
that it is taking steps to define responsibilities for monitoring the
7We issued two additional reports detailing the results of our review,
which discuss information deemed to be classified or sensitive security
information. The report containing sensitive security information is GAO,
Aviation Security: Screener Training and Performance Measurement
Strengthened, but More Work Remains, GAO-05-143SU (Washington D.C.: Feb.
28, 2005). The report containing classified and sensitive security
information is GAO, Aviation Security: Results of Transportation Security
Administration's Covert Testing for Passenger and Checked Baggage
Screening for September 2002 through September 2004, GAO-05-437C
(Washington D.C.: Apr. 7, 2005).
Background
completion of required training, and to insert this accountability into
performance plans of all TSA supervisors. The full text of DHS's comments
is included in appendix V.
The performance of passenger and checked baggage screeners in detecting
threat objects at the nation's airports has been a long-standing concern.
In 1978, screeners failed to detect 13 percent of the potentially
dangerous objects that Federal Aviation Administration (FAA) agents
carried through airport screening checkpoints during tests. In 1987,
screeners did not detect 20 percent of the objects in similar tests. In
tests conducted during the late 1990s, as the testing objects became more
realistic, screeners' abilities to detect dangerous objects declined
further. In April 2004, we, along with the DHS Office of the Inspector
General (OIG), testified that the performance of screeners continued to be
a concern. More recent tests conducted by TSA's Office of Internal Affairs
and Program Review (OIAPR) also identified weaknesses in the ability of
screeners to detect threat objects, and separate DHS OIG tests identified
comparable screener performance weaknesses. In its July 2004 report, The
National Commission on Terrorist Attacks Upon the United States, known
widely as the 9/11 Commission, also identified the need to improve
screener performance and to better understand the reasons for performance
problems.8
After the terrorist attacks of September 11, 2001, the President signed
the Aviation and Transportation Security Act (ATSA) into law on November
19, 2001, with the primary goal of strengthening the security of the
nation's aviation system. ATSA created TSA as an agency with
responsibility for securing all modes of transportation, including
aviation.9 As part of this responsibility, TSA oversees security
operations at the nation's more than 450 commercial airports, including
passenger and checked baggage screening operations. Prior to the passage
of ATSA, air carriers were responsible for screening passengers and
checked baggage, and most used private security firms to perform this
function. FAA was responsible for ensuring compliance with screening
regulations.
8The National Commission on Terrorist Attacks Upon the United States: The
9/11
Commission Report (Washington, D.C., July 2004).
9Consistent with ATSA, TSA was created as an agency within the Department
of
Transportation (DOT) with responsibility for securing all modes of
transportation,
including aviation. The Homeland Security Act of 2002, signed into law on
November 25, 2002, transferred TSA from the DOT to the new Department of
Homeland
Security (Pub. L. No. 107-296).
Today, TSA security activities at airports are overseen by FSDs. Each FSD
is responsible for overseeing security activities, including passenger and
checked baggage screening, at one or more commercial airports. TSA
classifies the over 450 commercial airports in the United States into one
of five security risk categories (X, I, II, III, and IV) based on various
factors, such as the total number of takeoffs and landings annually, the
extent to which passengers are screened at the airport, and other special
security considerations. In general, category X airports have the largest
number of passenger boardings and category IV airports have the smallest.
TSA periodically reviews airports in each category and, if appropriate,
updates airport categorizations to reflect current operations. Figure 1
shows the number of commercial airports by airport security category as of
December 2003.
Figure 1: Commercial Airports by Airport Security Category as of December 2003
21 Airports
61 Airports
51 Airports
124 Airports
Total: 454
Category
X
I
II
III
IV
Source: GAO analysis of TSA data.
Note: TSA periodically reviews and updates airport categories to reflect
current operations. We used the categories in place in December 2003 to
conduct our analyses during this review.
In addition to establishing TSA and giving it responsibility for passenger
and checked baggage screening operations, ATSA set forth specific
enhancements to screening operations for TSA to implement, with deadlines
for completing many of them. These requirements included
o assuming responsibility for screeners and screening operations at more
than 450 commercial airports by November 19, 2002;
o establishing a basic screener training program composed of a minimum
of 40 hours of classroom instruction and 60 hours of on-the-job training;
o conducting an annual proficiency review of all screeners;
o conducting operational testing of screeners;10
o requiring remedial training for any screener who fails an operational
test; and
o screening all checked baggage for explosives using explosives
detection systems by December 31, 2002.11
Passenger screening is a process by which authorized TSA personnel inspect
individuals and property to deter and prevent the carriage of any
unauthorized explosive, incendiary, weapon, or other dangerous item aboard
an aircraft or into a sterile area.12 Passenger screeners must inspect
individuals for prohibited items at designated screening locations.13 The
four passenger screening functions are:
10TSA defines an operational screening test as any covert test of a
screener conducted by TSA, on any screening function, to assess the
screener's threat item detection ability and/or adherence to TSA-approved
procedures.
11Pursuant to the Homeland Security Act, the deadline for screening all
checked baggage using explosive detection systems was, in effect, extended
until December 31, 2003.
12Sterile areas are located within the terminal where passengers wait
after screening to board departing aircraft. Access to these areas is
controlled by TSA screeners at checkpoints where they conduct physical
screening of passengers and their carry-on baggage for weapons and
explosives.
13Screeners must deny passage beyond the screening location to any
individual or property that has not been screened or inspected in
accordance with passenger screening standard operating procedures. If an
individual refuses to permit inspection of any item, that item must not be
allowed into the sterile area or aboard an aircraft.
o X-ray screening of property,
o walk-through metal detector screening of individuals,
o hand-wand or pat-down screening of individuals, and
o physical search of property and trace detection for explosives.
Checked baggage screening is a process by which authorized security
screening personnel inspect checked baggage to deter, detect, and prevent
the carriage of any unauthorized explosive, incendiary, or weapon onboard
an aircraft. Checked baggage screening is accomplished through the use of
explosive detection systems14 (EDS) or explosive trace detection (ETD)
systems,15 and through the use of alternative means, such as manual
searches, K-9 teams, and positive passenger bag match,16 when EDS and ETD
systems are unavailable on a temporary basis. Figure 2 provides an
illustration of passenger and checked baggage screening operations.
14Explosive detection systems use probing radiation to examine objects
inside baggage and identify the characteristic signatures of threat
explosives. EDS equipment operates in an automated mode.
15Explosive trace detection works by detecting vapors and residues of
explosives. Human operators collect samples by rubbing bags with swabs,
which are chemically analyzed to identify any traces of explosive
materials.
16Positive passenger bag match is an alternative method of screening
checked baggage, which requires that the passenger be on the same aircraft
as the checked baggage.
There are several positions within TSA for employees that perform and
directly supervise passenger and checked baggage screening functions.
Figure 3 provides a description of these positions.
Figure 3: Description of Screening-Related Positions
Source: GAO analysis of TSA data.
To prepare screeners to perform screening functions, to keep their skills
current, and to address performance deficiencies, TSA provides three
categories of required screener training. Table 1 provides a description
of the required training.
Table 1: Categories of Required Training Provided to TSA Screeners
Category of
training Description of training requirement
Basic training ATSA requires newly hired screeners to complete basic
training composed of a minimum of 40 hours of classroom instruction and 60
hours of on-the-job training before they independently perform screening
functions.
Recurrent training TSA policy requires screeners to complete 3 hours of
skills refresher training per week averaged over each quarter.
Remedial training ATSA requires screeners who fail any operational test
to complete remedial training on the function they failed before they
resume performing that function.
Source: ATSA and TSA.
In September 2003, we reported on our preliminary observations of TSA's
efforts to ensure that screeners were effectively trained and supervised
and to measure screener performance.17 We found that TSA had established
and deployed a basic screener training program and required remedial
training but had not fully developed or deployed a recurrent training
program for screeners or supervisors. We also reported that TSA had
collected limited data to measure screener performance. Specifically, TSA
had conducted limited covert testing, the Threat Image Projection System
was not fully operational, and TSA had not implemented the annual screener
proficiency testing required by ATSA. In subsequent products, we reported
progress TSA had made in these areas and challenges TSA continued to face
in making training available to screeners and in measuring and enhancing
screener performance. A summary of our specific findings is included in
appendix I.
17GAO, Airport Passenger Screening: Preliminary Observations on Progress
Made and Challenges Remaining, GAO-03-1173 (Washington, D.C.: Sept. 24,
2003).
TSA Has Enhanced and Expanded Training, but Some Screeners Have Encountered
Difficulty Accessing and Completing Recurrent Training
TSA has taken a number of actions to enhance the training of screeners and
Screening Supervisors but has encountered difficulties in providing access
to recurrent training. TSA has enhanced basic training by, among other
things, adding a dual-function (passenger and checked baggage) screening
course for new employees. Furthermore, in response to the need for
frequent and ongoing training, TSA has implemented an Online Learning
Center with self-guided training courses available to employees over TSA's
intranet and the Internet and developed and deployed a number of hands-on
training tools.18 Moreover, TSA now requires screeners to participate in 3
hours of recurrent training per week, averaged over each quarter year. TSA
has also implemented leadership and technical training programs for
Screening Supervisors. However, some FSDs, in response to open-ended
survey questions, identified a desire for more training in specific areas,
including leadership, communication, and supervision. Further, despite the
progress TSA has made in enhancing and expanding screener and supervisory
training, TSA has faced challenges in providing access to recurrent
training. FSDs reported that insufficient staffing and a lack of
high-speed Internet/intranet connectivity at some training facilities have
made it difficult to fully utilize these programs and to meet training
requirements. TSA has acknowledged that challenges exist in recurrent
screener training delivery and is taking steps to address these
challenges, including factoring training requirements into workforce
planning efforts and distributing training through written materials and
CD-ROMs until full Internet/intranet connectivity is achieved. However,
TSA does not have a plan for prioritizing and scheduling the deployment of
high-speed connectivity to all airport training facilities once funding is
available. The absence of such a plan limits TSA's ability to make prudent
decisions about how to move forward with deploying connectivity to all
airports to provide screeners access to online training.
TSA Has Enhanced Basic Screener Training
TSA has enhanced its basic screener training program by updating the
training to reflect changes to standard operating procedures, deploying a
new dual-function (passenger and checked baggage screening) basic training
curriculum, and allowing the option of training delivery by local staff.
As required by ATSA, TSA established a basic training program for
18The Online Learning Center is TSA's central, official electronic source
of all training and related performance/development accomplishments for
TSA employees. It serves as the delivery platform for online training and
is the official repository for TSA training records. TSA launched the
Online Learning Center on October 31, 2003.
screeners composed of a minimum of 40 hours of classroom instruction and
60 hours of on-the-job training. TSA also updated the initial basic
screener training courses at the end of 2003 to incorporate changes to
standard operating procedures and directives, which contain detailed
information on how to perform TSA-approved screening methods. However, a
recent study by the DHS OIG found that while incorporating the standard
operating procedures into the curricula was a positive step, a number of
screener job tasks were incompletely addressed in or were absent from the
basic training courses.19
In addition to updates to the training curriculum, in April 2004, TSA
developed and implemented a new basic screener training program,
dualfunction screener training that covers the technical aspects of both
passenger and checked baggage screening. Initially, new hire basic
training was performed by a contractor and provided a screener with
training in either passenger or checked baggage screening functions. A
screener could then receive basic training in the other function later, at
the discretion of the FSD, but could not be trained in both functions
immediately upon hire. The new dual-training program is modular in design.
Thus, FSDs can chose whether newly hired screeners will receive
instruction in one or both of the screening functions during the initial
training. In addition, the individual modules can also be used to provide
recurrent training, such as refreshing checked baggage screening skills
for a screener who has worked predominately as a passenger screener. TSA
officials stated that this new approach provides the optimum training
solution based on the specific needs of each airport and reflects the fact
that at some airports the FSD does not require all screeners to be fully
trained in both passenger and checked baggage screening functions.
Some FSDs, particularly those at smaller airports, have made use of the
flexibility offered by the modular design of the new course to train
screeners immediately upon hire in both passenger and checked baggage
screening functions. Such training up front allows FSDs to use screeners
for either the passenger or the checked baggage screening function,
immediately upon completion of basic training. Figure 4 shows that 58
percent (3,324) of newly hired screeners trained between April 1, 2004,
and September 1, 2004, had completed the dual-function training.
19Department of Homeland Security, Office of Inspector General: An
Evaluation of the Transportation Security Administration's Screener
Training and Methods of Testing, OIG-04-045 (Washington, D.C., Sept.
2004).
Figure 4: Newly Hired Screeners Trained as Dual-Function Screeners between
April 1, 2004, and September 1, 2004
Airport category: X I II III IV All airports
Total number of airports: 21 61 57 124 197 454
Percentage dual-trained
Percentage trained as passenger screeners only
Percentage trained as checked baggage screeners only
Source: GAO analysis of TSA data.
In April 2004, TSA also provided FSDs with the flexibility to deliver
basic screener training using local instructors. TSA's Workforce
Performance and Training Office developed basic screener training
internally, and initially, contractors delivered all of the basic
training. Since then, TSA has provided FSDs with the discretion to provide
the training using local TSA employees or to use contractors. The
flexibility to use local employees allows FSDs and members of the screener
workforce to leverage their first-hand screening knowledge and experience
and address situations unique to individual airports.20 As of December 10,
2004, TSA had trained 1,021 local FSD staff (representing 218 airports) in
how to instruct the dual-function screener training course. TSA officials
stated that they expect the use of TSA-approved instructors to increase
over time.
20For local employees to be approved as instructors by TSA, they must be
nominated by an FSD and have a current or prior instructor certification
by a recognized training and development organization or have had at least
2 years of experience as an instructor. In addition, local TSA instructors
must have successfully completed the course of instruction they will be
teaching and demonstrate instructional skills by assisting a TSA-approved
instructor in classroom instruction and monitoring actual classroom
instruction.
TSA Has Deployed Recurrent Screener Training and Provided Additional
Training Tools, but Some FSDs Identified the Need for More Courses
TSA has taken several steps to deploy a recurrent screener training
program, including implementing the Online Learning Center-which includes
several recurrent training modules developed by TSA- implementing a
recurrent training requirement, and providing hands-on training tools.
According to TSA, comprehensive and frequent training is key to a
screener's ability to detect threat objects. In September 2003, we
reported that TSA had not fully developed or deployed a recurrent training
program for passenger screeners. At that time, some FSDs expressed concern
with the lack of training available to the screener workforce, and in the
absence of headquarters-provided training, they were developing and
implementing locally based recurrent training programs. Similar to our
findings, TSA's April 2004 consultant study reported that most FSDs and
their staffs were generally unsatisfied with the training support provided
by headquarters.21 Specifically, the study found that:
"Numerous interviews revealed concerns with training curriculum,
communication, and coordination issues that directly affect security
screening. Unsatisfied with the quantity and breadth of topics, many
Training Coordinators have developed supplementary lectures on both
security and non-security related topics. These additional lectures...have
been very highly received by screeners."
In October 2003, TSA introduced the Online Learning Center to provide
screeners with remote access to self-guided training courses. As of
September 14, 2004, TSA had provided access to over 550 training courses
via the Online Learning Center and made the system available via the
Internet and its intranet. TSA also developed and deployed a number of
hands-on training modules and associated training tools for screeners at
airports nationwide. These training modules cover topics including
handwanding and pat-down techniques, physical bag searches, X-ray images,
prohibited items, and customer service. Additionally, TSA instituted
another module for the Online Learning Center called Threat in the
Spotlight, that, based on intelligence TSA receives, provides screeners
with the latest in threat information regarding terrorist attempts to get
threat objects past screening checkpoints. Appendix III provides a summary
of the recurrent training tools TSA has deployed to airports and the
modules currently under development.
21In April 2004, a TSA-sponsored independent evaluation of screening
operations was completed by a consultant (Private Screening Operations
Performance Evaluation Summary Report, BearingPoint, Apr. 16, 2004). The
study was designed to evaluate the performance of federal and private
screening operations.
In December 2003, TSA issued a directive requiring screeners to receive 3
hours of recurrent training per week averaged over a quarter year. One
hour is required to be devoted to X-ray image interpretation and the other
2 hours to screening techniques, review of standard operating procedures,
or other mandatory administrative training, such as ethics and privacy act
training.
In January 2004, TSA provided FSDs with additional tools to facilitate and
enhance screener training. Specifically, TSA provided airports with at
least one modular bomb set (MBS II) kit-containing components of an
improvised explosive device-and one weapons training kit, in part because
screeners had consistently told TSA's OIAPR inspectors that they would
like more training with objects similar to ones used in covert testing.22
Although TSA has made progress with the implementation of recurrent
training, some FSDs identified the need for several additional courses,
including courses that address more realistic threats. TSA acknowledged
that additional screener training is needed, and officials stated that the
agency is in the process of developing new and improved screener training,
including additional recurrent training modules (see app. III).
TSA Provides Leadership and Technical Training for Supervisors, but Some
FSDs Would Like More Training for Screening Supervisors and Lead Screeners
TSA has arranged for leadership training for screening supervisors through
the Department of Agriculture Graduate School and has developed leadership
and technical training courses for screening supervisors. However, some
FSDs reported the need for more training for Screening Supervisors and
Lead Screeners. The quality of Screening Supervisors has been a
long-standing concern. In testifying before the 9/11 Commission in May
2003, a former FAA Assistant Administrator for Civil Aviation Security
stated that following a series of covert tests at screening checkpoints to
determine which were strongest, which were weakest, and why, invariably
the checkpoint seemed to be as strong or as weak as the supervisor who was
running it. Similarly, TSA's OIAPR identified a lack of supervisory
training as a cause for screener covert testing failures. Further, in a
July 2003 internal study of screener performance, TSA identified poor
22The MBS II and weapons training kits were provided to airports to
address the identified training gap by allowing screeners to see and feel
the threat objects that they are looking for. These kits contain some of
the test objects used by TSA's OAIPR to conduct the covert testing. In
February 2004, TSA issued guidance to FSDs on use of these kits to conduct
local screener testing. These guidelines were updated in June 2004.
supervision at the screening checkpoints as a cause for screener
performance problems. In particular, TSA acknowledged that many Lead
Screeners, Screening Supervisors, and Screening Managers did not
demonstrate supervisory and management skills (i.e., mentoring, coaching,
and positive reinforcement) and provided little or no timely feedback to
guide and improve screener performance. In addition, the internal study
found that because of poor supervision at the checkpoint, supervisors or
peers were not correcting incorrect procedures, optimal performance
received little reinforcement, and not enough breaks were provided to
screeners. A September 2004 report by the DHS OIG supported these
findings, noting that Screening Supervisors and Screening Managers needed
to be more attentive in identifying and correcting improper or inadequate
screener performance.23
TSA recognizes the importance of Screener Supervisors and has established
training programs to enhance their performance and effectiveness. In
September 2003, we reported that TSA had begun working with the Department
of Agriculture Graduate School to tailor the school's off-the-shelf
supervisory course to meet the specific needs of Screening Supervisors,
and had started training the existing supervisors at that time through
this course until the customized course was fielded. According to TSA's
training records, as of September 2004, about 3,800 Screening Supervisors
had completed the course-approximately 92 percent of current Screening
Supervisors. In response to our survey, one FSD noted that the supervisory
training was long overdue because most of the supervisors had no prior
federal service or, in some cases, no leadership experience. This FSD also
noted that "leadership and supervisory skills should be continuously
honed; thus, the development of our supervisors should be an extended and
sequential program with numerous opportunities to develop skills-not just
a one-time class."
In addition to the Department of Agriculture Graduate School course, TSA's
Online Learning Center includes over 60 supervisory courses designed to
develop leadership and coaching skills. In April 2004, TSA included in the
Online Learning Center a Web-based technical training course-required for
all Lead Screeners and Screening Supervisors. This course covers technical
issues, such as resolving alarms at screening
23Department of Homeland Security Office of Inspector General, Audit of
Passenger and Baggage Screening Procedures at Domestic Airports, OIG-04-37
(Washington, D.C., September 2004).
checkpoints. TSA introduced this course to the field in March 2004, and
although the course is a requirement, TSA officials stated that they have
not set goals for when all Lead Screeners and Screening Supervisors should
have completed the course. In June 2004, TSA training officials stated
that a second supervisor technical course was planned for development and
introduction later in 2004. However, in December 2004, the training
officials stated that planned funding for supervisory training may be used
to support other TSA initiatives. The officials acknowledged that this
would reduce TSA's ability to provide the desired type and level of
supervisory training to its Lead Screener, Screening Supervisor, and
Screening Manager staff. TSA plans to revise its plans to provide Lead
Screener, Screening Supervisor, and Screening Manager training based on
funding availability.
Although TSA has developed leadership and technical courses for Screening
Supervisors, many FSDs, in response to our general survey, identified
additional types of training needed to enhance screener supervision. Table
2 provides a summary of the additional training needs that FSDs reported.
Table 2: Training Needed to Enhance Screener Supervision, as Reported by FSDs
Percentage of FSDs Training topic indicating this need
Leadership and management
Public speaking/communications
Conflict management
Human resource-related issues (hiring, termination, discipline)
Counseling, mentoring, and coaching
Writing skills
Additional supervisory training
Interpersonal skills
Crisis management and incident response 6
Source: GAO general survey of FSDs.
TSA training officials stated that the Online Learning Center provides
several courses that cover these topics. Such courses include
o Situation Leadership II;
o Communicating with Difficult People: Handling Difficult Co-Workers;
o Team Participation: Resolving Conflict in Teams;
o Employee Performance: Resolving Conflict;
o High Impact Hiring;
o Team Conflict: Overcoming Conflict with Communication;
o Correcting Performance Problems: Disciplining Employees;
o Team Conflict: Working in Diversified Teams;
o Correcting Performance Problems: Identifying Performance Problems;
o Resolving Interpersonal Skills;
o Grammar, Skills, Punctuation, Mechanics and Word Usage; and
o Crisis in Organizations: Managing Crisis Situations.
TSA training officials acknowledged that for various reasons FSDs might
not be aware that the supervisory and leadership training is available.
For example, FSDs at airports without high-speed Internet/intranet access
to the Online Learning Center might not have access to all of these
courses. It is also possible that certain FSDs have not fully browsed the
contents of the Online Learning Center and therefore are not aware that
the training is available.24 Furthermore, officials stated that online
learning is relatively new to government and senior field managers, and
some of the FSDs may expect traditional instructor-led classes rather than
online software.
Some FSDs Reported Impediments to Screeners Receiving Recurrent Training
Some FSDs responded to our general survey that they faced challenges with
screeners receiving recurrent training, including insufficient staffing to
allow all screeners to complete training within normal duty hours and a
lack of high-speed Internet/intranet connectivity at some training
facilities. According to our guide for assessing training, to foster an
environment conducive to effective training and development, agencies must
take actions to provide sufficient time, space, and equipment to employees
to complete required training.25 TSA has set a requirement for 3 hours of
recurrent training per week averaged over a quarter year, for both
full-time and part-time screeners. However, FSDs for about 18 percent (48
of 263) of the airports in our airport-specific survey reported that
screeners received less than 9 to 12 hours of recurrent training per
month.26 Additionally, FSDs for 48 percent (125 of 263) of the airports in
the survey
24TSA headquarters training officials stated that the Training
Coordinators at airports should be aware of the availability of the
supervisory and leadership training courses.
25GAO, Human Capital: A Guide for Assessing Strategic Training and
Development Efforts in the Federal Government, GAO-04-546G (Washington,
D.C., March 2004).
26Methods used by FSDs for determining and reporting recurrent training
time have not been verified by GAO.
reported that there was not sufficient time for screeners to receive
recurrent training within regular work hours.
At 66 percent of those airports where the FSD reported that there was not
sufficient time for screeners to receive recurrent training within regular
work hours, the FSDs cited screener staffing shortages as the primary
reason. We reported in February 2004 that FSDs at 11 of the 15 category X
airports we visited reported that they were below their authorized
staffing levels because of attrition and difficulties in hiring new staff.
In addition, three of these FSDs noted that they had never been successful
in hiring up to the authorized staffing levels. We also reported in
February 2004 that FSDs stated that because of staffing shortages, they
were unable to let screeners participate in training because it affected
the FSD's ability to provide adequate coverage at the checkpoints. In
response to our survey, FSDs across all categories of airports reported
that screeners must work overtime in order to participate in training. A
September 2004 DHS OIG report recommended that TSA examine the workforce
implications of the 3-hour training requirement and take steps to correct
identified imbalances in future workforce planning to ensure that all
screeners are able to meet the recurrent training standard. The
3-hours-per-week training standard represents a staff time commitment of
7.5 percent of fulltime and between 9 and 15 percent of part-time
screeners' nonovertime working hours. TSA headquarters officials have
stated that because the 3hours-per-week requirement is averaged over a
quarter, it provides flexibility to account for the operational
constraints that exist at airports. However, TSA headquarters officials
acknowledged that many airports are facing challenges in meeting the
3-hour recurrent training requirement. TSA data for the fourth quarter of
fiscal year 2004 reported that 75 percent of airports were averaging less
than 3 hours of recurrent training per week per screener. The current
screener staffing model, which is used to determine the screener staffing
allocations for each airport, does not take the 3-hours-per-week recurrent
training requirement into account.27 However, TSA headquarters officials
said that they are factoring this
27The staffing model took into account factors such as the number of
screening checkpoints and lanes at an airport; originating passengers;
projected air carrier service increases and decreases during calendar year
2003; and hours needed to accommodate some screener training, leave, and
breaks.
training requirement into their workforce planning efforts, including the
staffing model currently under development.28
Another barrier to providing recurrent training is the lack of high-speed
Internet/intranet access at some of TSA's training locations.29 TSA
officials acknowledged that many of the features of the Online Learning
Center, including some portions of the training modules and some Online
Learning Center course offerings, are difficult or impossible to use in
the absence of high-speed Internet/intranet connectivity. As one FSD put
it, "the delayed deployment of the high-speed Internet package limits the
connectivity to TSA HQ for various online programs that are mandated for
passenger screening operations including screener training." One FSD for a
category IV airport noted the lack of a high-speed connection for the one
computer at an airport he oversees made the Online Learning Center "nearly
useless."
TSA began deploying high-speed access to its training sites and
checkpoints in May 2003 and has identified high-speed connectivity as
necessary in order to deliver continuous training to screeners. TSA's July
2003 Performance Improvement Study recommended accelerating highspeed
Internet/intranet access in order to provide quick and systematic
distribution of information and, thus, reduce uncertainty caused by the
day-to-day changes in local and national procedures and policy. In October
2003, TSA reported plans to have an estimated 350 airports online with
high-speed connectivity within 6 months. However, in June 2004, TSA
reported that it did not have the resources to reach this goal.
28In May 2003, TSA hired a contractor to develop a staffing model for its
screening workforce. TSA officials told us that the contractor completed
the staffing model in June 2004, and all airports now have the capability
to use the contractor's stand-alone software. TSA completed installation
of the software on its intranet in March 2005. This installation provides
TSA headquarters with access to the staffing models used by airports. The
Intelligence Reform and Terrorism Prevention Act of 2004 requires TSA to
develop and submit to the appropriate congressional committees, standards
for determining aviation security staffing at commercial airports no later
than 90 days after December 17, 2004, the date of the act's enactment, and
GAO to conduct an analysis of these standards (Pub. L. No. 108-458). As of
April 15, 2005, these standards had not been submitted.
29High-speed Internet/intranet access is provided by a series of
technologies that give users the ability to send and receive data at
volumes and speeds far greater than Internet access over traditional
telephone lines. In addition to offering speed, the technology provides a
continuous, "always on" connection (no need to dial up) and a "two-way"
capability, that is, the ability to both receive and transmit data at high
speeds.
TSA records show that as of October 2004, TSA had provided high-speed
access for training purposes to just 109 airports, where 1,726 training
computers were fully connected.30 These 109 airports had an authorized
staffing level of over 24,900 screeners, meaning that nearly 20,100
screeners (45 percent of TSA's authorized screening workforce) still did
not have high-speed Internet/intranet access to the Online Learning Center
at their training facility. In October 2004, TSA officials stated that
TSA's Office of Information Technology had selected an additional 16
airport training facilities with a total of 205 training computers to
receive highspeed connectivity by the end of December 2004. As of January
19, 2005, TSA was unable to confirm that these facilities had received
high-speed connectivity. Additionally, they could not provide a time frame
for when they expected to provide high-speed connectivity to all airport
training facilities because of funding uncertainties. Furthermore, TSA
does not have a plan for prioritizing and scheduling the deployment of
high-speed connectivity to all airport training facilities once funding is
available. Without a plan, TSA's strategy and timeline for implementing
connectivity to airport training facilities is unclear. The absence of
such a plan limits TSA's ability to make prudent decisions about how to
move forward with deploying connectivity once funding is available. Figure
5 shows the percentage of airports reported to have high-speed
connectivity for their training computers by category of airport as of
October 2004.
30TSA defines fully connected as a training computer with the new network
image installed and connected to the TSA broadband network.
Figure 5: Percentage of Airports Reported to Have High-Speed Connectivity
for Training Purposes as of October 2004
Airport category: X I II III IV All airports
of airports: 21 61 51 124 197 454
Airports with high-speed connectivity
Airports without high-speed connectivity
Source: GAO analysis of TSA data.
Note: These data show airports with high-speed connectivity on training
computers. Some airports have high-speed connectivity, but for purposes
other than training.
To mitigate airport connectivity issues in the interim, on April 1, 2004,
TSA made the Online Learning Center courses accessible through public
Internet connections, which enable screeners to log on to the Online
Learning Center from home, a public library, or other locations. However,
TSA officials stated that the vast majority of screeners who have used the
Online Learning Center have logged in from airports with connectivity at
their training facilities. TSA also distributes new required training
products using multiple delivery channels, including written materials and
CD-ROMs for those locations where access to the Online Learning Center is
limited. Specifically, TSA officials stated that they provided airports
without high-speed connectivity with CD-ROMs for the 50 most commonly used
optional commercial courseware titles covering topics such as information
technology skills, customer service, and teamwork. Additionally, officials
stated that as technical courses are added to the Online Learning Center,
they are also distributed via CD-ROM and that until full connectivity is
achieved, TSA will continue to distribute new training products using
multiple delivery channels.
TSA Lacks Adequate Controls to Provide Reasonable Assurance That Screeners
Receive Required Training
Because of a lack of internal controls, TSA cannot provide reasonable
assurance that screeners are completing required training. First, TSA
policy does not clearly define responsibility for ensuring that screeners
have completed all required training. Additionally, TSA has no formally
defined policies or procedures for documenting completion of remedial
training, or a system designed to facilitate review of this documentation
for purposes of monitoring. Further, TSA headquarters does not have formal
policies and procedures for monitoring completion of basic training and
lacks procedures for monitoring recurrent training. Finally, at airports
without high-speed connectivity, training records must be entered
manually, making it challenging for some airports to keep accurate and
upto-date training records.
TSA Policy Does Not Clearly Define Responsibility for Ensuring That
Screeners Are in Compliance with Training Requirements
TSA's current guidance for FSDs regarding the training of the screener
workforce does not clearly identify responsibility for tracking and
ensuring compliance with training requirements. In a good control
environment, areas of authority and responsibility are clearly defined and
appropriate lines of reporting are established.31 In addition, internal
control standards also require that responsibilities be communicated
within an organization. The Online Learning Center provides TSA with a
standardized, centralized tool capable of maintaining all training records
in one system. It replaces an ad hoc system previously used during initial
rollout of federalized screeners in which contractors maintained training
records. A February 2004 management directive states that FSDs are
responsible for ensuring the completeness, accuracy, and timeliness of
training records maintained in the Online Learning Center for their
employees. For basic and recurrent training, information is to be entered
into the Online Learning Center within 30 days of completion of the
training activity. However, the directive does not clearly identify who is
responsible for ensuring that employees comply with training requirements.
Likewise, a December 2003 directive requiring that screeners complete 3
hours of training per week averaged over a quarter states that FSDs are
responsible for ensuring that training records for each screener are
maintained in the Online Learning Center. Although both directives include
language that requires FSDs to ensure training records are maintained in
the Online Learning Center, neither specifies whether FSDs or headquarters
officials are responsible for ensuring compliance with the basic,
recurrent, and remedial training requirements. Even so,
31GAO/AIMD-00-21.3.1.
TSA headquarters officials told us that FSDs are ultimately responsible
for ensuring screeners receive required training. However, officials
provided no documentation clearly defining this responsibility. Without a
clear designation of responsibility for monitoring training completion,
this function may not receive adequate attention, leaving TSA unable to
provide reasonable assurance that its screening workforce receives
required training. In April 2005, TSA officials responsible for training
stated that they were updating the February 2004 management directive on
training records to include a specific requirement for FSDs to ensure that
screeners complete required training. They expect to release the revised
directive in May 2005.
TSA Lacks Formal Policies and Procedures for Monitoring Completion of
Required Training
TSA has not established and documented policies and procedures for
monitoring completion of basic and recurrent training. Internal control
standards advise that internal controls should be designed so that
monitoring is ongoing and ingrained in agency operations.32 However, TSA
headquarters officials stated that they have no formal policy for
monitoring screeners' completion of basic training. They also stated that
they have neither informal nor formal procedures for monitoring the
completion of screeners' recurrent training requirements, and acknowledged
that TSA policy does not address what is to occur if a screener does not
meet the recurrent training requirement. Officials further stated that
individual FSDs have the discretion to determine what action, if any, to
take when screeners do not meet this requirement.
In July 2004, TSA training officials stated that headquarters staff
recently began running a report in the Online Learning Center to review
training records to ensure that newly hired screeners had completed
required basic training. In addition, they stated that in June 2004, they
began generating summary-level quarterly reports from the Online Learning
Center to quantify and analyze hours expended for recurrent screener
training. Specifically, TSA training officials stated that reports showing
airport-level compliance with the 3-hour recurrent requirement were
generated for the third and fourth quarters of fiscal year 2004 and
delivered to the Office of Aviation Operations for further analysis and
sharing with the field. However, Aviation Operations officials stated that
they did not use these reports to monitor the status of screener
compliance with the 3-hour recurrent training requirement and do not
provide them to the field unless
32GAO/AIMD-00-21.3.1.
requested by an FSD. TSA training officials said that while headquarters
intends to review recurrent training activity on an ongoing basis at a
national and airport level, they view FSDs and FSD training staff as
responsible for ensuring that individuals receive all required training.
Further, they acknowledged that weaknesses existed in the reporting
capability of the Online Learning Center and stated that they plan to
upgrade the Online Learning Center with improved reporting tools by the
end of April 2005. Without clearly defined policies and procedures for
monitoring the completion of training, TSA lacks a structure to support
continuous assurance that screeners are meeting training requirements.
TSA Lacks Clearly Defined Policies and Procedures for Documenting
Completion of Remedial Training
TSA has not established clear policies and procedures for documenting
completion of required remedial training. The Standards for Internal
Control state that agencies should document all transactions and other
significant events and should be able to make this documentation readily
available for examination.33 A TSA training bulletin dated October 15,
2002, specifies that when remedial training is required, FSDs must ensure
the training is provided and a remedial training reporting form is
completed and maintained with the screener's local records.34 However,
when we asked to review these records, we found confusion as to how and
where they were to be maintained. TSA officials stated that they are
waiting for a decision regarding how to maintain these records because of
their sensitive nature. In the meantime, where and by whom the records
should be maintained remains unclear.
In September 2004, officials from TSA's OIAPR-responsible for conducting
covert testing-stated that they maintain oversight to ensure screeners
requiring remedial training receive required training by providing a list
of screeners that failed covert testing and therefore need remedial
training to TSA's Office of Aviation Operations. Aviation Operations is
then to confirm via memo that each of the screeners has received the
necessary remedial training and report back to OIAPR. Accordingly, we
asked TSA for all Aviation Operations memos confirming
33GAO-AIMD-00-21.3.1.
34Consistent with ATSA, TSA requires remedial training for any passenger
or baggage screener who fails an operational (covert) test and prohibits
screeners from performing the screening function related to the test they
failed until they successfully complete the training.
completion of remedial training, but we were only able to obtain 1 of the
12 memos.35
In addition, during our review, we asked to review the remedial training
reporting forms at five airports to determine whether screeners received
required training, but we encountered confusion about requirements for
maintaining training records and inconsistency in record keeping on the
part of local TSA officials. Because of the unclear policies and
procedures for recording completion of remedial training, TSA does not
have adequate assurance that screeners are receiving legislatively
mandated remedial training.
Lack of High-Speed Connectivity Limits TSA's Ability to Document and Track
the Completion of Screener Training
Although training computers with high-speed Internet/intranet connectivity
automatically record completion of training in the Online Learning Center,
airports without high-speed access at their training facility must have
these records entered manually. The February 2004 management directive
that describes responsibility for entering training records into the
Online Learning Center also established that all TSA employees are
required to have an official TSA training record in the Online Learning
Center that includes information on all official training that is funded
wholly or in part with government funds. Without highspeed access, TSA
officials stated that it can be a challenge for airports to keep the
Online Learning Center up to date with the most recent training records.
TSA headquarters officials further stated that when they want to track
compliance with mandatory training such as ethics or civil rights
training, they provide the Training Coordinators with a spreadsheet on
which to enter the data rather than relying on the Online Learning Center.
As one FSD told us, without high-speed connectivity at several of the
airports he oversees, "this is very time consuming and labor intensive and
strains my limited resources." The difficulty that airports encounter in
maintaining accurate records when high-speed access is absent could
compromise TSA's ability to provide reasonable assurance that screeners
are receiving mandated basic and remedial training.
35According to TSA officials, between September 2002 and December 8, 2004,
the Office of Internal Affairs and Program Review issued 12 memorandums to
Aviation Operations that identified screeners requiring remedial training
based on covert testing conducted from September 2002 through September
30, 2004.
Progress Has Been Made in Implementing Tools to Measure and Enhance Screener
Performance, but Key Performance Targets Have Not Been Finalized
TSA has improved its efforts to measure and enhance screener performance.
However, these efforts have primarily focused on passenger screening
rather than checked baggage screening, and TSA has not yet finalized
performance targets for several key performance measures. For example, TSA
has increased the amount of covert testing it performs at airports. These
tests have identified that, overall, weaknesses and vulnerabilities
continue to exist in the passenger and checked baggage screening systems.
TSA also enabled FSDs to conduct local covert testing, fully deployed the
Threat Image Projection (TIP) system to passenger screening checkpoints at
commercial airports nationwide, and completed the 2003/2004 annual
screener recertification program for all eligible screeners. However, not
all of these performance measurement and enhancement tools are available
for checked baggage screening. Specifically, TIP is not currently
operational at checked baggage screening checkpoints, and the
recertification program does not include an image recognition component
for checked baggage screeners. However, TSA is taking steps to address the
overall imbalance in passenger and checked baggage screening performance
data, including working toward implementing TIP for checked baggage
screening and developing an image recognition module for checked baggage
screener recertification. To enhance screener and screening system
performance, TSA has also conducted a passenger screener performance
improvement study and subsequently developed an improvement plan
consisting of multiple action items, many of which TSA has completed.
However, TSA has not conducted a similar study for checked baggage
screeners. In addition, TSA has established over 20 performance measures
for the passenger and checked baggage screening systems as well as two
performance indexes (one for passenger and one for checked baggage
screening). However, TSA has not established performance targets for each
of the component indicators within the indexes, such as covert testing.
According to The Office of Management and Budget, performance goals are
target levels of performance expressed as a measurable objective, against
which actual achievement can be compared. Performance goals should
incorporate measures (indicators used to gauge performance); targets
(characteristics that tell how well a program must accomplish the
measure), and time frames. Without these targets, TSA's performance
management system, and these performance indexes, specifically, may not
provide the agency with the complete information necessary to assess
achievements and make decisions about where to direct performance
improvement efforts. Although TSA has not yet established performance
targets for each of the component indicators, TSA plans to finalize
performance targets for the indicators by the end of fiscal year 2005.
TSA Has Increased Its Covert Testing and Allowed Local Covert Testing at
Passenger Screening Checkpoints
TSA headquarters has increased the amount of covert testing it performs
and enabled FSDs to conduct additional local covert testing at passenger
screening checkpoints. TSA's OIAPR conducts unannounced covert tests of
screeners to assess their ability to detect threat objects and to adhere
to TSA-approved procedures. These tests, in which undercover OIAPR
inspectors attempt to pass threat objects through passenger screening
checkpoints and in checked baggage, are designed to measure
vulnerabilities in passenger and checked baggage screening systems and to
identify systematic problems affecting performance of screeners in the
areas of training, policy, and technology.36 TSA considers its covert
testing as a "snapshot" of a screener's ability to detect threat objects
at a particular point in time and as one of several indicators of system
wide screener performance.
OIAPR conducts tests at passenger screening checkpoints and checked
baggage screening checkpoints. According to OIAPR, these tests are
designed to approximate techniques terrorists might use. These covert test
results are one source of data on screener performance in detecting threat
objects as well as an important mechanism for identifying areas in
passenger and checked baggage screening needing improvement. In testimony
before the 9/11 Commission, the Department of Transportation Inspector
General stated that emphasis must be placed on implementing an aggressive
covert testing program to evaluate operational effectiveness of security
systems and equipment.37
Between September 10, 2002, and September 30, 2004, OIAPR conducted a
total of 3,238 covert tests at 279 different airports. In September 2003,
we reported that OIAPR had conducted limited covert testing but planned to
double the amount of tests it conducted during fiscal year 2004, based on
an anticipated increase in its staff from about 100 full-time equivalents
to about 200 full-time equivalents. TSA officials stated that based on
budget constraints, OIAPR's fiscal year 2004 staffing authorization was
limited to
36OIAPR designs its covert testing methods based, in part, on intelligence
regarding the most recent threats.
37Statement before the National Commission on Terrorist Attacks upon the
United States, Statement of the Honorable Kenneth M. Mead, Inspector
General, U.S. Department of Transportation, May 22, 2003.
183 full-time-equivalents,38 of which about 60 are located in the field.39
Despite a smaller than expected staff increase, by the end of the second
quarter of fiscal year 2004, OIAPR had already surpassed the number of
tests it performed during fiscal year 2003, as shown in table 3.
38Covert testing is an ancillary duty and not a full-time assignment for
the majority of OIAPR staff. According to OIAPR, of the approximately 123
full-time-equivalent positions in headquarters, 14 are dedicated fully to
the covert testing program, which includes covert testing of all modes of
transportation, not just airports. These 14 full-time-equivalents are in
OIAPR's Special Operations group and form the core of team leaders for the
covert testing trips. In addition, two full-time-equivalents from OIAPR's
Office of Program Analysis support the covert testing program full-time in
terms of data analysis, report writing, and quality assurance. The
remaining OIAPR staff in headquarters and the field are responsible for
conducting criminal and noncriminal investigations of employee misconduct;
conducting program reviews, inspections, and special inquiries into
security incidents; and managing OIAPR.
39TSA established five mission support centers staffed with OIAPR,
training, and other personnel. These centers are located in Atlanta,
Dallas, Detroit, Philadelphia, and San Francisco. In fiscal year 2004,
OIAPR began to use field staff to support its covert testing activities.
In addition, OIAPR has 12 investigators located at seven airports.
Table 3: Checkpoint and Checked Baggage Tests Conducted by OIAPR,
September 10, 2002-September 30, 2004
Testing period Passenger
fiscal screening Checked
year/quarter Airportsa checkpoint tests baggage testsb Total
tests
2002 fourth 2 30 3
2003 first 14 120 1 121
2003 second 31 231 27 258
2003 third 28 198 19 217
2003 fourth 25 217 23 240
2004 first 41 171 110 281
2004 second 111 770 182 952
2004 third 56 379 102 481
2004 fourth 64 527 128 655
Total 2,643 595 3,238
Source: GAO analysis of OIAPR data.
Note: Some airports have been tested more than once.
aOIAPR conducted covert testing at three additional airports in September
and October 2002. However, at the time of the testing, federal screeners
had not yet been deployed to these airports. We excluded these tests from
our analysis.
bTSA did not begin reporting the results of checked baggage tests until
January 2003. However, four of these tests were conducted in September and
October 2002.
In October 2003, OIAPR committed to testing between 90 and 150 airports by
April 2004 as part of TSA's short-term screening performance improvement
plan. OAIPR officials stated that this was a onetime goal to increase
testing. This initiative accounts for the spike in testing for the second
quarter of fiscal year 2004.
OIAPR has created a testing schedule designed to test all airports at
least once during a 3-year time frame. Specifically, the schedule calls
for OIAPR to test all category X airports once a year, category I and II
airports once every 2 years, and category III and IV airports at least
once every 3 years.
In September 2003 and April 2004, we reported that TSA covert testing
results had identified weaknesses in screeners' ability to detect threat
objects.40 More recently, in April 2005, we, along with the DHS OIG,
identified that screener performance continued to be a concern.
Specifically, our analysis of TSA's covert testing results for tests
conducted between September 2002 and September 2004 identified that
overall, weaknesses still existed in the ability of screeners to detect
threat objects on passengers, in their carry-on bags, and in checked
baggage. 41 Covert testing results in this analysis cannot be generalized
either to the airports where the tests were conducted or to airports
nationwide.42 These weaknesses and vulnerabilities were identified at
airports of all sizes, at airports with federal screeners, and airports
with private-sector screeners. For the two-year period reviewed, overall
failure rates for covert tests (passenger and checked baggage) conducted
at airports using privatesector screeners were somewhat lower than failure
rates for the same tests conducted at airports using federal screeners for
the airports tested during this period.43 Since these test results cannot
be generalized as discussed above, each airport's test results should not
be considered a comprehensive measurement of the airport's performance or
any individual screener's performance in detecting threat objects, or in
determining whether airports with private sector screeners performed
better than airports with federal screeners.
On the basis of testing data through September 30, 2004, we determined
that OIAPR had performed covert testing at 61 percent of the nation's
commercial airports. TSA has until September 30, 2005, to test the
additional 39 percent of airports and meet its goal of testing all
airports
40GAO, Airport Passenger Screening: Preliminary Observations on Progress
Made and Challenges Remaining, GAO-03-1173 (Washington, D.C.: Sept. 24,
2003). GAO, Aviation Security: Private Screening Contractors Have Little
Flexibility to Implement Innovative Approaches, GAO-04-544T (Washington,
D.C.: Apr 22, 2004).
41Results of TSA's covert testing of passenger and checked baggage
screening are classified and are discussed in a separate classified
report, GAO, Results of Transportation Security Administration's Covert
Testing for Passenger and Checked Baggage Screening for September 2002
through September 2004, GAO-05-437C (Washington, D.C.: Apr. 5, 2005).
42Test results cannot be generalized because sample tests were not
identified using the principles of probability sampling. In a probability
sample to assess screener detection of threat objects, each screening of a
passenger or baggage would have to have a chance of being selected. A
well-designed probability sample would enable failure rates to be
generalized to all airports. However, for cost and operational reasons,
probability sampling may not be feasible for passenger and checked baggage
screening because it would require a very large sample size and an
exhaustive examination of each sampled passenger or baggage to determine
if there was a threat object to detect.
43Test failure rates are classified and have been excluded from this
report.
within 3 years. Although officials stated that they have had to divert
resources from airport testing to conduct testing of other modes and that
testing for other modes of transportation may affect their ability to
conduct airport testing, they still expect to meet the goal.
In February 2004, TSA provided protocols to help FSDs conduct their own
covert testing of local airport passenger screening activities-a practice
that TSA had previously prohibited.44 Results of local testing using these
protocols are to be entered into the Online Learning Center. This
information, in conjunction with OAIPR covert test results and TIP threat
detection results, is intended to assist TSA in identifying specific
training and performance improvement efforts. In February 2005, TSA
released a general procedures document for local covert testing at checked
baggage screening locations.
TSA officials said that they had not yet begun to use data from local
covert testing to identify training and performance needs because of
difficulties in ensuring that local covert testing is implemented
consistently nationwide. These officials said that after a few months of
collecting and assessing the data, they will have a better idea of how the
data can be used.
TSA Has Reinstated the Threat Image Projection System and Plans to Use Its
Data to Improve the Screening System
TSA has nearly completed the reactivation of the TIP system at airports
nationwide and plans to use data it is collecting to improve the
effectiveness of the passenger screening system.45 TIP is designed to test
passenger screeners' detection capabilities by projecting threat images,
including guns, knives, and explosives, onto bags as they are screened
during actual operations. Screeners are responsible for identifying the
threat image and calling for the bag to be searched. Once prompted, TIP
identifies to the screener whether the threat is real and then records the
screener's performance in a database that could be analyzed for
44The local covert testing protocols were updated in June 2004 and August
2004 to provide information on alternative testing methods.
45TIP is not yet operational at one airport because of construction at the
screening checkpoint to prepare for its installation. However, the
TIP-ready X-ray machines have already been procured for the airport and
are expected to be installed once the construction issues have been
resolved.
performance trends.46 TSA is evaluating the possibility of developing an
adaptive functionality to TIP. Specifically, as individual screeners
become proficient in identifying certain threat images, such as guns or
knives, they will receive fewer of those images and more images that they
are less proficient at detecting, such as improvised explosive devices.
TIP was activated by FAA in 1999 with about 200 threat images, but it was
shut down immediately following the September 11 terrorist attacks because
of concerns that it would result in screening delays and panic, as
screeners might think that they were actually viewing threat objects. In
October 2003, TSA began reactivating and expanding TIP. In April 2004, we
reported that TSA was reactivating TIP with an expanded library of 2,400
images at all but one of the more than 1,800 checkpoint lanes nationwide.
To further enhance screener training and performance, TSA also plans to
develop at least an additional 50 images each month.
Despite these improvements, TIP is not yet available for checked baggage
screening. In April 2004, we reported that TSA officials stated that they
were working to resolve technical challenges associated with using TIP for
checked baggage screening on EDS machines and have started EDS TIP image
development. The DHS OIG reported in September 2004 that TSA plans to
implement TIP on all EDS machines at checked baggage stations nationwide
in fiscal year 2005. However, in December 2004, TSA officials stated that
because of severe budget reductions, TSA will be unable to begin
implementing a TIP program for checked baggage in fiscal year 2005. They
did not specify when such a program might begin.
TSA plans to use TIP data to improve the passenger screening system in two
ways. First, TIP data can be used to measure screener threat detection
effectiveness by different threats. Second, TSA plans to use TIP results
to help identify specific recurrent training needs within and across
airports and to tailor screeners' recurrent training to focus on threat
category areas that indicate a need for improvement. TSA considers
February 2004 as the first full month of TIP reporting with the new
library of 2,400 images. TSA began collecting these data in early March
2004 and is using the data to determine more precisely how they can be
used to measure screener
46The TIP database records both the TIP hit rate and TIP false alarm rate.
These two results are used to determine the probability of detection and
probability of false alarm, which determine overall TIP performance.
performance in detecting threat objects and to determine what the data
identify about screener performance.
TSA does not currently plan to use TIP data as an indicator of individual
screener performance because TSA does not believe that TIP by itself
adequately reflects a screener's performance. Nevertheless, in April 2004,
TSA gave FSDs the capability to query and analyze TIP data in a number of
ways, including by screener, checkpoint, and airport. FSDs for over 60
percent of the airports included in our airport-specific survey stated
that they use or plan to use TIP data as a source of information in their
evaluations of individual screener performance. Additionally, FSDs for 50
percent of the airports covered in our survey reported using data
generated by TIP to identify specific training needs for individual
screeners.
In September 2004, the DHS OIG reported that TSA is assessing the cost and
feasibility of modifying TIP so that it recognizes and responds to
specific threat objects with which individual screeners are most and least
competent in detecting, over time. This feature would increase the utility
of TIP as a training tool. The DHS OIG also reported that TSA is
considering linking TIP over a network, which would facilitate TSA's
collection, analysis, and information-sharing efforts around TIP user
results. The report recommended that TSA continue to pursue each of these
initiatives, and TSA agreed. However, in December 2004, TSA officials
stated that the availability of funding will determine whether or not they
pursue these efforts further.
TSA Has Completed Its First Round of Screener Recertification Testing, but
Testing for Checked Baggage Screeners Is Not as Comprehensive as for
Passenger Screeners
TSA has completed its first round of the screener recertification program,
and the second round is now under way. However, TSA does not currently
include an image recognition component in the test for checked baggage
screener recertification. ATSA requires that each screener receive an
annual proficiency review to ensure he or she continues to meet all
qualifications and standards required to perform the screening function.
In September 2003, we reported that TSA had not yet implemented this
requirement.47 To meet this requirement, TSA established a recertification
program, and it began recertification testing in October 2003 and
47GAO-03-1173.
completed the testing in March 2004. 48 The first recertification program
was composed of two assessment components, one of screeners' performance
and the other of screeners' knowledge and skills. During the performance
assessment component of the recertification program, screeners are rated
on both organizational and individual goals, such as maintaining the
nation's air security, vigilantly carrying out duties with utmost
attention to tasks that will prevent security threats, and demonstrating
the highest levels of courtesy to travelers to maximize their levels of
satisfaction with screening services. The knowledge and skills assessment
component consists of three modules: (1) knowledge of standard operating
procedures, (2) image recognition, and (3) practical demonstration of
skills. Table 4 provides a summary of these three modules.
Table 4: Modules Included in Recertification Knowledge and Skills
Assessment
Testing modules Description
Knowledge of standard operating Computerized 50-question multiple-choice
test. It procedures is either passenger- or baggage-specific.
Image recognition Computerized test that consists of 100 images and is
used to evaluate a screener's skill and ability in detecting threat or
prohibited objects within X-ray images.
Practical demonstration of skills Hands-on simulated work sample to
evaluate a screener's knowledge, skills, and ability when performing
specific screener tasks along with ability to provide customer service.
Source: TSA.
To be recertified, screeners must have a rating of "met" or "exceeded"
standards on their annual performance assessments and have passed each of
the applicable knowledge and skills modules. Screeners that failed any of
the three modules were to receive study time or remedial training as well
as a second opportunity to take and pass the modules. Screeners who failed
on their second attempt were to be removed from screening duties and
subject to termination. Screeners could also be terminated for receiving a
rating of below "met" standards.
48Some screeners, such as those on extended leave, leave without pay,
military leave, or leave because of an on-the-job injury were not tested.
These screeners are retested as they return to work.
TSA completed its analysis of the recertification testing and performance
evaluations in May 2004. TSA's analysis shows that less than 1 percent of
screeners subject to recertification failed to complete this requirement.
Figure 6 shows the recertification results.
Figure 6: Screener Recertification Results, October 2003-March 2004
42,690 Screeners recertified 352 Screeners not recertified
Screeners recertified Screeners not recertified
Source: GAO analysis of TSA data.
Across all airports screeners performed well on the recertification
testing. Over 97 percent of screeners passed the standard operating
procedures test on their first attempt. Screeners faced the most
difficulty on the practical demonstration of skills component. However,
following remediation, 98.6 percent of the screeners who initially failed
this component passed on their second attempt. Table 5 shows the results
of the recertification testing by module.
Table 5: Screener Recertification Module Testing Percentage Pass Rates,
October 2003-March 2004
Standard operating Practical skills
procedures Image recognition demonstration
First attempt 97.4 96.0 75.2
Retest 96.8 84.3 98.6
Overall 99.9 99.5 99.7
Source: TSA.
As shown in table 6, screeners hired as checked baggage screeners were not
required to complete the image recognition module in the first round of
the recertification testing.49
Table 6: Recertification Testing Modules by Screening Function
Checked
Testing modules Passenger baggage
Knowledge of standard operating procedures SQRT SQRT
Image recognition SQRT
Practical demonstration of skills SQRT SQRT
Source: TSA.
In addition, during the first year of recertification testing, which took
place from October 2003 through May 2004, dual-function screeners who were
actively working as both passenger and checked baggage screeners were
required to take only the recertification test for passenger screeners.
They were therefore not required to take the recertification testing
modules required for checked baggage, even though they worked in that
capacity.50
TSA began implementing the second annual recertification testing in
October 2004 and plans to complete it no later than June 2005. This
recertification program includes components for dual-function screeners.
However, TSA still has not included an image recognition module for
checked baggage screeners-which would include dual-function screeners
performing checked baggage screening. TSA officials stated that a decision
was made to not include an image recognition module for checked baggage
screeners during this cycle because not all checked baggage screeners
would have completed training on the onscreen resolution protocol by the
time recertification testing was conducted at their airports.51 In
December 2004, TSA officials stated that they plan on
49According to TSA records, between October 2003 and May 2004, 13,516
screeners completed the recertification testing as baggage screeners.
50As of January 7, 2005, TSA reported that its workforce included
approximately 25,947 dual-trained screeners who were certified to serve as
passenger or baggage screeners.
51TSA's onscreen resolution protocol requires that when an EDS machine
alarm goes off, indicating the possibility of explosives, TSA screeners,
by reviewing computer-generated images of the inside of the bag, attempt
to determine whether or not a suspect item or items are in fact explosive
materials. If the screener is unable to make this determination, the bag
is diverted from the main conveyor belt into an area where it receives a
secondary screening by a screener with an ETD machine.
developing an image recognition module for checked baggage and
dualfunction screeners, and that this test should be available for next
year's recertification program. The development and implementation of the
image recognition test will be contingent, they stated, upon the
availability of funds.
TSA Has Identified and Implemented Efforts to Enhance Screener
Performance, but These Efforts Primarily Focused on Passenger Screeners
TSA has implemented a number of improvements designed to enhance screener
performance, based on concerns it identified in a July 2003 Passenger
Screener Performance Improvement Study and recommendations from OIAPR. To
date, however, these efforts have primarily focused on the performance of
passenger screeners, and TSA has not yet undertaken a comparable
performance study for checked baggage screeners. The Passenger Screener
Performance Improvement Study relied in part on the findings of OIAPR's
covert testing. At the time the study was issued, OIAPR had conducted
fewer than 50 tests of checked baggage screeners. The July 2003 study
focused on and included numerous recommendations for improving the
performance of passenger screeners, but recommended waiting to analyze the
performance of checked baggage screeners until some time after
implementation of the recommendations, some of which TSA indicated, also
applied to checked baggage screeners. TSA officials told us that this
analysis has been postponed until they have reviewed the impact of
implementing the recommendations on passenger screening performance.
In October 2003, to address passenger screener performance deficiencies
identified in the Screener Performance Improvement Study, TSA developed a
Short-Term Screening Performance Improvement Plan. This plan included
specific action items in nine broad categories-such as enhance training,
increase covert testing, finish installing TIP, and expedite high-speed
connectivity to checkpoints and training computers- that TSA planned to
pursue to provide tangible improvements in passenger screener performance
and security (see app. IV for additional information on the action items).
In June 2004, TSA reported that it had completed 57 of the 62 specific
actions. As of December 2004, two of these actions still had not been
implemented-full deployment of high-speed connectivity
and a time and attendance package-both of which continue to be deferred
pending the identification of appropriate resources.52
In addition to the Performance Improvement Study and corresponding action
plans, TSA's OIAPR makes recommendations in its reports on covert testing
results. These recommendations address deficiencies identified during
testing and are intended to improve screening effectiveness. As of
December 2004, OIAPR had issued 18 reports to TSA management on the
results of its checkpoint and checked baggage covert testing.53 These
reports include 14 distinct recommendations,54 some of which were included
in TSA's screener improvement action plan. All but two of these reports
included recommendations on corrective actions needed to enhance the
effectiveness of passenger and checked baggage screening.
TSA Has Established Screening Performance Measures and Indexes but Has Not
Established Key Performance Targets
TSA has established performance measures, indexes, and targets for the
passenger and checked baggage screening systems, but has not established
targets for the various components of the screening indexes. The
Government Performance and Results Act of 1993 provides, among other
things, that federal agencies establish program performance measures,
including the assessment of relevant outputs and outcomes of each program
activity.55 Performance measures are meant to cover key aspects of
performance and help decision makers to assess program accomplishments and
improve program performance. A performance target is a desired level of
performance expressed as a tangible,
52TSA officials stated that in early fiscal year 2005, TSA's Office of
Information Technology committed to extending high-speed connectivity to
an additional 16 locations, but is still awaiting funding to further
expand network coverage and to extend the network to checkpoints.
53OIAPR has issued its reports to the TSA Administrator; the TSA
Administrator's Chief of Staff; Associate Undersecretary for Aviation
Operations; Associate Undersecretary for Workforce, Performance and
Training; Assistant Secretary and Chief Technology Officer; Assistant
Administrator and Chief Support Systems Officer; Chief Operating Officer,
the Office of Transportation Security and Intelligence, and the Office of
Security Policy. The report recommendations are directed to the office(s)
responsible for taking the corrective action.
54Some recommendations appear repeatedly in multiple reports issued by
OIAPR.
55According to the Government Performance and Results Act, the Office of
Management and Budget, and GAO, outcomes assess actual results as compared
with the intended results or consequences that occur from carrying out a
program or activity. Outputs count the goods and services produced by a
program or organization.
measurable objective, against which actual achievement will be compared.
By analyzing the gap between target and actual levels of performance,
management can target those processes that are most in need of
improvement, set improvement goals, and identify appropriate process
improvements or other actions.
An April 2004 consultant study commissioned by TSA found that FSDs and FSD
staffs generally believed the lack of key performance indicators available
to monitor passenger and checked baggage screening performance represented
a significant organizational weakness. Since then, TSA has established
over 20 performance measures for the passenger and checked baggage
screening systems. For example, TSA measures the percentage of screeners
meeting a threshold score on the annual recertification testing on their
first attempt, the percentage of screeners scoring above the national
standard level on TIP performance, and the number of passengers screened,
by airport category.
TSA also has developed two performance indexes to measure the
effectiveness of the passenger and checked baggage screening systems.56
These indexes measure overall performance through a composite of
indicators and are derived by combining specific performance measures
relating to passenger and checked baggage screening, respectively.
Specifically, these indexes measure the effectiveness of the screening
systems through machine probability of detection and covert testing
results;57 efficiency through a calculation of dollars spent per passenger
or bag screened; and customer satisfaction through a national poll,
customer surveys, and customer complaints at both airports and TSA's
national call center. According to TSA officials, the agency has finalized
targets for the two overall indexes, but these targets have not yet been
communicated throughout the agency. Further, TSA plans to provide the FSDs
with only the performance index score, not the value of each of the
components, because the probabilities of detection are classified as
secret and TSA is concerned that by releasing components, those
probabilities could be
56TSA analyzed data from fiscal years 2003 and 2004 to establish baselines
and establish performance targets.
57According to TSA, the machine probabilities of detection are established
by the certification standards for each particular model of machines, and
machines are not deployed unless they have met those standards.
deduced.58 Table 7 summarizes the components of the performance indexes
developed by TSA.
Table 7: Components of TSA's Performance Indexes
FY 2005-2010
performance
Weight target
Performance index Components (percent) (1-5 scale)
Passenger screening o Machine probability of
program detection x person
probability of detection
(covert testing) 50
o Cost per person screened 25
o Consumer satisfaction 25
Checked baggage o Machine probability of
screening program detection x person
probability of detection
(covert testing) 50
o Cost per bag screened 25
o Consumer satisfaction 25
Source: TSA.
TSA has not yet established performance targets for the various components
of the screening indexes, including performance targets for covert testing
(person probability of detection). TSA's strategic plan states that the
agency will use the performance data it collects to make tactical
decisions based on performance. The screening performance indexes
developed by TSA can be a useful analysis tool, but without targets for
each component of the index, TSA will have difficulty performing
meaningful analyses of the parts that add up to the index. For example,
without performance targets for covert testing, TSA will not have
identified a desired level of performance related to screener detection of
threat objects. Performance targets for covert testing would enable TSA to
focus its improvement efforts on areas determined to be most critical, as
100 percent detection capability may not be attainable. In January 2005,
TSA officials stated that the agency plans to track the performance of
individual index components and establish performance targets against
58TSA headquarters officials stated that their intent is to provide FSDs
with various mechanisms to assess their screening effectiveness at the
airports for which they are responsible. Specifically, they stated that
FSDs are provided with TIP data and the results of OIAPR's covert testing
at their airports. Additionally, they have access to the results of local
covert testing at their airports.
Conclusions
which to measure these components. They further stated that they are
currently collecting and analyzing baseline data to establish these
targets and plan to finalize them by the end of fiscal year 2005.
It has been over 2 years since TSA assumed responsibility for passenger
and checked baggage screening operations at the nation's commercial
airports. TSA has made significant accomplishments over this period in
meeting congressional mandates related to establishing these screening
operations. With the congressional mandates now largely met, TSA has
turned its attention to assessing and enhancing the effectiveness of its
passenger and checked baggage screening systems. An important tool in
enhancing screener performance is ongoing training. As threats and
technology change, the training and development of screeners to ensure
they have the competencies-knowledge, skills, abilities, and behaviors-
needed to successfully perform their screening functions become vital to
strengthening aviation security. Without addressing the challenges to
delivering ongoing training, including installing high-speed connectivity
at airport training facilities, TSA may have difficulty maintaining a
screening workforce that possesses the critical skills needed to perform
at a desired level. In addition, without adequate internal controls
designed to help ensure screeners receive required training that are also
communicated throughout the agency, TSA cannot effectively provide
reasonable assurances that screeners receive all required training. Given
the importance of the Online Learning Center in both delivering training
and serving as the means by which the completion of screener training is
documented, TSA would benefit from having a clearly defined plan for
prioritizing the deployment of high-speed Internet/intranet connectivity
to all airport training facilities. Such a plan would help enable TSA to
move forward quickly and effectively in deploying high-speed connectivity
once funding is available.
Additionally, history demonstrates that U.S. commercial aircraft have long
been a target for terrorist attacks through the use of explosives carried
in checked baggage, and covert testing conducted by TSA and DHS OIG have
identified that weaknesses and vulnerabilities continue to exist in the
passenger and checked baggage screening systems, including the ability of
screeners to detect threat objects. While covert test results provide an
indicator of screening performance, they cannot solely be used as a
comprehensive measure of any airport's screening performance or any
individual screener's performance, or in determining the overall
performance of federal versus private-sector screening. Rather, these data
should be considered in the larger context of additional performance data,
such as TIP and recertification test results, when assessing screener
performance. While TSA has undertaken efforts to measures and strengthen
performance, these efforts have primarily focused on passenger screening
and not on checked baggage screening. TSA's plans for implementing TIP for
checked baggage screening, and establishing an image recognition component
for the checked baggage screeners recertification testing-plans made
during the course of our review- represent significant steps forward in
its efforts to strengthen checked baggage screening functions.
Additionally, although TSA has developed passenger and checked baggage
screening effectiveness measures, the agency has not yet established
performance targets for the individual components of these measures. Until
such targets are established, it will be difficult for TSA to draw more
meaningful conclusions about its performance and how to most effectively
direct its improvement efforts. For example, performance targets for
covert testing would enable TSA to focus its improvement efforts on areas
determined to be most critical, as 100 percent detection capability may
not be attainable. We are encouraged by TSA's recent plan to establish
targets for the individual components of the performance indexes. This
effort, along with the additional performance data TSA plans to collect on
checked baggage screening operations, should assist TSA in measuring and
enhancing screening performance and provide TSA with more complete
information with which to prioritize and focus its screening improvement
efforts.
Recommendations for To help ensure that all screeners have timely and
complete access to screener training available in the Online Learning
Center and to help
Executive Action provide TSA management with reasonable assurance that
all screeners are receiving required passenger and checked baggage
screener training, we recommend that the Secretary of the Department of
Homeland Security direct the Assistant Secretary, Transportation Security
Administration, to take the following two actions:
o develop a plan that prioritizes and schedules the deployment of
high-speed Internet/intranet connectivity to all TSA's airport training
facilities to help facilitate the delivery of screener training and the
documentation of training completion, and
o develop internal controls, such as specific directives, clearly
defining responsibilities for monitoring and documenting the completion of
required training, and clearly communicate these responsibilities
throughout the agency.
Agency Comments
and Our Evaluation
We provided a draft of this report to DHS for review and comment. On
February 4, 2005, we received written comments on the draft report, which
are reproduced in full in appendix V. DHS generally concurred with the
findings and recommendations in the report, and agreed that efforts to
implement our recommendations are critical to successful passenger and
checked baggage screening training and performance. With regard to our
recommendation that TSA develop a plan that prioritizes and schedules the
deployment of high-speed Internet/intranet connectivity to all TSA's
airport training facilities, DHS stated that TSA has developed such a
plan. However, although we requested a copy of the plan several times
during our review and after receiving written comments from DHS, TSA did
not provide us with a copy of the plan. Therefore, we cannot assess the
extent to which the plan DHS referenced in its written comments fulfills
our recommendation. In addition, regarding our recommendation that TSA
develop internal controls clearly defining responsibilities for monitoring
and documenting the completion of required training, and clearly
communicate those responsibilities throughout TSA, DHS stated that it is
taking steps to define responsibility for monitoring the completion of
required training and to insert this accountability into the performance
plans of all TSA supervisors. TSA's successful completion of these ongoing
and planned activities should address the concerns we raised in this
report. DHS has also provided technical comments on our draft report,
which we incorporated where appropriate.
As agreed with your office, we will send copies of this report to relevant
congressional committees and subcommittees and to the Secretary of the
Department of Homeland Security. We will also make copies available to
others upon request. In addition, the report will be made available at no
charge on GAO's Web site at http://www.gao.gov.
If you have any questions about this report or wish to discuss it further,
please contact me at (202) 512-8777. Key contributors to this report are
listed in appendix VI.
Sincerely yours,
Cathleen A. Berrick Director, Homeland Security and Justice Issues
Appendix I: Summary of Previous Findings Related to Screener Training and
Performance
Title and GAO product Product date Summary of previous findings related to
screener training and performance number
September 2003 The Transportation Security Administration (TSA) had
deployed a basic screener Airport Passenger training program and required
remedial training but had not fully developed or Screening: Preliminary
deployed a recurrent training program for screeners or supervisors.
Observations Made and
TSA had collected little information to measure screener performance in
detecting Challenges Remaining
threat objects. GAO-03-1173
o TSA's Office of Internal Affairs and Program Review's (OIAPR) covert
testing was the primary source of information collected on screeners'
ability to detect threat objects. However, TSA did not consider the covert
testing a measure of screener performance.
o TSA was not using the Threat Image Projection system (TIP) but planned
to fully activate the system with significantly more threat images than
previously used in October 2003.
o TSA had not yet implemented an annual proficiency review to ensure
that screeners met all qualifications and standards required to perform
their assigned screening functions.
November 2003 Although little data existed on the effectiveness of
passenger screening, TSA was Aviation Security: Efforts to implementing
several efforts to collect performance data. Measure Effectiveness and
o TSA's OIAPR had conducted little covert testing of the screener
workforce but Address Challenges planned to double the number of tests it
conducted during fiscal year 2004. GAO-04-232T
o TSA only recently began activating TIP on a wide-scale basis and
expected it to be fully operational at every checkpoint at all airports by
April 2004.
o TSA only recently began implementing the annual recertification
program and did not expect to complete testing at all airports until March
2004.
o TSA was developing performance indexes for individual screeners and
the screening system as a whole but had not fully established these
indexes. TSA expected to have them in place by the end of fiscal year
2004.
November 2003 Although TSA had not fully developed or deployed recurrent
or supervisory Aviation Security: Efforts to training programs, it was in
the process of Measure Effectiveness and
o deploying six recurrent training modules and was pilot-testing an
online Strengthen Security learning management system, and Programs
o working with the U.S. Department of Agriculture's Graduate School to
tailor its GAO-04-285T off-the-shelf supervisory course to meet the
specific training needs of screening supervisors.
February 2004 While TSA had taken steps to enhance its screener training
programs, staffing imbalances, and lack of high-speed connectivity at
airport training facilities had made it difficult for screeners at some
airports to fully utilize these programs.
Although TSA was making progress in measuring the performance of passenger
screeners, it had collected limited performance data related to its
checked baggage screening operations. However, TSA had begun collecting
additional performance data related to its checked baggage screening
operations and planned to increase these efforts in the future.
As part of its efforts to develop performance indexes, TSA was developing
baseline data for fiscal year 2004 and planned to report the indexes to
DHS in fiscal year 2005.
Aviation Security: Challenges Exist in Stabilizing and Enhancing Passenger
and Baggage Screening Operations
GAO-04-440T
Appendix I: Summary of Previous Findings Related to Screener Training and
Performance
Title and GAO product number
Product date Summary of previous findings related to screener training and
performance
April 2004 With the exception of covert testing and recent TIP data, data
were not yet
Aviation Security: Private
available to assess how well screeners were performing and what steps if
any TSA needed to take to improve performance. Also, TSA was not using TIP
as a formal indicator of screening performance, but instead was using it
to identify individual screener training needs.
Screening Contractors Have Little Flexibility to Implement Innovative
Approaches
GAO-04-505T
Source: GAO.
Appendix II: Objectives, Scope, and Methodology
To examine efforts by the Transportation Security Administration to
enhance their passenger and checked baggage screening programs, we
addressed the following questions: (1) What actions has TSA taken to
enhance training for screeners and supervisors? (2) How does TSA monitor
compliance with screener training requirements? (3) What is the status of
TSA's efforts to assess and enhance screener performance in detecting
threat objects?
To determine how TSA has enhanced training for screeners and supervisors
and how TSA has monitored compliance with screener training requirements,
we obtained and analyzed relevant legislation, as well as TSA's training
plans, guidance, and curriculum. We reviewed data from TSA's Online
Learning Center and assessed the reliability of the Online Learning Center
database. We compared TSA's procedures for ensuring that screeners receive
required training according to Standards for Internal Controls in the
Federal Government. We interviewed TSA officials from the Office of
Workforce Performance and Training and the Office of Aviation Operations
in Arlington, Virginia. At the airports we visited, we interviewed Federal
Security Directors and their staffs, such as Training Coordinators. We
also met with officials from four aviation associations-the American
Association of Airport Executives, Airports Council International, the Air
Transport Association, and the Regional Airline Association. We did not
assess the methods used to develop TSA's screener training program, nor
did we analyze the contents of TSA's curriculum. Although we could not
independently verify the reliability of all of this information, we
compared the information with other supporting documents, when available,
to determine data consistency and reasonableness. We found the data to be
sufficiently reliable for our purposes.
To determine what efforts TSA has taken to assess and to enhance screener
performance in detecting threat objects, we reviewed related reports from
the Department of Transportation and the Department of Homeland Security
(DHS) Inspector General, Congressional Research Service, and TSA, as well
as prior GAO reports. We obtained and reviewed TSA's covert test data and
results of the annual recertification testing. (Results of the covert
testing are classified and will be the subject of a separate classified
GAO report.) We discussed methods for inputting, compiling, and
maintaining the data with TSA officials. We also assessed the methodology
of TSA's covert tests and questioned OIAPR officials about the procedures
used to ensure the reliability of the covert test data. When we found
discrepancies between the data OIAPR maintained in spreadsheets and the
data included in the hard copy reports we obtained
Appendix II: Objectives, Scope, and Methodology
from TSA, we worked with OIAPR to resolve the discrepancies. Further, we
visited TSA headquarters to review TSA's annual recertification testing
modules and discuss TSA's process for validating the recertification
exams. As a result, we determined that the data provided by TSA were
sufficiently reliable for the purposes of our review. We also reviewed
TSA's performance measures, targets, and indexes. Finally, we interviewed
TSA headquarters officials from several offices in Arlington, Virginia,
including Aviation Operations, Workforce Performance and Training,
Strategic Management and Analysis, and Internal Affairs and Program
Review.
In addition, in accomplishing our objectives, we also conducted site
visits at select airports nationwide to interview Federal Security
Directors and their staffs and conducted two Web-based surveys of Federal
Security Directors. Specifically, we conducted site visits at 29 airports
(13 category X airports, 9 category I airports, 3 category II airports, 3
category III airports, and 1 category IV airport) to observe airport
security screening procedures and discuss issues related to the screening
process with TSA, airport, and airline officials. We chose these airports
to obtain a crosssection of all airports by size and geographic
distribution. In addition, we selected each of the five contract screening
pilot airports. The results from our airport visits provide examples of
screening operations and issues but cannot be generalized beyond the
airports visited because we did not use statistical sampling techniques in
selecting the airports. The category X airports we visited were Baltimore
Washington International Airport, Boston Logan International Airport,
Chicago O'Hare International Airport, Dallas/Fort Worth International
Airport, Denver International Airport, Washington Dulles International
Airport, John F. Kennedy International Airport, Los Angeles International
Airport, Newark Liberty International Airport, Orlando International
Airport, Ronald Reagan Washington National Airport, San Francisco
International Airport, Seattle-Tacoma International Airport. The category
I airports we visited were Burbank-Glendale-Pasadena Airport, John Wayne
Airport, Chicago Midway International Airport, Dallas Love Field, Kansas
City International Airport, Little Rock National Airport, Metropolitan
Oakland International Airport, Portland International Airport, and Tampa
International Airport. The category II airports we visited were Jackson
International Airport, Dane County Regional Airport, and Greater Rochester
International Airport. The category III airports we visited were Idaho
Falls Regional Airport, Jackson Hole Airport, and Orlando Sanford
International Airport. The category IV airport we visited was Tupelo
Regional Airport.
Appendix II: Objectives, Scope, and Methodology
Further, we administered two Web-based surveys to all 155 Federal Security
Directors who oversee security at each of the airports falling under TSA's
jurisdiction. One survey, the general survey, contained questions covering
local and national efforts to train screeners and supervisors and the
status of TSA's efforts to evaluate screener performance, including the
annual recertification program and TIP. The second survey attempted to
gather more specific airport security information on an airport(s) under
the Federal Security Director's supervision. For the airport-specific
survey, each Federal Security Director received one or two surveys to
complete, depending on the number of airports they were responsible for.
Where a Federal Security Director was responsible for more than two
airports, we selected the first airport based on the Federal Security
Director's location and the second airport to obtain a cross-section of
all airports by size and geographic distribution. In all, we requested
information on 265 airports. However, two airports were dropped from our
initial selection because the airlines serving these airports suspended
operations and TSA employees were redeployed to other airports. As a
result our sample size was reduced to 263 airports, which included all 21
category X, and 60, 49, 73, and 60 category I through IV airports
respectively. In that we did not use probability sampling methods to
select the sample of airports that were included in our airport-specific
survey, we cannot generalize our findings beyond the selected airports.
A GAO survey specialist designed the surveys in combination with other GAO
staff knowledgeable about airport security issues. We conducted pretest
interviews with six Federal Security Directors to ensure that the
questions were clear, concise, and comprehensive. In addition, TSA
managers and an independent GAO survey specialist reviewed the survey.
We conducted these Web-based surveys from late March to mid-May 2004. We
received completed general surveys from all 155 Federal Security Directors
and completed airport-specific surveys for all 263 separate airports for
which we sought information, for 100 percent response rates. The surveys'
results are not subject to sampling errors because all Federal Security
Directors were asked to participate in the surveys and we did not use
probability-sampling techniques to select specific airports. However, the
practical difficulties of conducting any survey may introduce other
errors, commonly referred to as nonsampling errors. For example,
difficulties in how a particular question is interpreted, in the sources
of information that are available to respondents, or in how the data are
entered into a database or were analyzed can introduce unwanted
variability into the survey results. We took steps in the development of
the
Appendix II: Objectives, Scope, and Methodology
surveys, the data collection, and the data editing and analysis to
minimize these nonsampling errors. Also, in that these were Web-based
surveys whereby respondents entered their responses directly into our
database, there was little possibility of data entry or transcription
error. In addition, all computer programs used to analyze the data were
peer reviewed and verified to ensure that the syntax was written and
executed correctly.
We performed our work from May 2003 through April 2005 in accordance with
generally accepted government auditing standards. Certain information we
obtained and analyzed regarding screener training and performance are
classified or are considered by TSA to be sensitive security information.
Accordingly, the results of our review of this information are not
included in this report.1
1We issued two additional reports detailing the results of our review,
which discuss information deemed to be classified or sensitive security
information. The report containing sensitive security information is GAO,
Aviation Security: Screener Training and Performance Measurement
Strengthened, but More Work Remains, GAO-05-143SU (Washington D.C.: Feb.
28, 2005). The report containing classified and sensitive security
information is GAO, Results of Transportation Security Administration's
Covert Testing for Passenger and Checked Baggage Screening for September
2002 through September 2004, GAO-05-437C (Washington D.C.: Apr. 7, 2005).
Appendix III: TSA Screener Training Tools Designed to Help Improve Screener
Performance
Training tool Purpose Status reported by
TSA
Hand Held Metal Provide an informative and Deployed November
Detector effective learning tool to enhance 21, 2003
and Pat Down Video screeners' skills in the areas of
hand-wanding and pat-down
searches of passengers.
MBS II Weapons Kits This tool allows screeners to touch actual improvised
explosive Deployed January 26, 2004 device (IED) components and build
their own devices. This experiential learning will enable screeners to
more readily detect real IEDs during screening. These weapons are also
used to assist in training by using them for live testing conducted by FSD
staff.
Firearms Weapons This tool allows screeners to touch Deployed January
Kits actual firearms and begin to 26, 2004
understand how they can be broken down
into various parts. By
understanding this and experiencing it,
screeners are better able to
see the components of a firearm during
actual screening. These
weapons are also used to assist in
training by using them for live
testing conducted by FSD staff.
X-Ray Operator Video Maintain and enhance the screeners' X-ray image
operational skills. Deployed February 5, 2004
X-Ray Tutor Version 1 Provide a tool that includes about 14,000 image
combinations to Deployed February 15, 2004 practice threat identification.
Basic Supervisory Technical Assist screener supervisors to understand
additional roles they must Deployed April 16, 2004
Web-Based Training 1 perform, including enhanced technical skills needed
to adequately supervise screening functions and resolve alarms using
interactive, performance-based training tool
Mobile Training Assist These teams go into airports where data shows
performance needs
Teams (MTAT) attention. The team offers a variety of services to assist
in improving the performance, such as on-the-spot training and consulting
services. Team visits can be initiated by FSDs, Internal Affairs reports,
Quality Assurance trips, or MTAT Supervisors proactively visiting the
airport and FSD.
Basic Screener Supervisory Improve screener supervisors' knowledge of
federal government and Classroom Training TSA personnel rules and how to
effectively coach and communicate Site visits completed from October 2003
through December 3, 2004:
o North Central (37 visits)
o South Central (51 visits)
o Northeast (25 visits)
o Southeast (60 visits)
o Western (53 visits)
o 54 FSD Stakeholder Relations Meetings
Approximately 3,800 supervisors have been trained.
with employees.
ETD Maintenance Course Certification of screeners to perform supervisory
maintenance tasks Delivered April 1, 2004 above and beyond operator
training.
Credential Verification Provide students with basic skills needed to
verify the identity of flying Deployed April 15, 2004 Training armed law
enforcement officers.
Threat in the Spotlight This weekly product brings to light actual cases
of weapons being Deployed weekly in 2004 found by law enforcement, with an
explanation of how those weapons could be used to attack aviation.
EDS Operations Web-Provide interactive, performance based recurrent
Web-based training Deployed April 30, 2004 Based Training modules for
checked baggage explosive detection systems (EDS).
Appendix III: TSA Screener Training Tools Designed to Help Improve Screener
Performance
Training tool Purpose Status reported by TSA
Handwanding and Pat Improve screener performance by providing an
interactive tool Deployed April 30, 2004
Down Web-Based Training complementary to Hand Held Metal Detector and Pat
Down Video that allows the screener to practice proper techniques and
receive immediate feedback.
Customer Service Web-Reinforces TSA's customer service principles and
places the Deployed April 30, 2004 Based Training screener in various
situations requiring effective customer service
responses.
Checkpoint and Checked Provide interactive, performance-based recurrent
training modules for Deployed April 30, 2004
Baggage Operations Web-checkpoint and checked baggage operations.
Based Training
Physical Bag Search Video Maintain and enhance screeners' explosive trace
detection (ETD) Deployed April 30, 2004 and physical bag search skills for
carry-on and checked baggage.
ETD and Physical Bag Provide interactive recurrent Web-based training
modules for ETD Deployed April 30, 2004 Search Web-Based Training and
physical bag search.
Prohibited Items Web-Based Training
Provide an interactive, performance-based training tool to enhance
screener's ability to identify prohibited items.
Deployed June 25, 2004 Effectively Screening Provide an informative and
effective learning tool to maintain and Deployed December 16, 2004
Prosthetics Video enhance the skills of screeners in the areas of persons
with prosthetics.
X-Ray Tutor Version 2 Provide a tool to practice threat identification
with about 10,000,000 Scheduled to be deployed image combinations. during
the second quarter of fiscal year 2005
X-Ray Simulator Training Sharing the X-Ray Tutor Version 2 library, this
tool will allow Scheduled to be deployed screeners to practice finding
threat items using the full capabilities of during the second quarter of
the TIP-ready X-ray machines. fiscal year 2005
Basic Supervisory Technical Provide an interactive, performance-based tool
to convey how the To be developed Web-Based Training 2 supervisor is to
handle screening situations, handed off by the
screening, following standard operator procedures.
Safety, Lifting, and Twisting Provide a Web-based training that will
engage the student with 3-Scheduled to be deployed
Web-Based Training dimensional representations of the muscular frame,
showing proper during the second quarter of lifting techniques and the
results of improper techniques. fiscal year 2005
Source: TSA.
Appendix IV: Summary of TSA's Short-Term Action Items for Strengthening
Passenger Screener Performance
People Action item Description Benefit
Increase FSD support Hold FSDs accountable for screening Management
accountability is driven down to the local and accountability performance
and delivery of security airport
FSD performance is linked to screener performance, creating incentives for
maintaining and improving security
Enhance training Provide ongoing training for screeners
and supervisors to maintain their skills and provide new skills and
techniques based on evolving threats and lessons learned
Maintains and improves knowledge base of screeners
Ensures proper oversight by supervisors
Ensures that screeners are capable of addressing evolving threats
Increase Internal Increase the frequency of TSA covert Improved
identification of systemic vulnerabilities in Affairs covert testing
testing airport security systems
Immediate implementation of limited remedial actions
Continue to pursue Better understand reasons and causes for Reduces
human-based errors
human performance human errors and interactions with Increases workforce
morale and working conditions,
improvements technology in order to identify leading to improved
performance opportunities for performance improvements, with a goal of
identifying optimum work conditions
Technology
Continue to research Identifies threats
Continue to identify alternative more accurately and
quickly
screening technology technologies and seek Decreases number of
improvements short-term technological false positives from
solutions, especially for equipment
potential vectors
The TIP system is a series Maintains alertness of
Finish installing TIP of 2,400 screeners
images of threat objects Identifies individual
that can be automatically screener performance
fed into X-ray machines issues
during actual screening
Expedite high-speed Connect all TSA offices, checkpoints and Provides
immediate feedback on and response to
connectivity to screening equipment (X-rays, EDS screener performance
issues
checkpoints and machines) to the Internet in order to
training computers automate and improve processes that are Improves
communication with managers in the field
currently done manually or not at all
Process
Refresh aviation Conduct a thorough and expedited review Maintains
"freshness" of standard operating
operations policy, of all policies and procedures developed procedures
based on most recent intelligence about
procedures, and during the rollout of TSA with a focus on security threats
practice increasing screening performance and Removes or updates outdated
or unnecessary
capabilities screening techniques based on lessons learned
9 Improve workforce Determine the optimal workforce staffing Maximizes
utilization of existing resources management levels based on latest
passenger flows and other factors
Source: TSA.
Appendix V: Comments from the Department of Homeland Security
Appendix VI: GAO Contacts and Acknowledgments
GAO Contacts Cathleen A. Berrick (202) 512-8777 Maria D. Strudwick (202)
512-5419
Staff In addition to those named above, David Alexander, Leo Barbour,
Lisa Brown, Elizabeth Curda, Kevin Dooley, Kathryn Godfrey,
Acknowledgments David Hooper, Christopher Jones, Stuart Kaufman, Kim
Gianopoulos, Thomas Lombardi, Cady S. Panetta, Minette Richardson, Sidney
Schwartz, Su Jin Yon, and Susan Zimmerman were key contributors to this
report.
GAO's Mission
Obtaining Copies of GAO Reports and Testimony
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly
released reports, testimony, and correspondence on its Web site. To have
GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected] Programs Automated answering system: (800)
424-5454 or (202) 512-7470
Gloria Jarmon, Managing Director, [email protected] (202)
512-4400Congressional U.S. Government Accountability Office, 441 G Street
NW, Room 7125 Relations Washington, D.C. 20548
Public Affairs Paul Anderson, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
PRINTED ON RECYCLED PAPER
*** End of document. ***