Aviation Security: Flight and Cabin Crew Member Security Training Strengthened, but Better Planning and Internal Controls Needed (06-SEP-05, GAO-05-781). Training flight and cabin crew members to handle potential threats against domestic aircraft is an important element in securing our nation's aviation system. The responsibility for ensuring that crew members are prepared to handle these threats is a shared responsibility between the private sector--air carriers--and the federal government, primarily the Transportation Security Administration (TSA). This report addresses (1) actions TSA has taken to develop guidance and standards for flight and cabin crew member security training and to measure the effectiveness of the training, (2) how TSA ensures domestic air carriers comply with the training guidance and standards, and (3) efforts TSA has taken to develop and assess the effectiveness of its voluntary self-defense training program. -------------------------Indexing Terms------------------------- REPORTNUM: GAO-05-781 ACCNO: A35717 TITLE: Aviation Security: Flight and Cabin Crew Member Security Training Strengthened, but Better Planning and Internal Controls Needed DATE: 09/06/2005 SUBJECT: Air transportation Aviation security Counterterrorism Employee training Flight crews Performance measures Standards Strategic planning Terrorism ****************************************************************** ** This file contains an ASCII representation of the text of a ** ** GAO Product. ** ** ** ** No attempt has been made to display graphic images, although ** ** figure captions are reproduced. Tables are included, but ** ** may not resemble those in the printed version. ** ** ** ** Please see the PDF (Portable Document Format) file, when ** ** available, for a complete electronic file of the printed ** ** document's contents. ** ** ** ****************************************************************** GAO-05-781 United States Government Accountability Office GAO Report to Congressional Requesters September 2005 AVIATION SECURITY Flight and Cabin Crew Member Security Training Strengthened, but Better Planning and Internal Controls Needed GAO-05-781 [IMG] September 2005 AVIATION SECURITY Flight and Cabin Crew Member Security Training Strengthened, but Better Planning and Internal Controls Needed What GAO Found Since the terrorist attacks of September 11, 2001, TSA enhanced guidance and standards for flight and cabin crew member security training with input from stakeholders. Specifically, TSA revised the guidance and standards to include additional training elements required by law and to improve the organization and clarity of the guidance and standards. Some stakeholders we interviewed and our own review generally found that the revised guidance and standards improved upon previous versions in terms of organization and clarity of the information provided. However, some stakeholders identified concerns about, for example, the reasonableness of applying parts of the guidance and standards to both flight and cabin crew members and the difficulty in implementing some of the standards without additional information or training tools from TSA. Additionally, TSA has not established strategic goals and performance measures for assessing the effectiveness of the training because it considers its role in the training program as regulatory. In this regard, TSA views the individual air carriers as responsible for establishing performance goals and measures for their training programs, but has not required them to do so. Without goals and measures, TSA and air carriers will be limited in their ability to fully assess accomplishments and target associated improvements. TSA recently took steps to strengthen its efforts to oversee air carriers' flight and cabin crew security training to ensure they are complying with the required guidance and standards. For example, in January 2005, TSA added staff with expertise in designing training programs to review air carriers' crew member security training curriculums and developed a standard form for staff to use to conduct their reviews. However, TSA lacks adequate controls for monitoring and reviewing air carriers' crew member security training, including written procedures for conducting and documenting these reviews. TSA plans to develop written procedures, but has not established a timeframe for completing this effort. TSA has developed an advanced voluntary self-defense training program with input from stakeholders and implemented the program in December 2004, as required by law. However, stakeholders and our own analysis identified concerns about the training design and delivery, such as the lack of recurrent training and the lack of a realistic training environment. Also, TSA has not yet established performance measures for the program or established a time frame for evaluating the program's effectiveness. United States Government Accountability Office Contents Letter Results in Brief Background TSA Has Enhanced Guidance and Standards for Flight and Cabin Crew Security Training with Input from Stakeholders, but Better Planning Is Needed Recent Progress Made to Strengthen Oversight of Flight and Cabin Crew Member Security Training Program, but Weaknesses Remain TSA Developed and Implemented Crew Member Self-Defense Training, but Has Not Established Performance Measures or Evaluated Training Effectiveness Conclusions Recommendations Agency Comments and Our Evaluation 1 3 8 15 20 26 35 36 37 Appendix I Objectives, Scope, and Methodology Appendix IITSA's Deployment of an Advanced Voluntary Crew Member Self-Defense Training from December 2004 through September 2005, as of July 2005 Appendix IIIComments from the Department of Homeland Security Appendix IV GAO Contact and Staff Acknowledgments Tables Table 1: List of Basic Crew Member Security Training Elements Required By Law 13 Table 2: List of Advanced Voluntary Crew Member Self-Defense Training Elements Required By Law 14 Page i GAO-05-781 Aviation Security Table 3: Summary of Stakeholders' Concerns on TSA's Prototype Advanced Voluntary Self-Defense Training and Actions Taken by TSA in Response to the Concerns 29 Table 4: Crew Member Participation in Advanced Voluntary Crew Member Self-Defense Training from December 2004 through June 2005 30 Figure Figure 1: Number of Domestic Air Carriers by Carrier Group and Percentage of Passengers Flown Domestically by Carrier Group Abbreviations ATSA Aviation and Transportation Security Act DHS Department of Homeland Security DOD Department of Defense DOT Department of Transportation FAA Federal Aviation Administration FAMS Federal Air Marshal Service FBI Federal Bureau of Investigation GPRA Government Performance and Results Act OMB Office of Management and Budget TSA Transportation Security Administration Vision 100 Vision100-Century of Aviation Reauthorization Act This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Page ii GAO-05-781 Aviation Security United States Government Accountability Office Washington, DC 20548 September 6, 2005 The Honorable Daniel K. Inouye Co-Chairman Committee on Commerce, Science, and Transportation United States Senate The Honorable John D. Rockefeller, IV Ranking Minority Member Subcommittee on Aviation Committee on Commerce, Science, and Transportation United States Senate Training flight and cabin crew members (pilots and flight attendants) to handle potential threats against domestic commercial aircraft is an important element in securing our nation's aviation system. These flight and cabin crew members play a key role in ensuring the security of commercial aircraft and the safety of airline passengers and are the last line of defense in the event of an attempted terrorist attack onboard a commercial aircraft. The responsibility for ensuring that flight and cabin crew members are prepared to respond to these threats is a shared responsibility between the private sector-air carriers-and the federal government, primarily the Transportation Security Administration (TSA). In an effort to strengthen aviation security, title 49 of the United States Code, section 44918, requires that air carriers develop and deliver a basic security training program for flight and cabin crew members in accordance with training elements prescribed in the law. The law also requires that TSA monitor and periodically review air carriers' training programs to ensure they adequately prepare crew members for potential threat conditions. Furthermore, the law requires TSA to develop and provide an advanced voluntary self-defense training program for flight and cabin crew members. Although there has not been another successful terrorist attack against commercial aircraft in the United States since September 11, 2001, concerns remain about whether flight and cabin crew members are prepared to handle a potential hijacking and other threats to commercial aircraft. To determine the progress TSA has made in developing and monitoring flight and cabin crew security training, we examined TSA's efforts to develop guidance and standards for air carriers' flight and cabin crew security training, monitor air carriers' compliance with the guidance and standards, and develop and deliver advanced voluntary self-defense training for crew members. Specifically, this report addresses the following questions: (1) What actions has TSA taken to develop guidance and standards for flight and cabin crew security training and to measure the effectiveness of the training? (2) How does TSA ensure domestic air carriers comply with required training guidance and standards? (3) What efforts has TSA taken to develop, implement, and measure the effectiveness of advanced voluntary self-defense training for flight and cabin crew members? In conducting our work, we reviewed TSA and Federal Aviation Administration (FAA) documentation related to flight and cabin crew member security training guidance and standards and advanced voluntary crew member self-defense training. The security training guidance provides detailed guidance from which air carriers must develop their flight and cabin crew security training programs. The security training standards set forth the specific requirements for crew member security training, such as the training elements that must be included in air carriers' basic and recurrent (refresher) crew member security training programs. We also interviewed officials from 19 domestic air carriers, 2 air carrier associations, and 6 crew member labor organizations regarding security training guidance and standards, and advanced voluntary self-defense training.1 We selected domestic air carriers based on whether they were currently offering initial and/or recurrent security training2 and on the size of the air carrier in an effort to include a mixture of various domestic air carriers and air carriers of varying sizes. The size of an air carrier is based on the annual operating revenues and the number of revenue passenger boardings. We visited 8 of these air carriers to observe their training and to 1Foreign air carriers operating to and from the United States are not regulated under the mandates of the Aircraft Operator Security Program (under which domestic air carriers operate) but are subject to the security requirements of 49 C.F.R. part 1546. Foreign air carriers operating under part 1546 must maintain a security program that TSA accepts as providing passengers a level of protection similar to the level of protection provided by U.S. air carriers serving the same airports. See 49 U.S.C. S: 44906. Further, foreign air carriers must abide by the standards and recommended practices established by the International Civil Aviation Organization, which adopted expanded security training requirements for crew members in March 2002 that are similar to those found in the Aviation and Transportation Security Act (ATSA), Pub. L. No. 107-71, S: 107, 115 Stat. 611, 610-11 (2001). 2Initial security training is provided to newly hired flight and cabin crew members as well as crew members returning from a leave of absence of more than 3 years. Recurrent security training is annual refresher training provided to all flight and cabin crew members. interview flight and cabin crew members about the training. Because we selected a nonprobability sample of domestic air carriers, the information we obtained from these interviews and visits cannot be generalized to all domestic air carriers. We also observed two sessions of TSA's advanced voluntary crew member self-defense training. Information obtained from our interviews and visits cannot be generalized to all air carriers and stakeholders. Additionally, we interviewed officials responsible for developing security training guidance and standards and reviewing air carriers security training programs at TSA headquarters. We also interviewed officials at the FAA; Federal Bureau of Investigation (FBI); Bureau of Immigration and Customs Enforcement, Federal Air Marshal Service (FAMS); Department of Defense (DOD); crew member labor organizations; and associations representing air carriers to obtain information on their involvement in the development of crew member security training guidance and standards and advanced voluntary crew member self-defense training. A more detailed description of our scope and methodology is contained in appendix I. We conducted our work from June 2004 through August 2005 in accordance with generally accepted government auditing standards. Building on the legislatively mandated guidance developed by FAA in January 2002 and corresponding standards, TSA enhanced guidance and standards for flight and cabin crew member security training with input from various stakeholders. However, TSA has not established strategic goals and performance measures for assessing the effectiveness of crew member security training, nor required air carriers to do so. TSA officials stated that they revised the guidance and standards for two main reasons. First, the law, as amended by the Vision 100--Century of Aviation Reauthorization Act (Vision 100), enacted in December 2003, required that air carriers include additional training elements in their basic crew member security training programs to prepare flight and cabin crew Results in Brief members for potential threat conditions.3 Second, TSA determined that the guidance and standards needed to be better organized and to more clearly define security training elements, in part due to feedback from air carriers, flight and cabin crew member labor organizations, and associations representing air carriers. Some stakeholders stated and our own review found that the organization of the previous security training standards were difficult to follow and lacked clarity in some areas. For example, the previous guidance did not define what constitutes life-threatening behavior, whereas the revised guidance provides both a definition of this behavior and examples. Although the revised guidance and standards were an improvement over the previous versions in terms of organization and clarity, some stakeholders identified concerns about the reasonableness of applying parts of the guidance and standards to both flight and cabin crew members, difficulty in implementing some of the standards without additional information or training tools from TSA, and the vagueness of some of the guidance and standards. For example, some of the training standards remain generalized to both flight attendants and pilots, rather than targeted to their specific job functions in responding to a security threat. Additionally, TSA has not established strategic goals and performance measures for the flight and cabin crew member security training program. The Government Performance and Results Act (GPRA) of 1993 requires, among other things, that agencies use outcome-oriented goals and measures that assess results of a program or activity compared to its intended purpose. TSA training officials stated that they decided not to develop strategic goals or performance measures for flight and cabin crew security training because the officials view their role in the training program as regulatory-that is, monitoring air carriers' compliance with the training guidance and standards established by TSA. In this regard, officials stated that it is the individual air carriers' responsibility to establish performance goals and measures specific to their security 349 U.S.C. S: 44918. The Aviation and Transportation Security Act (ATSA), enacted in November 2001, created the Transportation Security Administration (TSA) and mandated the Federal Aviation Administration (FAA) to develop a detailed guidance for a scheduled passenger air carrier flight and cabin crew member training program. FAA continued to be responsible for overseeing flight and cabin crew members' security training until TSA assumed that responsibility pursuant to ATSA. Subsequently, the Vision 100-Century of Aviation Reauthorization Act (Vision 100), enacted in December 2003, revised the basic security training program requirements and mandated that TSA develop and implement an advanced voluntary self-defense training program for crew members. Pub. L. No. 108-176, S: 603, 117 Stat. 2490, 2563-65. Together, these acts mandated actions for TSA, FAA, and air carriers to strengthen flight and cabin crew security training under S: 44918, including the development of security training guidance and standards. training programs. However, without overall strategic goals developed by TSA, air carriers do not have a framework from which to develop their individual performance goals and measures. Further, TSA has not explicitly required air carriers to develop performance goals and measures or provided air carriers with guidance and standards for doing so. The absence of performance goals and measures for flight and cabin crew security training limits the ability of TSA and air carriers to fully assess the accomplishments of the flight and cabin crew member security training program in adequately training crew members, and to target appropriate improvements. Although TSA has recently taken steps to strengthen its efforts to oversee air carriers' flight and cabin crew security training to ensure carriers are complying with required guidance and standards, TSA lacks adequate internal controls for monitoring and reviewing air carriers' flight and cabin crew member security training. The law requires TSA to monitor and periodically review air carriers' security training to ensure that the training is adequately preparing crew members for potential threat conditions. The law also requires TSA to consider complaints from crew members in determining when to review an air carriers' crew member security training program. Further, the Comptroller General's Standards for Internal Control in the Federal Government, which describes the minimum level of quality acceptable for internal controls in government and the basis against which internal controls should be evaluated, calls for, among other things, that controls generally be designed to assure that ongoing monitoring occurs during the course of normal operations, transactions and other significant events be documented clearly, and documentation be readily available for examination.4 TSA has recently taken several steps to strengthen its review of air carriers' crew member security training curriculum, including (1) adding staff with expertise in designing training programs to review the overall design of the air carriers' crew member security training curriculums, rather than solely ensuring that each of the training elements is discussed in the training curriculums; (2) developing a standard form for TSA inspectors and training staff to use to conduct and document their reviews of air carrier security training curriculums to ensure training requirements stated in the revised security training standards are included in the air carriers' security training and to enhance consistency in the review process; (3) requiring air carriers to obtain 4GAO, Internal Control: Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999). written participant feedback at the end of flight and cabin crew member security training to capture information on participants' overall satisfaction with the training; and (4) reorganizing the inspection staff into a newly created Office of Compliance and issuing position announcements to fill vacant TSA inspector positions, which should allow for greater monitoring of air carriers' flight and cabin crew security training programs. Although TSA made these recent improvements, the agency lacks adequate internal controls for monitoring and reviewing air carriers' crew member security training. Specifically, TSA does not have written procedures for (1) completing the standard form used by TSA inspectors and training staff in reviewing air carriers' flight and cabin crew security training curriculum; (2) conducting and documenting observations of air carriers' classroom delivery of flight and cabin crew security training-a function performed by TSA inspectors-and selecting the air carriers to visit; (3) ensuring that air carriers track flight and cabin crew members' completion of required security training; and (4) considering flight and cabin crew members' written feedback on security training and securityrelated complaints from flight and cabin crew members in determining when to review air carriers' flight and cabin crew security training. TSA officials stated that they plan to develop a handbook that documents procedures for TSA inspectors to use in reviewing air carriers' flight and cabin crew members security training as well as guidance for the training staff to use during their reviews. However, officials stated that they have not established a time frame for completing these efforts. Until TSA documents procedures for monitoring and reviewing air carriers' flight and cabin crew member security training, TSA will continue to lack a structure that provides reasonable assurance that TSA inspectors and training staff are performing thorough assessments of air carriers' security training. TSA has developed an advanced voluntary self-defense training program for flight and cabin crew members with input from key stakeholders and implemented the program in December 2004 as required by law. However, some stakeholders and our own analysis identified concerns about the training design and delivery, and TSA has not yet developed performance measures for the program or established a time frame for evaluating the program's overall effectiveness. In developing the advanced voluntary selfdefense training program, TSA consulted with law enforcement experts, the Federal Air Marshal Service,5 air carriers, air carrier associations, crew member labor organizations, and other subject matter experts and selfdefense experts. TSA also piloted the training course in five cities and made refinements to the course based on feedback from participants. For example, training participants stated that the course included too much lecture time and that the training course was too long, particularly given that crew members must attend the training on their own time and must pay for their travel, lodging, and meals. In response, TSA increased the amount of hands-on training and shortened the training program from 4 days to 3 days. During the first 7 months of the training program, participation in the training was relatively low, with only 474 flight and cabin crew members (39 percent of total capacity) attending the training. TSA officials attributed the low participation to crew members having a difficult time obtaining 3 consecutive days of leave to attend the training. Although TSA incorporated some stakeholder input into the course design, some stakeholders, including individuals identified as experts by TSA, and our own analysis identified several concerns regarding the training design and delivery, such as the lack of recurrent training, the lack of a realistic setting in which to conduct the training, and instructors' lack of knowledge of crew members' actual work environment. Our prior human capital work has found that in implementing a training program, an agency should ensure that implementation involves effective and efficient delivery of training6-that is, the training should be conducted in a setting that approximates the participants' working conditions and be taught by individuals who are knowledgeable about the subject matter and work environment. Furthermore, TSA has not yet developed performance measures for the program or established a time frame for evaluating the program's overall effectiveness. TSA training officials stated that although they recognize the importance of measuring the effectiveness of the selfdefense training program, they have not been able to undertake this effort due to resource constraints within which they are operating and numerous internal process improvements currently under way in TSA. Without performance measures and an evaluation of the program's effectiveness, TSA will not have meaningful information from which to determine 5The Federal Air Marshal Service provides air security and promotes public confidence in our nation's civil aviation system through the deployment of Federal Air Marshals who are specially trained civil aviation security specialists that are deployed onboard aircraft to protect passengers, crew, and aircraft from terrorist activities on both domestic and international flights. 6GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, GAO-04-546G (Washington, D.C.: March 2004). whether the voluntary training program is achieving its intended results and to make any needed improvements. To assist TSA in further strengthening its flight and cabin crew security training program and to provide the tools necessary to monitor the delivery and accomplishments of the training, we are making a number of recommendations to the Secretary of the Department of Homeland Security (DHS). These recommendations include directing the Assistant Secretary, Transportation Security Administration, to develop a framework from which to assess the accomplishments of flight and cabin crew member security training, including establishing strategic goals for the training; developing guidance for air carriers to use in establishing performance goals and measures for their individual training programs; and reviewing air carriers' goals and measures as part of its monitoring efforts. We are also recommending that the Assistant Secretary, Transportation Security Administration, strengthen internal controls for monitoring and reviewing air carriers' flight and cabin crew member security training programs and for documenting the results of its monitoring efforts; and establish performance measures for the advanced voluntary crew member self-defense training program and a time frame for evaluating the effectiveness of the training. We provided a draft of this report to the Department of Homeland Security for its review and comment. DHS, in its written comments, generally concurred with the findings and recommendations in the report, and agreed that efforts to implement our recommendations are critical to a successful flight and cabin crew member security training program. DHS described some actions TSA has taken or planned to take to implement these recommendations. For example, DHS stated that TSA has begun to establish strategic goals for the flight and cabin crew member security training program. DHS also stated that TSA is in the process of developing a monitoring plan, to the extent that resources permit, and a handbook for reviewing air carriers' flight and cabin crew member security training programs. Additionally, DHS stated that TSA is currently working with the Office of Management and Budget (OMB) to establish performance measures for use in OMB's Performance Assessment Rating Tool for TSA's flight security training. The full text of DHS's comments is included in appendix III. Background After the terrorist attacks of September 11, 2001, the President signed the Aviation and Transportation Security Act (ATSA) into law on November 19, 2001, with the primary goal of strengthening the security of the nation's aviation system. ATSA created TSA as the agency responsible for securing all modes of transportation, including aviation.7 The President also issued the National Strategy for Homeland Security in July 2002. The National Strategy for Homeland Security sets forth a plan to strengthen homeland security through the cooperation of federal, state, local, and private-sector organizations in various areas. The National Strategy for Homeland Security aligns and focuses homeland security functions into six critical mission areas: (1) intelligence and warning, (2) border and transportation security, (3) domestic counterterrorism, (4) protecting critical infrastructures and key assets, (5) defending against catastrophic threats, and (6) emergency preparedness and response. A theme of the national strategy is that homeland security is a shared responsibility among these stakeholders, not solely the responsibility of the federal government. In the case of flight and cabin crew member security training, air carriers and TSA both play an important role. Air carriers are responsible for developing and delivering security training programs for their crew members. TSA (and previously FAA) is responsible for developing the guidance and standards that air carriers are to use to design and deliver their security training and for monitoring air carriers' flight and cabin crew member security training programs for compliance with the guidance and standards.8 If TSA finds an air carrier to be noncompliant with developing and conducting the required flight and cabin crew member security training, TSA has a range of actions it can take, including imposing fines, and in extreme circumstances, force the air carrier to shut down its operations. The Bureau of Transportation Statistics reported that 105 domestic passenger air carriers were operating in the United States in 2004. Of the 7ATSA created TSA as an agency within the Department of Transportation (DOT). The Homeland Security Act of 2002, Pub. L. No. 107-296, 116 Stat. 2135, signed into law on November 25, 2002, transferred TSA from the DOT to the new Department of Homeland Security. FAA continued to be responsible for overseeing flight and cabin crew members' security training until TSA assumed that responsibility pursuant to ATSA. On or about February 17, 2002, TSA assumed responsibility for flight and cabin crew members' security training. Initially, however, TSA was neither staffed nor organized to meet that responsibility on that date. Accordingly, while TSA "ramped up" its operations FAA's Office of Aviation Security, while being largely absorbed into TSA, continued to assist TSA in its new roles as regulator and overseer. 8The flight and cabin crew security training implemented by air carriers pursuant to their security programs, the corresponding guidance and standards, and the related oversight discussed in this report are the responsibility of TSA, and not of FAA, which is responsible for approved training programs required under 14 C.F.R. part 121. 105 air carriers, 12 (11 percent) are major air carriers that carried over 76 percent of the passengers in 2004. With a few exceptions for small aircraft, every commercial flight in the United States has at least two flight crew members and one cabin crew member onboard. These crew members are viewed as the last line of defense in what TSA describes as its layered security system, which includes perimeter security (e.g., airport security fencing), 100 percent passenger and checked baggage screening, hardened flight deck doors, armed federal air marshals, and armed pilots.9 Figure 1 provides the number of domestic air carriers by carrier group (major, national, and regional) and the percentage of passengers flown domestically by carrier group during fiscal year 2004.10 9Pilots (commercial, charter, and all cargo aircraft) who volunteer to participate in the Federal Flight Deck Officers Program are trained and armed to protect the aircraft cockpit. The pilots are deputized as federal flight deck officers. 10The Bureau of Transportation Statistics defines major carriers as those with over $1 billion in annual operating revenues, national carriers as those with $100 million to $1 billion in annual operating revenues, and regional (large and medium air carriers) as those with up to $100 million in annual operating revenues. Figure 1: Number of Domestic Air Carriers by Carrier Group and Percentage of Passengers Flown Domestically by Carrier Group Major National Regional Source: GAO analysis of Bureau of Transportation Statistics data. Federal guidance for air carriers to use to develop their flight and cabin crew security training programs has been in place for over 20 years. FAA developed the crew member security training guidance, referred to as Common Strategy I, in the early 1980's in response to numerous hijacking incidents in the late 1970's. Common Strategy I generally instructed air carriers to develop training programs that called for flight and cabin crew members to cooperate with threatening passengers or hijackers and slow compliance with their demands. Based on this guidance, FAA also developed corresponding security training standards that set forth the requirements for flight and cabin crew member security training. Air carriers were required to incorporate the guidance and standards into their security training programs. FAA principal security inspectors and principal operations inspectors were responsible for monitoring air carriers' compliance with the security training standards. The nature of the terrorist attacks on September 11, 2001, however, demonstrated that the philosophy of Common Strategy I-to cooperate with hijackers-was flawed as it presumed that hijackers would not use aircraft as weapons of mass destruction. Following the events of September 11, 2001, section 107 of ATSA required FAA, in consultation with TSA and other stakeholders, to develop detailed guidance for flight and cabin crew security training programs within 60 days after the enactment of the act. FAA developed and issued security training guidance, in accordance with the requirements of ATSA, on January 19, 2002. In February 2002, TSA assumed responsibility for monitoring air carriers' security training for United States passenger air carriers and the air carrier security inspections function was transferred from FAA to TSA.11 Following the enactment of ATSA, the President signed into law two acts that amended the flight and cabin crew training requirements codified at title 49 of the U.S. Code, section 44918-the Homeland Security Act of 2002 and Vision 100. The Homeland Security Act, enacted on November 25, 2002, amended the law by, among other things, mandating that, if TSA updated training guidance, it must issue a rule to include elements of selfdefense in the training programs.12 Vision 100, subsequently enacted on December 12, 2003, amended the flight and cabin crew security training law in its entirety to require that o air carriers providing scheduled passenger air transportation carry out a training program that addresses the 10 elements listed in table 1; o TSA approve the air carrier's training programs; o TSA, in consultation with FAA, monitor air carrier training programs and periodically review an air carrier's training program to ensure the program is adequately preparing crew members for potential threat conditions; o TSA, in consultation with FAA, order air carriers to modify training programs to reflect new or different security threats; and o TSA develop and provide an advanced voluntary self-defense training program to provide both classroom and effective hands-on training in, at least, the six training elements listed in table 2. 11TSA currently oversees about 84 air carriers. Bureau of Transportation Statistics data show that there were 105 commercial passenger air carriers in fiscal year 2004. TSA officials stated that this difference of 21 air carriers may be due to air carriers merging or going out of business. 12TSA, however, took no action pursuant to S: 1403 of the Homeland Security Act. TSA maintained the standards prescribed by ATSA until the passage of Vision 100, which prompted TSA to revise crew member security training guidance and standards. Table 1 lists the minimum training elements required by law, as enacted by ATSA and as amended by Vision 100, for basic crew member security training. Table 1: List of Basic Crew Member Security Training Elements Required By Law Legislative requirements for crew member security training ATSA Vision 100 Determination of the seriousness of any occurrence o o Crew communication and coordination o o Appropriate responses to defend oneself o o Use of protective devices assigned to crew members o o Psychology of terrorists to cope with hijacker behavior o o and passenger responses (Live) situational training exercises regarding various o o threat conditions Flight deck procedures or aircraft maneuvers to defend o o the aircraft Recognizing suspicious activities o The proper commands to give passengers and attackers o The proper conduct of a cabin search, including explosive o device recognition Source: ATSA and Vision 100. Table 2 lists the training elements that TSA must include in an advanced voluntary self-defense training program for flight and cabin crew members under the law, as amended by Vision 100. Table 2: List of Advanced Voluntary Crew Member Self-Defense Training Elements Required By Law Legislative requirements for crew member security training Deterring a passenger who might present a threat Advanced control, striking, and restraint techniques Training to defend oneself against edged or contact weapons Methods to subdue and restrain an attacker Use of available items aboard the aircraft for self-defense Appropriate and effective responses to defend oneself including the use of force against an attacker Source: Vision 100. Over the years, our work on best practices in training has found that generally high-performing organizations follow certain key steps in developing and measuring the effectiveness of training programs.13 These steps include o planning-developing a strategic approach that establishes priorities and leverages investments in training to achieve agency results and identify the competencies-commonly referred to as knowledge, skills, abilities, and behaviors-needed to achieve organizational missions and goals, and measure the extent to which their employees possess these competencies; o design and development-identifying specific initiatives that the agency will use, along with other strategies, to include individual and organizational performance; o implementation-ensuring effective and efficient delivery of training opportunities in an environment that supports learning and change; and o evaluation-assessing the extent to which training efforts contribute to improved performance and results. 13GAO-04-546G and GAO, Human Capital: Selected Agencies' Experiences and Lessons Learned in Designing Training and Development Program, GAO-04-291 (Washington, D.C.: January 30, 2004). TSA Has Enhanced Guidance and Standards for Flight and Cabin Crew Security Training with Input from Stakeholders, but Better Planning Is Needed TSA Enhanced Flight and Cabin Crew Member Security Training Guidance and Standards with Input from Stakeholders, but Stakeholders Continue to Have Concerns