    Chemical Demilitarization: Additional Management Actions Needed
    to Meet Key Performance Goals of DOD's Chemical Demilitarization
    Program (06-DEC-07, GAO-08-134).

    Destruction of the nation's remaining stockpile of chemical
    weapons in a safe, efficient, and timely manner is essential to
    meet Chemical Weapons Convention treaty obligations and to reduce
    the risk of a potential catastrophic event. The Department of
    Defense (DOD) established the Chemical Demilitarization Program
    to manage the destruction of the remaining stockpile. GAO was
    asked to evaluate the (1) progress DOD and the Army have made in
    addressing GAO's prior recommendations to strengthen program
    management, (2) reasonableness of schedule milestones, (3)
    reliability of cost estimates, and (4) effectiveness of efforts
    to provide monetary incentives to the systems contractors. GAO
    reviewed relevant planning documents, schedules, cost estimates,
    and contracts; interviewed program and contractor officials; and
    visited chemical agent destruction sites.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-08-134
        ACCNO:   A78748
      TITLE:     Chemical Demilitarization: Additional Management Actions
    Needed to Meet Key Performance Goals of DOD's Chemical
    Demilitarization Program
         DATE:   12/06/2007
      SUBJECT:   Chemical regulation
    	     Chemical weapons
    	     Chemical weapons disposal
    	     Cost analysis
    	     Cost control
    	     Defense cost control
    	     Evaluation criteria
    	     Performance measures
    	     Program evaluation
    	     Program management
    	     Rates
    	     Risk assessment
    	     Risk management
    	     Schedule slippages
    	     Strategic planning
    	     Program goals or objectives
    	     Program implementation
    	     Chemical Weapons Convention

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-08-134

    This text file was formatted by the U.S. Government Accountability
    Office (GAO) to be accessible to users with visual impairments, as part
    of a longer term project to improve GAO products' accessibility. Every
    attempt has been made to maintain the structural and data integrity of
    the original printed product. Accessibility features, such as text
    descriptions of tables, consecutively numbered footnotes placed at the
    end of the file, and the text of agency comment letters, are provided
    but may not exactly duplicate the presentation or format of the printed
    version. The portable document format (PDF) file is an exact electronic
    replica of the printed version. We welcome your feedback. Please E-mail
    your comments regarding the contents or accessibility features of this
    document to Webmaster@gao.gov.

    This is a work of the U.S. government and is not subject to copyright
    protection in the United States. It may be reproduced and distributed
    in its entirety without further permission from GAO. Because this work
    may contain copyrighted images or other material, permission from the
    copyright holder may be necessary if you wish to reproduce this
    material separately.

    Report to Congressional Requesters:

    United States Government Accountability Office:

    GAO:

    December 2007:

    Chemical Demilitarization:

    Additional Management Actions Needed to Meet Key Performance Goals of
    DOD's Chemical Demilitarization Program:

    Chemical Demilitarization:

    GAO-08-134:

    GAO Highlights:

    Highlights of GAO-08-134, report to congressional requesters.

    Why GAO Did This Study:

    Destruction of the nation�s remaining stockpile of chemical weapons in
    a safe, efficient, and timely manner is essential to meet Chemical
    Weapons Convention treaty obligations and to reduce the risk of a
    potential catastrophic event. The Department of Defense (DOD)
    established the Chemical Demilitarization Program to manage the
    destruction of the remaining stockpile. GAO was asked to evaluate the
    (1) progress DOD and the Army have made in addressing GAO�s prior
    recommendations to strengthen program management, (2) reasonableness of
    schedule milestones, (3) reliability of cost estimates, and (4)
    effectiveness of efforts to provide monetary incentives to the systems
    contractors. GAO reviewed relevant planning documents, schedules, cost
    estimates, and contracts; interviewed program and contractor officials;
    and visited chemical agent destruction sites.

    What GAO Found:

    DOD and the Army have taken steps in addressing GAO�s prior
    recommendations to strengthen program management by establishing an
    overall strategy and supporting implementation plan, but some key
    elements, such as annual performance measures for some key goals,
    including interim destruction goals, are not fully developed. Moreover,
    actions DOD and the Army have taken to identify and mitigate the risk
    of future program schedule extensions and cost growth have not been
    effective because the Chemical Materials Agency�s risk management
    process has not been fully developed or integrated with DOD�s risk
    management process. As a result, managers lack an integrated and
    systematic approach to evaluate and manage risk.

    Recently achieved destruction rates may indicate that adjusted schedule
    milestones are overly conservative. Moreover, the program�s schedule
    was extended in 2005 in response to the slower-than-anticipated
    destruction rates experienced since 2003, when more optimistic
    milestones were adopted. GAO�s review of processing rates achieved at
    the sites since the revised schedule determined that most sites are
    significantly ahead of the revised, extended program estimates for
    recently completed munitions campaigns. For example, Umatilla, Oregon,
    completed destruction of one of its rocket munitions by August 2007,
    about 12 months earlier than forecasted.

    GAO was not able to verify the accuracy of the program�s cost estimates
    because of shortcomings in the underlying cost data, such as undefined
    facility closure requirements and unstable baseline costs. GAO
    determined that the usefulness of the earned value management tool to
    provide visibility over the program�s cost performance is limited
    because of these shortcomings. The program�s projected cost growth is
    largely attributable to longer schedules and increased costs associated
    with facility closure estimates. GAO estimated that a 1-month schedule
    extension would cost an additional $9.5 million. Also, schedules used
    for the closure cost estimates for destruction sites have increased,
    from 12 months to about 30 months at some sites, but they have not been
    finalized or independently reviewed.

    Since 2003, the Army has actively managed criteria for determining
    monetary fees�largely award fees�it provided systems contractors to
    incentivize their performance. These fees had emphasized safety and
    environmental compliance, and Army officials acknowledged that award
    fees did not successfully control schedule and cost growth. In 2006,
    after audit results and DOD direction, and while extending the
    schedule, the Army reinstated schedule as a performance measure. The
    award fee plan allows site project managers flexibility to weigh
    performance measures within a range, and GAO determined that the high
    end of the range for management was generally selected, rather than for
    schedule and cost. Most criteria in the program�s award fee plans are
    subjective and are not linked to objective outcomes as much as
    possible, as called for in DOD guidance. Given the program�s maturity,
    GAO found that the Army could apply more weight to performance
    measures, such as schedule and cost, and add objective criteria in
    these plans

    What GAO Recommends:

    GAO recommends that DOD and the Army develop interim destruction goals,
    approaches, and milestones; establish time frames to complete its risk
    management approach; develop realistic schedule and closure cost
    estimates; finalize and independently review cost estimates; and
    determine whether a greater emphasis can be placed on schedule and
    cost, and develop more objective award fee performance evaluation
    criteria. DOD concurred or partially concurred with 12 of GAO�s 13
    recommendations.

    To view the full product, including the scope and methodology, click on
    [hyperlink, http://www.GAO-08-134]. For more information, contact Davi
    D'Agostino at (202) 512-5431 or dagostinod@gao.gov.

    [End of section]

    Contents:

    Letter:

    Results in Brief:

    Background:

    DOD and the Army Have Taken Steps to Strengthen Program Management, but
    Have Not Fully Developed an Integrated Approach for Measuring
    Performance and Managing Risk:

    Recently Improved Destruction Rates for Chemical Munitions May Indicate
    Overly Conservative Schedule Milestones at the Four Incineration Sites:

    Shortcomings in Underlying Cost Data Make Earned Value Management
    Results Unreliable and Limited Our Ability to Determine Cost Estimate
    Reliability:

    The Army Has Actively Managed Award Fees and Could Further Emphasize
    Schedule and Cost and Link Award Fees to More Objective Performance
    Measures:

    Conclusions:

    Recommendations for Executive Action:

    Agency Comments and Our Evaluation:

    Appendix I: Scope and Methodology:

    Appendix II: Two Recently Introduced Performance-Based Incentive Fees:

    Appendix III: Comments from the Department of Defense:

    Appendix IV: GAO Contact and Staff Acknowledgments:

    Related GAO Products:

    Tables:

    Table 1: Status of DOD's Chemical Agent Stockpile Sites and Chemical
    Agent Destroyed as of September 30, 2007:

    Table 2: Comparison of Some Key Destruction Objectives Identified in
    DOD's Strategic Plan and CMA's Strategic Plan:

    Table 3: Comparison of Performance Measures Identified in the PART, OSD
    Strategic Plan, CMA Strategic Plan, and Selected Acquisition Report:

    Table 4: Comparison of Estimated Operation Completion Dates:

    Table 5: Number of Days Saved per Site since 2005 Program Office
    Schedule Estimates:

    Table 6: Potential Operational Cost Savings Associated with Days Saved
    since 2005 Schedule Estimate:

    Table 7: Number of Months of Improvement in 2007 Program Current
    Estimate for Operations Completion versus 2005 Army Program Office
    Estimate and 2005 DOD Independent Estimate:

    Table 8: CMA Assessed Confidence in Achieving DOD Schedule Milestone
    and April 29, 2012, Treaty Deadline:

    Table 9: Changes in Estimated Total Program Cost:

    Table 10: Comparison of GAO's Independent Estimates to Complete
    Operations and Closure Activities with the Systems Contractor's and
    Program Manager's Estimates:

    Table 11: Percentage of Award Fees Earned at the Incineration Sites and
    Newport, from Fiscal Year 2003 to Fiscal Year 2006:

    Table 12: Performance Measures and Weights in Award Fee Plans, Fiscal
    Years 2003-2007:

    Figures:

    Figure 1: U.S. Stockpile of Chemical Agents and Munitions:

    Figure 2: Umatilla Chemical Agent Disposal Facility, Umatilla Chemical
    Depot, Oregon:

    Figure 3: Delivery of a Ton Container to the Processing Building at the
    Newport Chemical Agent Disposal Facility, Newport Chemical Depot,
    Indiana:

    Figure 4: Intermodal Storage Containers at the Newport Chemical Agent
    Disposal Facility, Newport Chemical Depot, Indiana:

    Figure 5: 2005 Program Office Schedule Estimates by Processing Rates,
    Downtime, and Risk:

    Figure 6: Comparison of the Performance Baseline with the Planned
    Budget and Systems Contractors' Estimate at Completion:

    Figure 7: Comparison of the Monetary Incentive Fees Earned by Systems
    Contractors at the Incineration Sites and at Newport during Fiscal Year
    2006:

    Abbreviations:

    ACWA: Assembled Chemical Weapons Alternatives:

    APB: Acquisition Program Baseline:

    CAIG: Cost Analysis Improvement Group:

    CMA: Chemical Materials Agency:

    CWC: Chemical Weapons Convention:

    DOD: Department of Defense:

    DPPBI: Director's Programmatic Performance Based Incentive:

    EAC: estimate at completion:

    EVM: earned value management:

    OMB: Office of Management and Budget:

    OSD: Office of the Secretary of Defense:

    PART: Program Assessment Rating Tool:

    SAR: Selected Acquisition Report:

    United States Government Accountability Office:

    Washington, DC 20548:

    December 6, 2007:

    The Honorable Adam Smith:
    Chairman:
    The Honorable Mac Thornberry:
    Ranking Member:
    Subcommittee on Terrorism and Unconventional Threats and Capabilities:
    Committee on Armed Services:
    House of Representatives:

    The Honorable Jim Saxton:
    House of Representatives:

    Destruction of the nation's remaining stockpile of lethal chemical
    weapons in a safe, efficient, and timely manner is essential to meeting
    U.S. treaty obligations under the Chemical Weapons Convention
    (CWC)[Footnote 1] and to reducing the risk to the public and the
    environment of a potential catastrophic event. In 1985, Congress
    directed the Department of Defense (DOD) to destroy the U.S. stockpile
    of lethal chemical agents and munitions in the United States with the
    requirement that DOD was to ensure maximum protection for the
    environment, the general public, and the personnel involved in the
    destruction efforts. DOD designated the Department of the Army as its
    executive agent, and the Army established the Chemical Demilitarization
    Program to destroy the stockpile at nine storage sites.[Footnote 2] The
    United States and more than 180 other countries are signatories to the
    CWC, which went into effect in 1997. The CWC prohibits the use of
    chemical weapons and mandates a deadline of April 29, 2012, for the
    United States to destroy the existing stockpiles.[Footnote 3]

    DOD's Chemical Demilitarization Program has not experienced a chemical
    event since its inception in 1985 that resulted in an environmental
    catastrophe or death, but the program has experienced some potentially
    serious events and has been plagued by numerous schedule revisions,
    cost growth, and long-standing management weaknesses. The Chemical
    Demilitarization Program is managed as a major defense acquisition
    program and, as such, has cost and schedule milestones that are
    contained in an Acquisition Program Baseline (APB). As of September 30,
    2006, Congress had appropriated about $15.8 billion for the program. In
    2005, DOD reported that the program had exceeded its milestones under
    the 2003 APB in excess of Nunn-McCurdy thresholds for cost and schedule
    performance[Footnote 4] for the second time in 3 years,[Footnote 5] and
    new schedule milestones and cost estimates were being developed. These
    revisions increased DOD's estimated costs for the entire Chemical
    Demilitarization Program by 33 percent from about $24 billion in 2003
    to about $32 billion in 2005. Moreover, the Army's revised cost
    estimates for the portion of the program responsible for destroying the
    stockpile, the Chemical Stockpile Elimination Project,[Footnote 6]
    increased by more than 53 percent from about $15.8 billion in 2003 to
    about $24.2 billion in 2005. A large percentage of this increase was
    attributable to revised government cost estimates for the destruction
    facility contracts. In April 2006, DOD announced that the program will
    not meet the extended CWC deadline of April 29, 2012, for total
    destruction of its stockpile, and it estimated that only 66 percent of
    the stockpile will be destroyed by then. DOD also announced that it
    would seek resources to complete destruction as close to the CWC
    deadline as possible and separately requested authority to provide
    additional monetary incentives if systems contractors operating the
    incineration sites could accelerate their destruction activities to
    safely exceed, meet, or finish destruction near the CWC deadline.
    Moreover, the program's remaining four incineration sites--Anniston
    Chemical Agent Disposal Facility, Anniston Chemical Activity
    (Anniston), Alabama; Pine Bluff Chemical Agent Disposal Facility, Pine
    Bluff Chemical Activity (Pine Bluff), Arkansas; Tooele Chemical Agent
    Disposal Facility, Deseret Chemical Depot (Tooele), Utah; and Umatilla
    Chemical Agent Disposal Facility, Umatilla Chemical Depot (Umatilla),
    Oregon--were forecasted to miss the CWC deadline for the first
    time.[Footnote 7]

    In the past, the program has had long-term problems in meeting all of
    the schedule milestones for destroying the stockpile, in part because
    of the lengthy environmental permitting processes prior to operation
    start-up and unexpected conditions of the aging munitions. The
    stockpile destruction program managed by the Chemical Materials Agency
    (CMA) has now reached a more mature phase, and CMA and its systems
    contractors have more experience in incinerating each type of chemical
    agent and munition in its stockpile as well as neutralizing bulk nerve
    and blister agents.

    DOD, the Army, CMA, and the systems contractors that operate the
    destruction facilities face a difficult and hazardous task, as well as
    the need to overcome some significant technical challenges, in
    destroying the U.S. stockpile of lethal chemical agents while ensuring
    the safety of workers, the public, and the environment. Recognizing
    these difficulties and the unique program risks, DOD uses a contract
    that reimburses the systems contractors for all reasonable, allowable,
    and allocable costs. To encourage defense contractors to perform in an
    innovative, efficient, and effective way in areas deemed important to
    the program's success, DOD gives its systems contractors the
    opportunity to earn millions of dollars through monetary incentives
    known as award and incentive fees.

    We reported in 2003 that DOD and the Army had not developed an overall
    strategy and risk management approach for the chemical demilitarization
    program to guide the program with a set of principles and a means to
    manage risk, understand its evolution and implementation, and determine
    whether initiatives are achieving their desired results.[Footnote 8]
    The absence of an overall strategy and risk management approach left
    the program without a clear road map to monitor program performance and
    meet the goal of destroying the chemical stockpile in a safe and timely
    manner. We recommended that DOD and the Army adopt practices that
    leading organizations embrace for effectively implementing and managing
    programs, to include promulgating a comprehensive mission statement,
    long-term and annual performance goals, measurable performance
    indicators, and evaluation and corrective action plans. DOD concurred
    with our recommendations and stated that it would establish these
    management tools. A list of related GAO products regarding chemical
    demilitarization and storage and disposal of the chemical weapons
    stockpile is included at the end of this report.

    We also reported that overall DOD programs have paid contractors large
    amounts of award fees on acquisitions that are falling behind schedule,
    overrunning costs, and experiencing significant technical
    problems.[Footnote 9] In addition, we recommended, among other things,
    that DOD instruct the military services to move toward more outcome-
    based award fee criteria that are both achievable and promote
    accountability for acquisition outcomes.[Footnote 10] DOD concurred
    with the recommendation and, in a March 2006 policy memorandum,
    addressed desired outcomes and the role the award fee should play in
    the overall acquisition strategy. The memorandum stated that "while
    award fee contracts are used when it is neither feasible nor effective
    to devise predetermined objective performance targets, it is imperative
    that award fees be tied to identifiable interim outcomes, discrete
    events or milestones, as much as possible."

    This report responds to a request from the House Committee on Armed
    Services' Subcommittee on Terrorism and Unconventional Threats and
    Capabilities that we conduct a review of DOD's Chemical
    Demilitarization Program. Specifically, this report evaluates the (1)
    progress DOD and the Army have made in addressing our prior
    recommendations to strengthen program management, whether this progress
    has been sufficient to correct past program weaknesses, and the
    effectiveness of actions DOD and the Army have taken to identify and
    mitigate the risk of future program schedule extensions and cost
    growth; (2) reasonableness of the methodology used to determine
    schedule milestones; (3) reliability of its cost estimates; and (4)
    effectiveness of DOD's and the Army's use of monetary incentives to
    improve systems contractors' performance.

    To assess the progress DOD and the Army have made in implementing GAO's
    prior recommendations to strengthen program management, whether it has
    been sufficient, and the effectiveness of actions taken to identify and
    mitigate the risk of future program schedule extensions and cost
    growth, we obtained and reviewed strategic and implementation plans and
    interviewed cognizant officials to identify whether key elements were
    present, such as a program mission statement, long-term goals and
    objectives, delineation of roles and responsibilities of DOD and Army
    offices, and near-term performance measures. To assess the
    reasonableness of the methodology used to determine schedule milestones
    as well as the reliability of cost estimates, we reviewed current
    program estimates, destruction schedules, earned value management
    (EVM)[Footnote 11] data, and other documents. We also obtained and
    reviewed the program's risk management plans and related documents. In
    addition, we identified the issues that had caused delays and
    ascertained approaches being used to reduce the potential for delays in
    the future. To analyze the effectiveness of DOD's and the Army's use of
    monetary incentives to improve systems contractors' performance, we
    obtained and reviewed contract documents on the award fees offered to
    the systems contractors, and interviewed DOD and Army officials as well
    as systems contractors' representatives. We obtained and reviewed the
    incentive agreement plan and incentive fees for the Aberdeen Chemical
    Agent Disposal Facility (Aberdeen), Maryland, and Newport Chemical
    Agent Disposal Facility, Newport Chemical Depot (Newport), Indiana, to
    determine the challenges those sites faced in negotiating the incentive
    fee agreements. Also, we interviewed officials with the U.S. Army
    Sustainment Command, CMA, and CMA and systems contractors at Tooele and
    Umatilla. In addition, we met with DOD and Army program officials and
    interviewed officials at two of the chemical agent destruction sites.

    This report focuses primarily on the seven CMA-managed chemical
    destruction sites--Aberdeen, Anniston, Johnston Atoll in the Pacific
    Ocean, Newport, Pine Bluff, Tooele, and Umatilla. We selected these
    sites for our review because most are in the operational phase,
    represent about 82 percent of the remaining stockpile of chemical
    agents to be destroyed, and have reached a level of maturity in
    processing a variety of types of munitions and agents that will help
    meet CWC goals. Two sites--Aberdeen and Johnston Atoll--have already
    completed operations and have either been closed or are being closed.
    This report does not focus on the two chemical agent destruction sites
    that are part of the Assembled Chemical Weapons Alternatives (ACWA)
    program--Blue Grass Chemical Agent-Destruction Pilot Plant, Blue Grass
    Chemical Activity (Blue Grass), Kentucky, and Pueblo Chemical Agent-
    Destruction Pilot Plant, Pueblo Chemical Depot (Pueblo), Colorado--
    because they are currently in the design and construction phase to
    destroy chemical agents, report directly to DOD rather than to the
    Army, and because these facilities are not forecasted to begin
    destruction operations before the CWC deadline, although the stockpiles
    at these two sites are required to be destroyed by the deadline as
    well.

    We examined the reliability of the data used in this report by
    verifying EVM system certification, applying analytical tests to the
    EVM data, and examining EVM system effectiveness in providing
    meaningful performance measurement to program management. We determined
    that the EVM system as used at each site was not reliable, which is
    discussed more fully in the report. We conducted this performance audit
    from May 2006 through July 2007 in accordance with generally accepted
    government auditing standards. A more thorough description of our scope
    and methodology is provided in appendix I.

    Results in Brief:

    While DOD and the Army have taken steps toward addressing our prior
    recommendations to strengthen program management by establishing
    strategic and supporting implementation plans, they have not fully
    developed an integrated approach for measuring performance or for
    managing risk.[Footnote 12] DOD issued a strategic plan in June 2005,
    and CMA issued a strategic plan in July 2005 and a risk management plan
    in May 2006. While these plans identify challenges and contain
    performance measures to monitor progress, we found that the plans lack
    schedule goals for key phases. Neither DOD's 2006 baseline nor its
    April 2006 and April 2007 Selected Acquisition Reports (SAR) to
    Congress include schedule milestones extending beyond 2012. The program
    further lacks annual performance goals linked to long-term strategic
    goals, such as meeting the extended April 29, 2012, CWC deadline.
    Without this linkage, program officials will lack the information on
    interim progress needed to determine whether corrective action is
    needed to safely destroy the stockpile by the treaty deadline.
    Furthermore, while CMA's integrated risk management plan identified a
    process for conducting risk management, the agency has not developed
    the tools needed to implement its approach, such as a risk trade-off
    analysis. For example, the program's plans identified contamination of
    the mustard agent stockpile at one site as a key risk but lacked a
    method for analyzing that risk or managing its potential effects on
    cost and schedule. The later discovery of the extent of the
    contamination delayed planned operations by more than 12 months. Also,
    our analysis showed that DOD's and the Army's processes for identifying
    risk are not integrated across sites or organizations, and CMA's
    identified risks have not been fully aligned with those identified in
    DOD's strategic plan. Consequently, program managers are hampered in
    their information sharing and their progress in safely destroying
    chemical agents, as well as in their ability to systematically evaluate
    and manage risk. Finally, unlike more useful annual performance plans
    we have reviewed, which include historical performance trends and
    multiyear future goals, CMA's annual performance plan has a single-year
    focus. We are recommending that the Secretary of the Army incorporate
    historical trends and multiyear future goals in its annual performance
    plan; develop interim destruction goals, approaches, and milestones;
    and establish a time frame for implementing its risk management
    approach, including integration across sites and with DOD.

    Recently achieved processing rates have considerably exceeded
    forecasted rates at the operational sites and indicate that DOD's
    revised schedule milestones may be too conservative to be a realistic
    measure. DOD extended its Chemical Demilitarization Program schedule in
    2005 in response to the slower actual destruction rates experienced
    than had been estimated when more optimistic milestones were adopted in
    2003. As we have previously reported,[Footnote 13] best practices for
    schedule development call for assumptions that are both up-to-date and
    reasonable. Using a new schedule methodology, and as a result of
    factoring in slower processing rates and greater risk, CMA extended
    schedules at each site between 62 months and 80 months beyond the
    baselines established in 2003. The extensions prompted budget and
    acquisition reviews, including an independent cost and schedule
    assessment, which in turn led DOD to further extend the program's
    schedule milestones for completing destruction operations and closure
    activities. Our analysis of recently achieved processing rates showed
    that at the time of our review, the sites were between 2 months and 39
    months ahead of the revised estimates for munitions destruction.
    Revised schedule estimates published in the April 2007 SAR further
    reflect processing rates that considerably exceed forecasts.
    Nevertheless, DOD has no current plans to reassess the program schedule
    milestones, as such reassessments are usually prompted by schedule
    slippages. Higher destruction rates than scheduled indicate that an
    independent reassessment of the program's scheduled milestones may be
    warranted, thus assuring DOD and Congress of the reasonableness of the
    baseline against which its progress is being measured and of its costs
    as the current schedule is not providing DOD and Congress with a
    reasonable baseline against which to measure progress and cost. We are
    recommending that the Secretary of Defense direct the Cost Analysis
    Improvement Group to assess the reasonableness of schedule milestones,
    and that DOD periodically review the program's processing rates to
    determine whether the milestones are still reasonable and include
    schedule milestones for completing destruction operations and facility
    closure activities in the program's baseline and acquisition report.

    Shortcomings in the underlying data used in the program's EVM showed
    that the cost estimate is unreliable. Our review showed that
    performance baselines lacked validity and had not been stable because
    all requirements, such as closure costs, are not fully identified and
    included in program cost estimates and because large amounts of
    additional costs are added to the performance baseline annually. In
    addition, systems contractors' cost estimates were significantly lower
    than the Army's cost estimates, due largely to the Army's expectations
    of longer operational periods and greater closure requirements. The
    program's cost increases are largely attributable to extended schedules
    and facility closure estimates. Best practices indicate that cost
    estimates should be comprehensive, and DOD guidance further reflects
    the need for valid and stable performance baselines from which to make
    projections and perform trend analysis. DOD and Congress currently lack
    accurate estimates for budgeting for and funding all activities through
    facility closure. Similarly, given the significant differences between
    the contractors' and Army's cost estimates, an independent cost review
    would provide DOD and Congress assurance that cost estimates are
    realistic. We are recommending that the Army define closure
    requirements and develop accurate and realistic closure cost estimates;
    define all contract requirements, including authorized unpriced work
    associated with facility closure activities, and establish a time frame
    for placing these requirements on contract; and that an independent
    review of the cost estimates be done and linked with reasonable
    schedule milestones.

    According to the Army and based on our analysis, award fees--the
    predominant fees--offered to contractors have thus far not
    satisfactorily resulted in the control of schedule performance outcomes
    and cost growth. Army officials indicate that the majority of the award
    fee incentives are dedicated to achieving high standards for safety,
    surety, and environmental compliance in recognition of the lethal
    nature of the chemical agent stockpile and the congressional mandate
    for maximum protection of the workforce, the public, and the
    environment. The Army has actively reviewed and adjusted criteria for
    determining award fees it has provided to the systems contractors to
    incentivize aspects of their performance. Based on our analysis, from
    2003 through 2006, the award fee plans have heavily emphasized safety
    and environmental compliance and did not have a separate performance
    measure for schedule until 2006. Army officials acknowledged that award
    fees have not satisfactorily resulted in the control of schedule and
    cost growth. In response to our previous recommendations, in March
    2006, DOD issued policy guidance directing that award fees be tied to
    identifiable interim outcomes, discrete events, or milestones as much
    as possible. CMA issued its award fee policy in September 2006, and our
    review of this policy indicated that CMA did not clearly or fully
    incorporate the concept of tying "award fees to identifiable interim
    outcomes, discrete events or milestones as much as possible," as
    directed in the March 2006 DOD guidance. In addition, the current award
    fee plan for the five chemical demilitarization sites we reviewed
    allows government site project managers flexibility in weighting each
    performance measure within a set range. We found that managers have
    selected the high end of the range for management, and the low end of
    the range for schedule and cost, when determining award fee
    percentages. Moreover, many performance measures in the award fee plans
    contain subjective award fee criteria, and award fees for the most part
    are not linked to objective or measurable criteria. Our analysis showed
    that it is possible for CMA to further link award fee criteria in this
    program to objective or measurable outcomes in at least three
    performance measures. Given the maturity of the operations phase at the
    incineration sites and Newport, although not legally required, we
    identified opportunities, from a management perspective, to apply more
    weight to performance measures, such as schedule and cost, to better
    align CMA's award fee policy with DOD's policy, and to make greater use
    of objective, measurable criteria in award fee plans to increase the
    likelihood of meeting the April 29, 2012, treaty deadline. Two recently
    established and planned performance-based incentive fees for the
    incineration sites are focused on schedule. One fee was established in
    2005 to encourage and reward collaboration among the incineration site
    systems contractors with the goal of improving the overall program
    schedule and cost performance. This fee is awarded based on sites'
    collective performance and, in fiscal year 2007, this incentive fee
    focused mostly on schedule performance associated with achieving the
    CWC treaty deadline. The other planned incentive fee, as authorized by
    Congress, is structured to accelerate destruction operations and
    facility closure at the incineration sites. We could not determine how
    effective these incentives might be due to the collective manner in
    which the fee to encourage and reward collaboration is earned, its
    relative size, and recent focus on objective schedule performance, and
    because the other planned incentive fee has not yet been implemented.
    We are recommending that the Army (1) determine whether greater weight
    can be applied to key performance measures, such as schedule and cost,
    in award fee plans; (2) better align its award fee policy with DOD's
    March 2006 guidance on award fees; and (3) link award fee criteria, as
    much as possible, to performance measures that focus on identifiable
    interim outcomes, discrete events, or milestones.

    We provided a draft of this report to DOD in October 2007 for its
    review and comment. In written comments on a draft of this report, DOD
    concurred or partially concurred with 12 of our 13 recommendations. DOD
    did not concur with our recommendation regarding defining closure
    requirements and developing accurate and realistic closure cost
    estimates for each of DOD's chemical demilitarization sites. DOD
    recommended that we delete it, since DOD officials believe that a
    subsequent recommendation defining all contract requirements, including
    those associated with facility closure activities, was more
    encompassing. Based on DOD's comments, rather than deleting the
    recommendation, we clarified it and the related recommendation to more
    clearly distinguish the difference between the two recommendations, and
    consolidated it with our two other recommendations related to defining
    contract requirements. DOD also stated that the department will
    continue to use acquisition management tools and discipline to ensure
    the destruction of the United States' chemical weapons stockpile in a
    safe and secure manner, while being economical and meeting the U.S.
    commitments under the CWC. DOD identified a number of actions that it
    already has initiated in response to our recommendations. DOD also
    provided us with technical comments, which we incorporated in the
    report as appropriate. DOD's response is reprinted in appendix III.

    Background:

    In 1985, Congress directed DOD to destroy the U.S. stockpile of lethal
    chemical agents and munitions in the United States.[Footnote 14] DOD
    designated the Department of the Army as its executive agent for the
    program, and the Army established the Chemical Demilitarization
    Program, which was charged with the destruction of the original
    stockpile at nine storage sites. Figure 1 shows the locations of the
    nine storage sites; the number of items, such as projectiles, rockets,
    mortars, land mines, and bombs, originally stored at each site; and the
    number of items remaining to be destroyed as of September 30, 2007.

    Figure 1: U.S. Stockpile of Chemical Agents and Munitions:

    This figure is a map showing stockpile of chemical agents and
    munitions.

    Umatilla Chemical Depot, Oregon;
    Number of items: 220,603;
    Remaining items: 65,060.

    Deseret Chemical Depot, Utah;
    Number of items: 1,138,785;
    Remaining items: 122,361.

    Newport Chemical Depot, Indiana;
    Number of items: 1,690;
    Remaining items: 579.

    Aberdeen Proving Ground, Maryland;
    Number of items: 1,817;
    Remaining items: 0.

    Blue Grass Army Depot, Kentucky;
    Number of items: 101,764;
    Remaining items: 101,764.

    Anniston Army Depot, Alabama;
    Number of items: 661,531;
    Remaining items: 446,712.

    Pine Bluff Arsenal, Arkansas;
    Number of items: 123,095;
    Remaining items: 32,684.

    Pueblo Chemical Depot, Colorado;
    Number of items: 780,078;
    Remaining items: 780,078.

    Johnston Atoll, Pacific Ocean;
    Number of items: 412,798;
    Remaining items: 60.

    [See PDF for image]

    Source: GAO analysis of CMA data.

    [End of figure]

    The remaining U.S. stockpile, as of June 2007, consists of two nerve
    agents--GB and VX[Footnote 15]--and a blister agent--mustard[Footnote
    16]--which currently are stored at seven sites in the continental
    United States. Incineration was initially selected as the method to
    destroy the stockpile at all nine sites, and destruction began in 1990
    at Johnston Atoll. This location was selected to be the operational
    prototype facility to test the destruction process prior to it being
    conducted in the continental United States. Figure 2 shows an example
    of a chemical agent disposal facility like those used at the four
    operational incineration sites (Anniston, Pine Bluff, Tooele, and
    Umatilla).

    Figure 2: Umatilla Chemical Agent Disposal Facility, Umatilla Chemical
    Depot, Oregon:

    This figure is a photograph of Umatilla Chemical Agent Disposal
    Facility, in Umatilla Chemical Depot, Oregon.

    [See PDF for image]

    Source: Umatilla Chemical Agent Disposal Facility.

    [End of figure]

    Subsequent to the decision to use incineration to destroy the remaining
    stockpile, Congress directed DOD to evaluate the possibility of using
    alternative technologies to incineration.[Footnote 17] In 1994, the
    Army initiated a project to develop nonincineration technologies for
    use at the two bulk-agent-only sites at Aberdeen and Newport. These
    sites were selected in part because their stockpiles were relatively
    simple--each site had only one type of agent and this agent was stored
    in bulk-agent (ton) containers, which are used to store various
    chemical agents at the chemical agent destruction sites.[Footnote 18]
    Figure 3 shows a ton container being transported to the processing
    building at Newport.

    Figure 3: Delivery of a Ton Container to the Processing Building at the
    Newport Chemical Agent Disposal Facility, Newport Chemical Depot,
    Indiana:

    This figure is a photograph of the delivery of a ton container to the
    processing building at Newport Chemical Agent Disposal Facility, in
    Newport, Indiana.

    [See PDF for image]

    Source: Newport Chemical Agent Disposal Facility.

    [End of figure]

    In 1997, DOD approved pilot testing of a neutralization technology at
    these two sites. Figure 4 shows the type of hydrolysate-filled
    intermodal storage containers used at Newport. The hydrolysate is
    awaiting final disposition.[Footnote 19]

    Figure 4: Intermodal Storage Containers at the Newport Chemical Agent
    Disposal Facility, Newport Chemical Depot, Indiana:

    This figure is a photograph of intermodal storage containers at the
    Newport Chemical Agent Disposal Facility in Newport Chemical Depot,
    Indiana.

    [See PDF for image]

    Source: Newport Chemical Agent Disposal Facility.

    [End of figure]

    Also in 1996, Congress directed DOD to evaluate the use of alternative
    technologies and suspended incineration planning activities at Blue
    Grass and Pueblo, two sites with assembled weapons.[Footnote 20]
    Furthermore, Congress directed that these two sites be managed in a
    program independent of the Army's Chemical Demilitarization Program and
    report to DOD directly instead of the Army.[Footnote 21] Thus, the ACWA
    program was established.

    Table 1 shows DOD's nine chemical agent stockpile sites; the primary
    destruction method used (i.e., incineration or neutralization) and the
    secondary disposal method used for the treatment of the resulting
    wastewater, if applicable; the types of munitions destroyed, such as
    projectiles, rockets, mortars, land mines, and bombs in addition to ton
    containers; the types of agents (GB, VX, and mustard) and their status
    (e.g., destroyed, ongoing destruction, or planned destruction); the
    original agent tonnage; the tonnage destroyed; the percentage of the
    stockpile destroyed; the remaining agent tonnage; and the status of
    each facility (closed, operational, or under construction) as of
    September 30, 2007.

    Table 1: Status of DOD's Chemical Agent Stockpile Sites and Chemical
    Agent Destroyed as of September 30, 2007:

    Site: Johnston Atoll;
    Destruction method: Incineration;
    Munition types destroyed: Projectiles, land mines, mortars, and ton
    containers;
    Type of agent and status[A]: GB, VX, and mustard (destroyed);
    Original agent tonnage: 705[B];
    Tonnage destroyed: 705;
    Percentage of stockpile destroyed: 100;
    Remaining agent tonnage (as of September 2007): 0;
    Facility status: Closed.

    Site: Tooele;
    Destruction method: Incineration;
    Munition types destroyed: Ton containers, mines, rockets, projectiles,
    spray tanks, and mortars;
    Type of agent and status[A]: GB and VX (destroyed) and mustard
    (ongoing);
    Original agent tonnage: 13,617;
    Tonnage destroyed: 9,292;
    Percentage of stockpile destroyed: 68;
    Remaining agent tonnage (as of September 2007): 4,325;
    Facility status: Operational.

    Site: Anniston;
    Destruction method: Incineration;
    Munition types destroyed: Rockets and projectiles;
    Type of agent and status[A]: GB (destroyed), VX (ongoing), and mustard
    (planned);
    Original agent tonnage: 2,254;
    Tonnage destroyed: 726;
    Percentage of stockpile destroyed: 32;
    Remaining agent tonnage (as of September 2007): 1,528;
    Facility status: Operational.

    Site: Umatilla;
    Destruction method: Incineration;
    Munition types destroyed: Rockets, bombs, and projectiles;
    Type of agent and status[A]: GB (ongoing) and VX and mustard (planned);
    Original agent tonnage: 3,719;
    Tonnage destroyed: 1,016;
    Percentage of stockpile destroyed: 27;
    Remaining agent tonnage (as of September 2007): 2,703;
    Facility status: Operational.

    Site: Pine Bluff;
    Destruction method: Incineration;
    Munition types destroyed: Rockets and ton containers;
    Type of agent and status[A]: GB (destroyed) and VX and mustard
    (planned);
    Original agent tonnage: 3,850;
    Tonnage destroyed: 484;
    Percentage of stockpile destroyed: 12;
    Remaining agent tonnage (as of September 2007): 3,366;
    Facility status: Operational.

    Site: Aberdeen;
    Destruction method: Neutralization and off-site treatment of
    wastewater;
    Munition types destroyed: Ton containers;
    Type of agent and status[A]: Mustard (destroyed) and off-site
    wastewater treatment (complete);
    Original agent tonnage: 1,622;
    Tonnage destroyed: 1,622;
    Percentage of stockpile destroyed: 100;
    Remaining agent tonnage (as of September 2007): 0;
    Facility status: Closed.

    Site: Newport;
    Destruction method: Neutralization and off-site treatment of
    wastewater;
    Munition types destroyed: Ton containers;
    Type of agent and status[A]: VX (ongoing) and off-site wastewater
    treatment (ongoing)[C];
    Original agent tonnage: 1,269;
    Tonnage destroyed: 587;
    Percentage of stockpile destroyed: 46;
    Remaining agent tonnage (as of September 2007): 682;
    Facility status: Operational.

    Site: Pueblo;
    Destruction method: Neutralization and on-site biotreatment of
    wastewater;
    Munition types destroyed: Projectiles and mortars;
    Type of agent and status[A]: Mustard (planned);
    Original agent tonnage: 2,611;
    Tonnage destroyed: 0;
    Percentage of stockpile destroyed: 0;
    Remaining agent tonnage (as of September 2007): 2,611;
    Facility status: Destruction pilot plant under construction.

    Site: Blue Grass;
    Destruction method: Neutralization and on-site water oxidation of
    wastewater;
    Munition types destroyed: Rockets and projectiles;
    Type of agent and status[A]: GB, VX, and mustard (planned);
    Original agent tonnage: 523;
    Tonnage destroyed: 0;
    Percentage of stockpile destroyed: 0;
    Remaining agent tonnage (as of September 2007): 523;
    Facility status: Destruction pilot plant under construction.

    Total;
    Destruction method: [Empty];
    Munition types destroyed: [Empty];
    Type of agent and status[A]: [Empty];

    Original agent tonnage: 30,170;
    Tonnage destroyed: 14,432;
    Percentage of stockpile destroyed: 48;
    Remaining agent tonnage (as of September 2007): 15,738;
    Facility status: [Empty].

    Source: DOD.

    Note: The Non-Stockpile Chemical Materiel Project is the CMA
    organization responsible for all chemical warfare materiel that resides
    outside of a declared stockpile. This materiel consists of chemical
    agent identification sets, which consist of glass ampoules, vials and
    bottles of chemical agent, and actual chemical weapons. The project
    provides centralized management and direction to DOD for the disposal
    of nonstockpile chemical warfare materiel in a safe, environmentally
    sound, and cost-effective manner. It is responsible for destroying the
    nerve agents GB and VX in projectiles, vials, bottles, glass ampoules,
    and ton containers, As of June 2007, the Non-Stockpile Chemical
    Materiel Project had destroyed 563 tons of chemical agent, which is
    counted toward meeting the CWC 45 percent destruction milestone.

    [A] The stockpile includes two nerve agents (GB and VX) and a blister
    agent (mustard).

    [B] The site at Johnston Atoll destroyed 1,326 tons of agent before the
    CWC entry into force in 1997. Overall, the site at Johnston Atoll has
    destroyed a total of 2,031 tons of agent.

    [C] On January 5, 2007, CMA began shipping Newport VX hydrolysate to a
    commercial wastewater treatment facility in Port Arthur, Texas. Because
    the VX hydrolysate is being transported to Texas for final treatment,
    the United States receives credit for destruction of the Newport
    stockpile under CWC. Of an estimated 1.8 million gallons of VX
    hydrolysate at Newport, as of June 2007, approximately 33 percent has
    already been shipped to Texas for incineration.

    [End of table]

    In 1997, the United States ratified the CWC, which prohibits the use of
    these weapons and mandates the elimination of existing stockpiles by
    April 29, 2012, which represents a 5-year extension from the initial
    deadline.[Footnote 22] The CWC also contains a series of interim
    deadlines applicable to the U.S. stockpile[Footnote 23] for destroying
    1 percent, 20 percent, and 45 percent of the stockpile. The United
    States met the 1 percent and 20 percent destruction deadlines, but did
    not meet the original 45 percent destruction deadline. The United
    States received an extension from April 2004 to December 2007 for the
    45 percent deadline. As of June 2007, program officials reported that
    45 percent of the U.S stockpile had been destroyed, 6 months earlier
    than the interim CWC deadline for destroying that amount, and the
    entire stockpile at two of the nine sites has been eliminated.

    DOD designated the Chemical Demilitarization Program as a major
    acquisition program.[Footnote 24] As such, the overall program cost and
    schedule--and its milestones for major phases of the program--are
    identified in the APB and are reported to Congress in the SAR.
    Additionally, the Army, through its CMA, developed program estimates
    that include schedule milestones to support its day-to-day management
    responsibilities of the program. Finally, systems contractors that
    operate the chemical agent destruction facilities have established
    schedule milestones for completing major phases identified in their
    individual contracts with the Army.

    From 1998 through 2005, the total program cost estimates increased by
    113 percent, resulting in the program breaching approved program
    baselines in 2001 and 2005. As a result, the program was required to
    certify to Congress in each instance that new cost estimates were
    reasonable and that the management structure is adequate to manage and
    control costs. The bulk of these program costs will go to systems
    contractors that construct and operate the facilities that destroy the
    chemical weapons.

    DOD and the Army Have Taken Steps to Strengthen Program Management, but
    Have Not Fully Developed an Integrated Approach for Measuring
    Performance and Managing Risk:

    DOD and the Army have taken steps in addressing our prior
    recommendations to strengthen program management by establishing an
    overall strategy and supporting implementation plans, but some key
    elements related to performance measures and risk management are not
    fully developed or implemented. DOD and the Army have identified a
    number of risks across the program that could affect the program's cost
    and schedule, and CMA has developed a risk management plan, but the
    program's risk management approach has not been effective because it
    has not been fully implemented or integrated across the program.
    Moreover, while DOD and the Army have established key program goals,
    performance measures have not always been linked to these goals.

    DOD Addressed GAO's Prior Recommendations by Establishing Strategic and
    Supporting Implementation Plans:

    DOD has taken a number of actions to address our past recommendations
    that it establish strategic and supporting implementation plans. We
    recommended in 2003 that DOD and the Army develop an overall strategy
    and implementation plan for the chemical demilitarization program that
    would articulate a program mission statement, identify the program's
    long-term goals and objectives, delineate the roles and
    responsibilities of all DOD and Army offices, and establish near-term
    performance measures.[Footnote 25] We also recommended that DOD and the
    Army implement a risk management approach that anticipates and
    influences internal and external factors that could adversely affect
    program performance. DOD concurred with our recommendations and has
    strengthened program management by establishing an overall strategy and
    supporting implementation plans.

    In response to our 2003 recommendations, DOD and the Army have
    developed various strategic and supporting plans. For example, in June
    2005, DOD published the Strategic Plan for Destruction of Lethal
    Chemical Agents and Munitions.[Footnote 26] The purpose of this plan is
    to serve as an overarching plan for destruction of the U.S. stockpile
    of lethal chemical agents and munitions and nonstockpile items. It
    applies to both portions of the program implemented by CMA as well as
    the ACWA program. (In this report, we only address the parts of the
    plan pertaining to the CMA stockpile program.) This plan included many
    of the elements that we recommended, such as a program mission
    statement to enhance national security by eliminating chemical warfare
    munitions while protecting the workforce, the public, and the
    environment and meeting requirements specified in the CWC. It also
    included overall goals and objectives and performance measures.

    Also, in July 2005, CMA published the U.S. Army's Chemical Materials
    Agency Strategic Plan (2005-2010). This plan encompasses strategies for
    all CMA organizational elements and programs--including the chemical
    stockpile storage program, the Non-Stockpile Chemical Materiel Project,
    the Elimination of Chemical Weapons program, and the Pine Bluff Arsenal
    industrial base--together with those of the AWCA program. According to
    the CMA strategic plan, its purpose is to be a road map for the safe
    elimination of the U.S. chemical weapons stockpile. This strategic plan
    articulated program goals and objectives and also described a number of
    supporting implementation plans that we recommended should be
    developed, including annual performance plans that would provide
    performance measures and an integrated risk management plan that would
    address the program's challenges.

    In its July 2005 strategic plan, CMA identified 11 supporting plans
    that were directly or indirectly related to the stockpile program.
    According to CMA officials, 9 of the 11 supporting plans had been
    developed, as of July 2007. These plans include the CMA resource
    management plan, a risk management plan, and a risk management
    directorate plan.

    The DOD strategic plan articulated four key goals for the Chemical
    Demilitarization Program: (1) eliminate the stockpile while protecting
    the workforce, the public, and the environment, and comply with U.S
    obligations under the CWC; (2) employ sound business practices to
    ensure efficient life cycle management of program resources; (3)
    institute a program of continuous improvement regarding safety,
    environment protection, efficient operations and maintenance, and
    facility closure to mitigate risks and to ensure compliance with
    statutory, regulatory, and policy requirements and decisions; and (4)
    maintain communications with the public and local and national
    officials. Performance objectives with the associated performance
    metrics were developed to address these goals. Table 2 contains some
    key objectives to meet goals contained in the DOD and CMA strategic
    plans.

    Table 2: Comparison of Some Key Destruction Objectives Identified in
    DOD's Strategic Plan and CMA's Strategic Plan:

    DOD's Strategic Plan: Achieve CWC extended 45 percent destruction
    milestone by December 31, 2007, while ensuring the safety of the
    workers, the public, and the environment;
    CMA's Strategic Plan: Destroy the U.S. stockpile of lethal chemical
    agents and munitions while meeting the CWC deadlines and containing
    total life cycle cost by using an integrated risk management approach.

    DOD's Strategic Plan: Achieve CWC extended 100 percent destruction
    deadline by April 29, 2012, while ensuring the safety of the workers,
    the public, and the environment;
    CMA's Strategic Plan: No objective linked to meeting CWC extended
    deadline by April 29, 2012.

    DOD's Strategic Plan: Manage life cycle costs to the program within
    fiscal resources and consistent with the APB;
    CMA's Strategic Plan: No objective linked to meeting the APB.

    DOD's Strategic Plan: Develop and institute a risk management culture
    that ensures that no chemical agent releases, exposures, and accidents
    occur across the entire demilitarization community;
    CMA's Strategic Plan: Meet and maintain Army, state, and federal safety
    and security standards and environmental laws, regulations, and permit
    conditions.

    Source: GAO analysis of DOD and CMA data.

    [End of table]

    CMA also jointly issued the program's Fiscal Year 2007 Annual
    Performance Plan in September 2006.[Footnote 27] The performance plan
    contains, among other things, annual goals for meeting the CWC's
    December 2007 interim destruction deadline. However, Tooele is the only
    site that has approved annual plans, which it is implementing. CMA
    officials stated that they will request annual performance plans from
    all of the sites for fiscal year 2008. Moreover, the plans lacked some
    attributes that would make them more useful to decision makers and to
    Congress. We have previously found that performance plans can be more
    effective if they (1) use intermediate goals and measures to show
    progress or contribution to intended results, particularly if it may
    take years to meet the intended results; (2) show baseline and trend
    data for past performance; and (3) permit an agency to convey what it
    expects to achieve in the long term by including multiyear performance
    goals in its performance plan.[Footnote 28] With baseline and trend
    data, the more useful performance plans provided a context for drawing
    conclusions about whether performance goals are reasonable and
    appropriate. By including multiyear performance goals, a performance
    plan can provide congressional and other decision makers with an
    indication of the incremental progress the agency expects to make in
    achieving results. CMA's performance plan only contains goals related
    to the specified fiscal year and does not show previous years' trends
    or show how achievement of the specified goals would contribute to the
    program's long-term goal.

    Risk Management Approach Is Not Fully Developed or Integrated, and Some
    Key Program Goals Lack Clearly Linked Performance Measures:

    DOD has partially implemented our recommendations to establish
    performance measures and implement a risk management approach that
    anticipates and influences internal and external factors that could
    adversely affect program performance. Specifically, CMA has not yet
    completed all the risk management tools identified in its risk
    management plan or integrated its risk management efforts across sites
    and organizations. Further, while DOD did establish performance
    measures as we recommended, some key program goals still lack clearly
    linked performance measures. Moreover, the program is using four
    different performance measurement approaches and measurements that are
    not well-aligned with one another. Without annual performance goals and
    measures linked with long-term strategic goals, program officials will
    lack information on the interim progress being achieved and whether
    corrective action is needed to safely destroy the stockpile by the
    treaty deadline.

    CMA Has Made Progress in Developing an Integrated Risk Management
    Approach but Has Not Completed Its Risk Management Tools and
    Integration Efforts:

    As identified in its U.S. Army Chemical Materials Agency Integrated
    Risk Management Plan, CMA has developed a process to identify program-
    level risks and reports them as part of the Army's Probability of
    Program Success Report.[Footnote 29] The program office, in conjunction
    with the site project managers, also has developed a "risk landscape,"
    which is used to capture risks that could affect the operating and
    closure schedules. However, no programwide approach exists to integrate
    the risk management efforts at the site, CMA, and DOD levels to ensure
    that all risks are being addressed in an integrated manner. For
    example, CMA officials acknowledged that risks that are included in the
    Probability of Program Success Reports and the site risk landscapes
    were developed independently of each other and thus may not be fully
    integrated. Also, while the sites were developing risk plans that are
    at different stages of maturity, CMA has not yet developed a single set
    of risk management procedures to be used by the site-level project
    managers and site systems contractors when developing these plans.
    Lastly, the risks identified in DOD plans and those identified by CMA
    were not identified as part of an integrated process. CMA officials
    stated that the risks it identified are an aggregate of some site risks
    and CMA program-level risks, whereas the risk identified by DOD in its
    strategic plan represents a general analysis of the major factors that
    the senior leadership has selected as appropriate for long-term
    attention. CMA officials stated that CMA is working to establish a
    formal process for aligning risks among all planning and reporting
    documents.

    Other steps that are needed to fully implement this approach have not
    been completed. For instance, CMA identified other tools to aid in
    implementing an overarching risk management program to complete the
    steps in this process, but these tools have not been fully developed.
    These tools include a risk trade-off analysis and a risk management
    information system database. A risk trade-off analysis is normally
    included in this risk planning step. According to CMA officials, while
    risk trade-off analyses are discussed in their risk management plan,
    the program has not formally used this analysis in assessing program
    risk, and thus this analysis has not been available to help decision
    makers consider how to address program challenges. For example, while
    the condition of the stockpile of mustard agent at Tooele was
    identified as a key program risk, a process was not in place to perform
    a comprehensive analysis to consider the risks to the program's cost
    and schedule if assumptions about the level of mercury contamination
    were understated. Because the extent of the problem was not discovered
    sooner, needed facility modifications were delayed, extending the
    planned operations schedule by 12 months.

    CMA has also not completed the development of its integrated risk
    information management system, which would involve the collection,
    documentation, and reporting of risk data in support of the steps in
    the overall risk management process. According to CMA, an initial
    requirement analysis for the development of an integrated risk
    information system determined that initially considered in-house
    solutions would require considerable additional development. A market
    survey will be conducted to consider external solutions. While a
    comprehensive risk management approach could manage risks, and CMA is
    still in the process of developing such an approach, CMA has not
    established time frames for the completion and implementation of its
    risk management approach. Without clear expectations for fully
    developing and implementing its risk management approach within a set
    time frame, CMA could continue to be limited in its efforts to control
    some of the issues that have been affecting the program.

    Some Key Program Goals Lack Clearly Linked Performance Measures:

    While DOD and the Army have made progress in establishing program
    goals, we found that some program goals were not linked to one another
    or to key performance measures, and making such linkages is a practice
    of leading organizations.[Footnote 30] For example, in its fiscal year
    2007 annual performance plan, CMA had developed an annual performance
    measure linked to the interim CWC destruction goal of 45 percent by
    December 2007. However, neither DOD nor CMA nor the Army has developed
    annual performance measures that are linked to achieving the ultimate
    goal of destroying the total stockpile at each remaining site by the
    April 29, 2012, treaty deadline. Additionally, while the Army has
    recently established annual site goals for chemical agent tons
    destroyed, these targets have been linked either to Army or DOD
    schedules that far exceed the extended April 29, 2012, treaty deadline.
    Without this linkage, it is difficult for program officials to track
    the progress that the program is making toward its goals, and program
    officials have little assurance that program activities are linked to
    program goals.

    The program is using four different performance measurement approaches:
    the SAR, the Office of Management and Budget's (OMB) Program Assessment
    Rating Tool (PART),[Footnote 31] and DOD's and CMA's strategic plans.
    As this is a major acquisition program, some performance measures
    related to cost and schedule are contained in the SAR. However, while
    DOD and CMA officials cited the schedule milestones in the APB as the
    schedule the program is accountable to, neither the 2006 approved APB,
    the April 2006 SAR, nor the April 2007 SAR contained schedule milestone
    dates beyond April 2012, but rather stated that schedule milestones are
    "to be determined." DOD officials stated that the dates are not
    included in the APB because it was approved about 1 week before DOD
    officially announced to Congress that all sites would not meet the
    extended April 29, 2012, treaty deadline. Although 18 months (as of
    October 2007) have elapsed since DOD made that announcement, the dates
    have still not been included in these documents. DOD and CMA officials
    stated that the milestone dates are still being used for performance
    tracking despite not being included in the most recent SARs. While the
    program may have a measurement for tracking progress, its exclusion
    from the SAR limits program oversight by Congress.

    The program also developed additional performance measures beyond those
    used in the acquisition reporting system. For example, the program
    developed performance measures in (1) the Office of the Secretary of
    Defense's (OSD) strategic plan, (2) OMB's PART, and (3) CMA's strategic
    plan. However, these measures have not always been linked to each other
    and have not always been linked to key program goals. For example,
    while the achievement of destruction of the stockpile by the treaty
    deadline is an overall program goal in both the DOD and CMA strategic
    planning documents, neither DOD nor CMA has developed annual
    performance measures that are linked to achieving that goal even though
    they had established goals linked to the interim 45 percent destruction
    goal.

    Leading organizations seek to establish clear hierarchies of
    performance goals and measures and link these goals and measures for
    each successive organizational level to the overall program
    goal.[Footnote 32] Our analysis of the various performance measures
    also determined that they were not always linked to one another or tied
    to key program goals. For example, while OSD identified 14 performance
    measures that support the four goals contained in the strategic plan
    that directly relate to the Chemical Demilitarization Program's
    stockpile program, CMA's performance measures are not always aligned
    with OSD's performance measures. Also, while OSD identified performance
    measures that are similar to those tracked by OMB in the PART, two
    performance measures in the PART are not included in OSD's strategic
    plan. These two measures were the planned cost per ton of agent
    destroyed against actual cost and the statistical confidence of meeting
    the 45 percent CWC deadline. The SAR contained performance measures
    related to safety and environmental performance, such as those related
    to chemical exposures and releases, but did not have performance
    measures related CWC tonnage goals. Table 3 compares measures
    identified in the PART, the OSD strategic plan, the CMA strategic plan,
    and the SAR.

    Table 3: Comparison of Performance Measures Identified in the PART, OSD
    Strategic Plan, CMA Strategic Plan, and Selected Acquisition Report:

    OMB PART measures: Number of chemical agent exposures;
    OSD strategic plan: No chemical agent exposures as defined in the APB;
    CMA strategic plan: Number of chemical exposures as defined in the APB;
    SAR (CMA and Newport): Number of chemical agent exposures.

    OMB PART measures: Number of chemical agent releases;
    OSD strategic plan: No chemical releases as defined in the APB;
    CMA strategic plan: Number of chemical agent releases as defined in the
    APB;
    SAR (CMA and Newport): Number of chemical agent releases.

    OMB PART measures: Tons of chemical agent destroyed against CWC 45
    percent destruction deadline;
    OSD strategic plan: Percentage of actual amount of stockpiled chemical
    materiel destroyed against CWC 45 percent destruction deadline;
    CMA strategic plan: No tonnage destroyed performance measure against
    CWC 45 percent destruction deadline;
    SAR (CMA and Newport): No CWC tonnage destroyed performance measure.

    OMB PART measures: Tons of chemical agent destroyed against CWC 100
    percent destruction deadline;
    OSD strategic plan: Percentage of actual amount of stockpiled chemical
    materiel destroyed against CWC 100 percent destruction deadline;
    CMA strategic plan: No tonnage destroyed performance measure against
    CWC 100 percent destruction deadline;
    SAR (CMA and Newport): No CWC tonnage destroyed performance measure.

    OMB PART measures: Recordable incidence rate;
    OSD strategic plan: Recordable incidence rate;
    CMA strategic plan: Recordable incidence rate;
    SAR (CMA and Newport): Meet Army, state, and/or federal requirements.

    OMB PART measures: Percentage of stockpile destroyed against CWC 45
    percent destruction deadline;
    OSD strategic plan: No related measure;
    CMA strategic plan: Statistical confidence in achieving CWC 45 percent
    destruction deadline;
    SAR (CMA and Newport): No related measure.

    OMB PART measures: Planned cost per ton of agent destroyed against
    actual cost;
    OSD strategic plan: No related measure;
    CMA strategic plan: Plan cost per ton of agent destroyed against actual
    cost;
    SAR (CMA and Newport): No related measure.

    Source: GAO analysis of DOD and CMA data.

    [End of table]

    This lack of integration between key program goals and performance
    measures hampers program officials' ability to effectively track the
    progress that the program is making because the performance plan lacks
    destruction goals, approaches, and milestones that are directly linked
    to overall program goals for safely meeting CWC deadlines.

    Recently Improved Destruction Rates for Chemical Munitions May Indicate
    Overly Conservative Schedule Milestones at the Four Incineration Sites:

    Since DOD extended the program office schedules and DOD schedule
    milestones in 2005, actual processing rates have considerably exceeded
    the forecasted rates at the four incineration sites, indicating that
    schedule milestones being used to measure the program's progress may be
    too conservative to be realistic. DOD's Chemical Demilitarization
    Program schedule has been extended in recent years largely because of
    slower-than-expected processing rates for destroying chemical agent-
    filled munitions. Program office schedules were extended in 2005
    because of a greater recognition of slower processing rates for
    destroying chemical munitions than schedule milestones adopted in the
    more optimistic 2003 APB. In addition, the program office employed a
    new schedule forecasting methodology that also contributed to the
    schedule extensions because it accounted for more equipment downtime
    and for the risks that have delayed processing in the past. The
    schedule extensions exceeded the existing 2003 baseline and led DOD to
    adopt new schedule milestones in 2006. Finally, program confidence
    levels for meeting the April 29, 2012, CWC milestone are low.

    Slower-Than-Anticipated Processing Rates and Time Frames Led to
    Schedule Extensions:

    The program office schedules have been extended since 2003 largely
    because forecasted processing rates for destroying chemical agent-
    filled munitions were reduced based on the program's analysis of
    historical processing rates being slower than anticipated. Our review
    of best practices indicates that estimating assumptions should be
    realistic and backed by historical data to minimize uncertainty and
    risk.[Footnote 33] Best practices also stipulate that estimates are
    accurate when they are not overly conservative or optimistic. Best
    practices also recommend that when underlying data or other assumptions
    change, cost and schedule estimates should be revised to reflect the
    current status. According to program officials, previous program office
    schedule estimates developed in the early 2000s were based on
    assumptions about processing rates derived largely from peak processing
    experiences at Johnston Atoll, which began operations in 1990, and
    early munitions processing campaigns at Tooele--the first two sites in
    operation. According to CMA officials, program office schedule
    estimates used from 2001 to 2003 were derived from using the 5 best
    weeks of these two sites' production rates. The program's assumption
    was that the follow-on sites, which were similarly designed, would be
    able to achieve these rates because of the program's years of
    experience with the various munitions and agents. Program officials
    said that in practice those rates were achievable only for short
    periods and were not sustainable because of problems with individual
    munitions, such as difficulty in draining the agent (much slower than
    assumed), equipment jams, and processing restrictions specified in
    facility operating permits.

    New Schedule Methodology Includes More Time for Downtime and Processing
    Risks:

    The program office adopted a new schedule development methodology in
    2004 that not only used modeling to determine processing rates, but
    gave greater recognition to the issues that cause delays, such as plant
    downtime days for equipment maintenance and the inclusion of risks,
    such as those associated with the unexpected condition of the stored
    munitions that further extended the schedule. While previous schedule
    estimates were point estimates, in that a single processing rate was
    developed and used for each campaign and the individual campaigns were
    added together, statistical modeling was used to develop a distribution
    of different schedule outcomes. Processing rates were evaluated by
    munitions type, such as projectiles, rockets, and mines, to develop a
    histogram of daily processing outcomes.[Footnote 34] These results were
    used as part of the modeling inputs. For current estimates, unlike past
    estimates, schedule estimators also collected data on issues that had
    caused processing delays in the past and identified possible impacts on
    the schedule and probabilities of occurrence.

    Data were also collected on past program incidents that had caused
    sites to remain idle for extended periods, and the estimators also
    developed probabilities for these "downtime events." The model was run
    to produce a distribution of 5,000 schedule outcomes. The median
    result--referred to as the 50 percent confidence level--was selected as
    the program office schedule estimate. Selection of the median result
    was also considered conservative, since the previous program office
    schedules have used the mode, which is the outcome that happens most
    frequently. According to schedule estimators, the median result is
    typically to the right of or longer than the mode.

    Thus, when the schedule estimates were updated in 2005 using the median
    schedule outcomes, they were not only more conservative, but also
    considered more realistic. Figure 5 shows how processing rates,
    downtime, and risk contributed to the schedule estimate.

    Figure 5: 2005 Program Office Schedule Estimates by Processing Rates,
    Downtime, and Risk:

    This figure is a bar graph showing 2005 program office schedule
    estimates by processing rates, downtime, and risk.

    [See PDF for image]

    Source: CMA.

    [End of figure]

    Program Office Schedule Extensions Led to Revised Program Baseline:

    When these schedule estimates exceeded the schedule milestones
    contained in the April 2003 APB and resulted in major changes to the
    life cycle cost estimates, an acquisition-related process was initiated
    to develop new schedule milestones. The new schedule estimate was
    approved by the Army Cost Review Board in May 2005 and resulted in the
    new Army Cost Position. Concurrently, DOD was also conducting a program
    review through its Defense Acquisition Board process that would result
    in new schedule estimates to be included in the program's Approved
    Acquisition Program Baseline. As part of this process, the DOD's Cost
    Analysis Improvement Group (CAIG) developed independent schedule
    estimates.[Footnote 35] CAIG's schedule forecasts were also based on
    modeling to select the median outcome. Unlike those based on the
    program rebaselining that occurred from 2001 to 2003, the program
    office schedule estimates and those developed by CAIG were relatively
    close in most cases. The schedule milestones in the 2003 APB are
    compared with those in the 2005 Army and DOD schedule estimates in
    table 4.

    Table 4: Comparison of Estimated Operation Completion Dates:

    Site: Anniston;
    2003 Acquisition Program Baseline: May 2011;
    2005 Army program office estimate: September 2016;
    Difference from 2003 Acquisition Program Baseline: 64 months;
    2005 DOD independent estimate: June 2016;
    Difference from 2003 Acquisition Program Baseline: 61 months;
    Program current estimate: May 2015.

    Site: Newport;
    2003 Acquisition Program Baseline: January 2006;
    2005 Army program office estimate: October 2011;
    Difference from 2003 Acquisition Program Baseline: 75 months;
    2005 DOD independent estimate: February 2012;
    Difference from 2003 Acquisition Program Baseline: 73 months;
    Program current estimate: October 2009.

    Site: Pine Bluff;
    2003 Acquisition Program Baseline: November 2009;
    2005 Army program office estimate: January 2015;
    Difference from 2003 Acquisition Program Baseline: 62 months;
    2005 DOD independent estimate: November 2015;
    Difference from 2003 Acquisition Program Baseline: 72 months;
    Program current estimate: June 2013.

    Site: Tooele;
    2003 Acquisition Program Baseline: February 2008;
    2005 Army program office estimate: October 2014;
    Difference from 2003 Acquisition Program Baseline: 80 months;
    2005 DOD independent estimate: September 2015;
    Difference from 2003 Acquisition Program Baseline: 91 months;
    Program current estimate: November 2014.

    Site: Umatilla;
    2003 Acquisition Program Baseline: January 2011;
    2005 Army program office estimate: June 2017;
    Difference from 2003 Acquisition Program Baseline: 77 months;
    2005 DOD independent estimate: March 2017;
    Difference from 2003 Acquisition Program Baseline: 74 months;
    Program current estimate: December 2014.

    Source: GAO analysis of DOD and CMA data.

    [End of table]

    Recently Improved Processing Rates May Make the Current Schedule
    Milestones Overly Conservative, but Program Confidence Levels for
    Meeting Treaty Milestones Are Low:

    Our review of processing rates that have been achieved at the sites
    since the program office estimates were developed in 2005 determined
    that most sites are significantly ahead of the program office estimate
    for recently completed munition campaigns, indicating that schedule
    milestones being used to measure the program's progress may be too
    conservative to be realistic. For example, the 2005 program office
    schedule forecasted that Umatilla would complete destruction of its
    91,442 sarin nerve agent rocket munitions by August 2007. Those
    operations were actually completed in August 2006--about 12 months
    ahead of the program office 2005 estimate. This processing rate was
    achieved despite delays to the site operator internal schedule related
    to the unanticipated fires associated with some of the rockets.
    Additional progress made through May 2007 has placed Umatilla about
    1,184 days ahead of 2005 program office schedule estimate. We found
    similar occurrences at the other incineration sites, and actual and
    near-term projected performance by sites reveals that significant
    schedule progress has been made since the 2005 schedule-estimating
    process. Table 5 shows actual and projected schedule savings as
    compared to the 2005 program office estimated schedule for campaigns
    recently completed or under way.

    Table 5: Number of Days Saved per Site since 2005 Program Office
    Schedule Estimates:

    Site: Anniston;
    Cumulative days saved: 443;
    Actual or projected days saved: Actual;
    End date of last completed campaign completed to date: March 8, 2007;
    Campaigns ongoing and completed or started and completed from May 2005
    through July 2007: GB 8-inch projectiles;
    GB 155-mm projectiles;
    GB M360 105-mm projectiles;
    VX M55 rockets/warheads.

    Site: Newport;
    Cumulative days saved: 658;
    Actual or projected days saved: Projected;
    End date of last completed campaign completed to date: December 2,
    2009[A];
    Campaigns ongoing and completed or started and completed from May 2005
    through July 2007: VX agent neutralization.

    Site: Pine Bluff;
    Cumulative days saved: 375;
    Actual or projected days saved: Actual;
    End date of last completed campaign completed to date: May 19, 2007;
    Campaigns ongoing and completed or started and completed from May 2005
    through July 2007: GB M55 rockets/ warheads.

    Site: Tooele;
    Cumulative days saved: 61;
    Actual or projected days saved: Actual;
    End date of last completed campaign completed to date: August 17, 2006;
    Campaigns ongoing and completed or started and completed from May 2005
    through July 2007: VX land mines;
    Agent changeover.

    Site: Umatilla;
    Cumulative days saved: 1,184;
    Actual or projected days saved: Actual;
    End date of last completed campaign completed to date: January 24,
    2007;
    Campaigns ongoing and completed or started and completed from May 2005
    through July 2007: GB M55 rockets/ warheads;
    GB MC-1 bombs;
    GB MK-166 bombs;
    GB 8-inch projectiles;
    GB 155-mm projectiles.

    Site: Aberdeen;
    Cumulative days saved: 750;
    Actual or projected days saved: Actual;
    End date of last completed campaign completed to date: February 13,
    2006;
    Campaigns ongoing and completed or started and completed from May 2005
    through July 2007: Ton container cleanout.

    Total;
    Cumulative days saved: 3,471;
    Actual or projected days saved: [Empty];
    End date of last completed campaign completed to date: [Empty];
    Campaigns ongoing and completed or started and completed from May 2005
    through July 2007: [Empty].

    Source: GAO analysis of Army data.

    [A] Figures listed above for Anniston, Pine Bluff, Tooele, and Umatilla
    represent actual savings. Figures for Newport are projected savings for
    current ongoing campaigns based on the historical processing rates at
    those sites. The Newport forecast for completing the current campaign
    is December 2009.

    [End of table]

    We calculate that to date, almost $1.3 billion in cost savings has
    accrued based on schedule savings since the 2005 program office
    estimates were established. Table 6 shows actual and projected schedule
    savings as compared to the 2005 estimated schedule savings for
    campaigns recently completed or under way.

    Table 6: Potential Operational Cost Savings Associated with Days Saved
    since 2005 Schedule Estimate:

    Dollars in millions.

    Anniston;
    Cumulative days saved: 443;
    Potential cost savings: $178.

    Newport;
    Cumulative days saved: 658;
    Potential cost savings: 227[A].

    Pine Bluff;
    Cumulative days saved: 375;
    Potential cost savings: 144.

    Tooele;
    Cumulative days saved: 61;
    Potential cost savings: 25.

    Umatilla;
    Cumulative days saved: 1,184;
    Potential cost savings: 452.

    Aberdeen;
    Cumulative days saved: 750;
    Potential cost savings: 236.

    Total;
    Cumulative days saved: 3,471;
    Potential cost savings: $1,262.

    Source: GAO analysis of Army data.

    Note: Time savings were converted to a cost by using the average fiscal
    year 2005 and fiscal year 2006 budgets expressed in dollars per day.

    [A] Figures listed above for Anniston, Pine Bluff, Tooele, and Umatilla
    represent actual savings. Figures for Newport are projected savings for
    current ongoing campaigns based on the historical processing rates at
    those sites. The Newport forecast for completing the current campaign
    is December 2009.

    [End of table]

    The April 2007 SAR for the CMA sites (including Newport) reported new
    program office schedule estimates, which were a significant improvement
    from the Army's 2005 program office schedule estimates that generated
    the program's rebaselining and also were significantly ahead of the
    independent DOD schedule estimates that are used as the program's
    unofficial schedule baseline and were developed by CAIG. Table 7 shows
    the number of months that the estimated schedule for completing
    operations has improved from 2005 to 2007.

    Table 7: Number of Months of Improvement in 2007 Program Current
    Estimate for Operations Completion versus 2005 Army Program Office
    Estimate and 2005 DOD Independent Estimate:

    Site: Anniston;
    2005 Army program office estimate: September 2016;
    2005 DOD independent estimate: June 2016;
    2007 program current estimate: May 2015;
    Improvement from 2005 program office estimate: 16 months;
    Improvement from 2005 independent DOD estimate: 13 months.

    Site: Newport;
    2005 Army program office estimate: October 2011;
    2005 DOD independent estimate: February 2012;
    2007 program current estimate: October 2009;
    Improvement from 2005 program office estimate: 24 months;
    Improvement from 2005 independent DOD estimate: 28 months.

    Site: Pine Bluff;
    2005 Army program office estimate: January 2015;
    2005 DOD independent estimate: November 2015;
    2007 program current estimate: June 2013;
    Improvement from 2005 program office estimate: 19 months;
    Improvement from 2005 independent DOD estimate: 29 months.

    Site: Tooele;
    2005 Army program office estimate: October 2014;
    2005 DOD independent estimate: September 2015;
    2007 program current estimate: November 2014;
    Improvement from 2005 program office estimate: (1 month);
    Improvement from 2005 independent DOD estimate: 10 months.

    Site: Umatilla;
    2005 Army program office estimate: June 2017;
    2005 DOD independent estimate: March 2017;
    2007 program current estimate: December 2014;
    Improvement from 2005 program office estimate: 30 months;
    Improvement from 2005 independent DOD estimate: 27 months.

    Source: GAO analysis of DOD and CMA data.

    [End of table]

    In addition to its annual schedule update process, CMA tracks on a
    monthly basis the confidence levels of achieving various program
    schedule estimates. CMA tracks these levels after updating its model to
    incorporate updated processing rates and any changes to estimates for
    downtime or risk. These updates have shown upward trends in the
    confidence level, which is the percentage of schedule outcomes that
    meet or exceed a given date. As previously discussed, the 2005 program
    office estimate reflected a 50 percent confidence level. By the end of
    fiscal year 2006, which was about 18 months after the 2005 program
    office estimates were developed, only Tooele, at 48 percent, was
    slightly below the 50 percent confidence level. The other three sites'
    confidence levels for achieving the 2005 program office estimate for
    completing operations had increased significantly, ranging from 71
    percent at Anniston to 83 percent at Umatilla. Moreover, confidence
    levels based on monthly updates conducted in April 2007 showed that the
    program office estimates for meeting the DOD schedule milestones were
    as high as 98 percent for completing operations, but confidence levels
    were very low for meeting the April 29, 2012, treaty deadline, as shown
    in table 8.

    Table 8: CMA Assessed Confidence in Achieving DOD Schedule Milestone
    and April 29, 2012, Treaty Deadline:

    Percentages: Site: Anniston;
    Program's assessed confidence in achieving operations schedule
    milestone: 73;
    Program's assessed confidence in sites completing destruction by April
    29, 2012, CWC milestone: 5.

    Percentages: Site: Newport;
    Program's assessed confidence in achieving operations schedule
    milestone: 98;
    Program's assessed confidence in sites completing destruction by April
    29, 2012, CWC milestone: 98.

    Percentages: Site: Pine Bluff;
    Program's assessed confidence in achieving operations schedule
    milestone: 81;
    Program's assessed confidence in sites completing destruction by April
    29, 2012, CWC milestone: 19.

    Percentages: Site: Tooele;
    Program's assessed confidence in achieving operations schedule
    milestone: 42;
    Program's assessed confidence in sites completing destruction by April
    29, 2012, CWC milestone: 1.

    Percentages: Site: Umatilla;
    Program's assessed confidence in achieving operations schedule
    milestone: 84;
    Program's assessed confidence in sites completing destruction by April
    29, 2012, CWC milestone: 2.

    Source: CMA.

    [End of table]

    While the program office has updated its schedule estimates to reflect
    the latest processing rates and changes to other assumptions, resulting
    in schedule improvements in excess of 18 months in most cases,
    independent schedule estimates have not been developed since the CAIG
    2005 estimates because these estimates are usually performed in
    conjunction with program rebaselinings because of schedule or cost
    growth. Also, when schedules or other assumptions change, cost and
    schedule estimates should be revised to reflect current status. Because
    these steps have not been taken, DOD and Congress can have little
    confidence that the program is being held accountable to a reasonable
    baseline to measure program progress and that program costs are
    reasonable.

    Shortcomings in Underlying Cost Data Make Earned Value Management
    Results Unreliable and Limited Our Ability to Determine Cost Estimate
    Reliability:

    Because of shortcomings in the underlying data, the usefulness of the
    EVM data to make cost projections is limited, which makes results
    unreliable, and our ability to independently verify and validate the
    reliability of the cost estimates was restricted. The Chemical
    Demilitarization Program's projected cost growth is largely
    attributable to longer schedules and increased costs associated with
    facility closure estimates. Shortcomings in the underlying data include
    invalid and unstable performance baselines. The performance baselines
    are problematic because all requirements, such as closure costs, are
    not fully identified and included in program cost estimates and because
    large amounts of additional costs are added to the performance baseline
    annually. Moreover, there were significant differences between the
    systems contractor cost estimates through completion and the government
    cost estimates through completion because the government anticipates
    longer operational periods and greater closure requirements.

    Projected Cost Growth Is Largely Attributable to Longer Schedules and
    Increased Facility Closure Estimates:

    The Chemical Demilitarization Program's continuing growth in cost
    estimates is largely attributable to longer schedules and the increased
    costs associated with plant closure. Longer schedules led to increases
    in cost estimates for sites currently in the operations phase because
    of additional labor costs, which are a high percentage of site costs.
    In addition, according to CMA officials, labor costs are very expensive
    for this program because of the need for highly specialized labor that
    cannot be easily replaced. According to these officials, each staff
    member, for example, must pass the Personal Reliability Program, which
    provides specialized training and security clearances for working at
    the chemical deconstruction plants. Program officials have stated that
    for this reason, it is often more cost-effective during plant shutdowns
    or extended delays to retain this specialized workforce than to lay off
    and rehire new staff. This workforce costs approximately $9.5 million
    per month based on an average monthly spend rate across the five sites
    we examined, which means that a 1-month schedule extension could
    increase the life cycle cost estimates by about that amount. Table 9
    shows the increase in program cost as the schedule was extended.

    Table 9: Changes in Estimated Total Program Cost:

    Then-year dollars in billions.

    Total life cycle cost estimate;
    2003 APB: $19.6;
    2005 Army cost position: $29.2;
    Increase: $9.6;
    April 2007 SAR: $28.6.

    Source: U.S. Army.

    [End of table]

    While the Chemical Demilitarization Program Is Using Earned Value
    Management as a Tool, Some Underlying Data Make Results Unreliable:

    EVM is an important program management tool required by DOD and OMB
    that measures performance by comparing the value of work accomplished
    with the value of work planned and the cost of the work performed.
    Using EVM data, early warning of schedule delays and cost overruns can
    be determined. EVM uses contractor-reported data to examine variances
    reported in contractor performance reports between actual cost and time
    of performing work tasks and the budgeted or estimated cost and time.
    As a result, DOD and OMB policy and guidance require the use of EVM to
    measure program performance.[Footnote 36]

    The Chemical Demilitarization Program uses EVM[Footnote 37] as a tool
    that allows both government and systems contractor managers to have
    visibility over cost performance on their contracts. However, the
    ability of the program's EVM system to reliably manage and forecast the
    program's cost estimates may be affected by large variances between the
    systems contractors' estimated cost to complete operations and closure
    versus the government's estimate; systems contractors' baselines that
    have not been carefully controlled, leading to high percentages of
    authorized work that is unpriced for long periods of time; and limited
    identification of requirements for closure costs at individual sites.

    Because of the shortcomings in the underlying cost data, the usefulness
    of the EVM data to make cost projections is limited and our ability to
    independently verify and validate the reliability of the cost estimates
    was restricted. In other words, the mechanics of the EVM system are
    working properly; however, we found that the performance management
    baseline does not accurately reflect the remaining life cycle costs of
    the Chemical Demilitarization Program. DOD's EVM implementation guide
    states that "the "to-go" plan should reflect a realistic schedule of
    how the remaining work actually is to be done and the new budget should
    be adequate and reflect a realistic estimate, remaining program risks,
    and contain an appropriate amount of management reserve."[Footnote 38]
    According to best practices, an EVM system should be devoid of data
    errors and anomalies that may distort the analysis.[Footnote 39] In
    addition, any cost or schedule variances should be explained and
    corrective actions identified. Furthermore, the EVM system should have
    a valid and stable performance baseline from which to make projections
    and do trend analysis.[Footnote 40] In the case of the Chemical
    Demilitarization Program, the cost account managers are inserting their
    costs into the system properly, system certification verifies that the
    EVM reported costs match the costs in the accounting system, the EVM
    reports are being prepared with minimal errors, and cost account
    managers are explaining any variances. However, the actual data are
    measured against a baseline that does not reflect all of the work to be
    performed, such as facility closure activities, which distorts cost and
    schedule performance metrics. This undermines the effectiveness of the
    EVM system in providing management with accurate and realistic metrics
    for making informed management decisions.[Footnote 41]

    Performance Baselines Lacked Validity and Have Been Unstable:

    Our review showed that the performance baselines lack validity and have
    not been stable because CMA and site systems contractors have not fully
    identified all requirements, such as closure costs, and included them
    in program cost estimates and because large amounts of additional
    costs[Footnote 42] are added to the performance baseline annually. EVM
    guidelines state that once a contract is awarded, the contractor should
    establish a performance baseline that will serve as a time-phased
    budget plan for accomplishing work and measuring contract
    performance.[Footnote 43] This is essentially the spending plan for the
    program and should closely equate to the planned budget, which is the
    value of work planned through the project's completion. The performance
    baseline should include all costs associated with completing the
    program, including direct and indirect labor, material, and other
    direct costs associated with the authorized work. However, we found
    that each of the sites was lacking a stable and reliable EVM
    performance baseline, which hindered our ability to perform an
    independent EVM analysis.[Footnote 44]

    While the planned budget and the performance baseline should closely
    equate, we found that the performance baseline is significantly less
    than the planned budget (close to $2 billion less) because most sites
    have significant amounts of work that are authorized by the systems
    contractors and the Army but are not yet placed on contract. For
    example, the April 2007 SAR reports that while the contract value at
    Umatilla is $1.2 billion through December 2006, an additional $700
    million has been authorized but is pending negotiation. Also, the
    Estimate at Completion (EAC) cost estimate, which represents all
    estimated costs through the end of the contracts, is $4 billion more
    than the performance baseline. Figure 6 shows the extent that planned
    budget and the EAC exceed the performance baseline for all sites
    combined over a 19-month period from August 2005 to February 2007.

    Figure 6: Comparison of the Performance Baseline with the Planned
    Budget and Systems Contractors' Estimate at Completion:

    This figure is a combination line graph comparing the performance
    baseline with the planned budget and systems contractors' estimate at
    completion. The X axis is the year and month, and the Y axis represents
    dollars in millions. One line represents planned budget, another
    represents estimate at completion, and another represents performance
    baseline.

    [See PDF for image]

    Source: GAO analysis of Army data.

    [End of figure]

    The performance baseline does not include all requirements, such as
    closure costs, which are not fully identified and included in program
    cost estimates. Best practices state that a cost estimate should be
    comprehensive in order to be considered credible.[Footnote 45] Systems
    contractor officials stated that in many instances what was included in
    their estimated completion costs were placeholder amounts, which they
    believed to be unrealistic. According to program office officials, the
    government and the systems contractors have not yet completely defined
    the requirements for closure, so a realistic closure cost estimate is
    not currently available. However, since two sites--Johnston Atoll and
    Aberdeen--have already been closed historical data are now available to
    develop more accurate closure estimates at the remaining
    sites.[Footnote 46] For example, based on the Army's experience of
    destroying blister and nerve agents at Johnston Atoll, closure costs
    for other destruction sites are increasing from initial estimates. As a
    result of incorporating these actual data, assumptions for conducting
    closure activities have increased from 12 months to about 30 months at
    some sites.

    The performance baseline also is unstable because large amounts of
    additional costs are added to the performance baseline
    annually.[Footnote 47] Each fiscal year, a portion of the unpriced work
    that is part of the planned budget is negotiated (priced), and it is
    moved into the performance baseline. The negotiated funding is often
    greater than what was used as a "placeholder" budget, increasing the
    contract baseline to more than what was included as unpriced work. For
    example, the fiscal year 2006 negotiated increase in the contract value
    at Anniston was $6 million more than the reduction in authorized
    unpriced work. Program office officials explained that the performance
    baseline at each site is constantly changing because of yearly contract
    renegotiations[Footnote 48] for future operations and closure costs.
    Program officials attribute the yearly changes to the contracts using
    Operating and Maintenance funds, which expire within 1 fiscal year.
    This situation created the need to put on contract a year's worth of
    work in conjunction with each fiscal year's appropriation.

    CMA officials stated that they are making efforts to stabilize the
    baseline by placing the full life cycle costs associated with all
    activities at each site. In its technical comments on a draft of this
    report, DOD stated that three of the five CMA sites (Anniston, Pine
    Bluff, and Umatilla) had completed contract performance baselines and
    negotiations were ongoing at the two remaining sites (Newport and
    Tooele). DOD expected that the negotiations for these two sites would
    be completed by June 30, 2008. Once this happens, CMA officials believe
    that there will be less of a need for annual negotiations and more
    realistic performance baselines should result. Until program management
    has a performance baseline that reflects the current plan to complete
    remaining work and follows a disciplined process for incorporating
    changes, management will not be able to benefit from the EVM system's
    ability to reveal actual program performance and detect program risks.

    Program Manager-Estimated Completion Costs Are Significantly Higher
    Than Completion Costs Generated from EVM Data, Suggesting Data Are
    Incomplete:

    It is a best practice to develop a range of independent EACs using
    commonly accepted EVM equations because doing so informs management of
    the likely range of costs for completing the program so appropriate
    action can be taken.[Footnote 49] In addition, DOD guidance states that
    remaining work should reflect a realistic schedule and budget that also
    reflect remaining risks.[Footnote 50] Finally, EVM guidelines state
    that program managers should, at least on a monthly basis, rely on EVM
    data to generate EACs that are derived based on the cost of work
    completed to date along with an estimate of what it will cost to
    complete the rest of the program.[Footnote 51] By relying on past
    performance and using well-defined EVM indices, a variety of EACs can
    be generated. However the EACs generated are only as good as the
    underlying data in the EVM system.

    Based on our analysis, program cost estimates often exceeded the worst
    case of our range of independently developed completion costs estimates
    because of shortcomings in the underlying data of the EVM systems
    maintained at the chemical demilitarization sites. It is a best
    practice to develop a range of independent cost estimates using
    commonly accepted EVM equations. Doing so informs management of the
    likely range of costs for completing the program so that appropriate
    action can be taken. We found that when we developed independent EACs
    using the systems contractor's EVM data and commonly accepted EVM
    forecasting equations in all cases the systems contractors' EACs are
    either at the upper end of or higher than our estimated EAC range based
    on our analysis shown in table 10. In addition, the program manager's
    EAC was far outside of our projected range and, in many cases, was
    almost twice as high as our EAC. For example, our estimated cost range
    for Anniston based on the EVM data was from $1.6 billion to $1.7
    billion. The systems contractor's most likely cost estimate was $100
    million above the upper limit of our range, and the program manager's
    estimate was $1.4 billion higher than the upper range limit. We
    attribute this difference to an understated performance baseline, which
    is critical to the estimating process. Table 10 compares our
    independent EAC to that of the systems contractor's and program
    manager's EACs.

    Table 10: Comparison of GAO's Independent Estimates to Complete
    Operations and Closure Activities with the Systems Contractor's and
    Program Manager's Estimates:

    Then-year dollars in billions.

    Anniston;
    Original contract cost: $0.6;
    Current planned budget: $1.7;
    Systems contractor's most likely EAC: $1.8;
    GAO's independent EAC range: $1.6-$1.7;
    2006 Selected Acquisition Report program manager's EAC: $3.1.

    Newport;
    Original contract cost: 0.3;
    Current planned budget: 1.0;
    Systems contractor's most likely EAC: 1.1;
    GAO's independent EAC range: 0.9-1.1;
    2006 Selected Acquisition Report program manager's EAC: 1.8.

    Pine Bluff;
    Original contract cost: 0.5;
    Current planned budget: 1.5;
    Systems contractor's most likely EAC: 1.9;
    GAO's independent EAC range: 1.4-1.5;
    2006 Selected Acquisition Report program manager's EAC: 2.4.

    Tooele;
    Original contract cost: 0.2;
    Current planned budget: 1.6;
    Systems contractor's most likely EAC: 2.4;
    GAO's independent EAC range: 1.4-1.6;
    2006 Selected Acquisition Report program manager's EAC: 3.3.

    Umatilla;
    Original contract cost: 0.5;
    Current planned budget: 1.8;
    Systems contractor's most likely EAC: 1.9;
    GAO's independent EAC range: 1.6-1.8;
    2006 Selected Acquisition Report program manager's EAC: 3.8.

    Source: GAO analysis of Army data.

    [End of table]

    In addition, the large difference between the program manager's EAC and
    our independent EACs indicates that there is more work that is being
    considered by the program manager. More work, in turn, will affect
    future costs and requirements than is predicted by the EVM data and,
    therefore, calls into question the usefulness of the EVM data to
    effectively manage the program.[Footnote 52] The ability to respond
    quickly to program problems depends on having early visibility into
    what is causing them. Thus, if the EVM performance baseline is missing
    some of the remaining work, then accurate progress assessments will not
    be available to provide management with a better picture of program
    status. This lack of insight may hinder decision making and program
    success because problems may go undetected.

    Without accurate and realistic closure requirements used to develop
    completion cost estimates, DOD and Congress will not be able to
    accurately identify and adequately budget for all activities through
    facility closure. Similarly, without an independent cost review once
    the cost estimates are complete and schedule milestones have been
    updated and reviewed, DOD and Congress will not have assurance that
    cost estimates for the closure of chemical demilitarization sites are
    realistic.

    The Army Has Actively Managed Award Fees and Could Further Emphasize
    Schedule and Cost and Link Award Fees to More Objective Performance
    Measures:

    Our review of the Army's use of monetary fees to incentivize certain
    aspects of the systems contractors' performance at the five sites in or
    near beginning operations in 2003 showed that the contract monetary fee
    structures at the incineration sites differed from those at Newport's
    neutralization site. Award fees are predominant at the incineration
    sites while incentive fees are predominant at Newport. We found that
    while the Army has actively reviewed and adjusted the types of monetary
    fees used and the criteria and weighting for determining monetary fees,
    there be may be opportunities to further emphasize schedule and cost.
    For example, during the period we reviewed, award fees have emphasized
    safety and environmental compliance at each site, but have not resulted
    in the control of schedule and cost growth. In fiscal year 2006, after
    several external organizations recommended that it do so, the Army
    reinstituted the use of a distinct award fee performance measure for
    schedule.[Footnote 53] However, site project managers often weighted
    schedule at the lower end of the specified range. Also, while DOD has
    published guidance that award fees should be tied to identifiable
    interim outcomes, discrete events, or milestones as much as possible,
    we found that several performance measures, such as management and
    cost, had evaluation factors that were all or mostly all subjective and
    were not clearly linked to a measurable outcome. In addition to award
    fees, in 2005, the Army established the Director's Programmatic
    Performance Based Incentive (DPPBI), a performance-based incentive fee
    to encourage the incineration site systems contractors to establish a
    collaborative relationship with the goal to share lessons learned in
    areas that would result in schedule reduction across all sites. Before
    fiscal year 2007, this fee was based on subjective schedule performance
    measures. By fiscal year 2007, DPPBI had evolved to focus mostly on the
    sites' collective schedule performance linked to the CWC treaty
    deadline. It is unclear how effective this incentive fee will be in
    motivating individual systems contractors' schedule performance due to
    the collective manner in which the fee is earned, its relative size,
    and recent focus on objective schedule performance. An additional
    incentive fee, authorized by section 923 of the National Defense
    Authorization Act for Fiscal Year 2007, provides up to $165 million at
    each incineration site for the completion of destruction and facility
    closure by or close to the CWC deadline. This incentive fee, as
    planned, is structured to accelerate destruction operations and
    facility closure at the incineration sites. Because this fee has not
    yet been implemented, we have no basis at this time to determine how
    effective this incentive might be at controlling schedule.

    Contract Monetary Fee Structures at the Incineration Sites Differ from
    Those at Newport's Neutralization Site, and Award Fees Are Predominant
    at the Incineration Sites:

    CMA uses cost-plus-award-fee contracts with incentive fees to encourage
    and reward the systems contractors to perform in an innovative,
    efficient, and effective way in areas deemed important to the program's
    success. Under these contracts, the systems contractors' allowable,
    incurred costs are reimbursable to the extent prescribed in the
    contract. According to CMA officials, the Chemical Demilitarization
    Program systems contracts are cost reimbursable, based on the fact that
    it is not possible to accurately describe the requirements in
    sufficient detail to address the myriad risks and uncertainties
    associated with destroying aging and deteriorating munitions that are
    filled with chemical agent. As a result of this high-risk environment,
    the government accepts most of the cost risk. Thus, when problems
    arise, such as rocket fires or mercury contamination, government and
    systems contractor resources are focused on fixing the problems or
    mitigating the risk rather than fixing blame.

    The base fee for this program is a fixed amount of money negotiated at
    the beginning of each year of contract operations, and is usually paid
    on a monthly basis to the systems contractor for its performance,
    though this fee is not tied to specific performance criteria as are the
    award and incentive fees. According to program officials, the rationale
    for providing a base fee is due to the program's complexity and to
    provide the systems contractors with a level cash flow for financial
    viability. Award fees are intended to motivate systems contractor
    performance in areas that are susceptible to judgmental and qualitative
    measurement and evaluation and, as a result, award fee criteria and
    evaluations tend to be subjective. Typically, award fee contracts
    emphasize multiple aspects of contractor performance in a wide variety
    of areas, such as quality, timeliness, technical ingenuity, and cost-
    effective management.[Footnote 54] Finally, incentive fee contracts use
    what is considered an objective evaluation of the systems contractor's
    performance to adjust the fee paid, which usually involves applying a
    fee determination formula specified in the contract.

    The contracts at each of the four operational incineration sites--
    Anniston, Pine Bluff, Tooele, and Umatilla--include (1) a base fee
    (representing 3 percent of the negotiated total cost of the contract),
    which is a fixed amount not subject to performance evaluation; (2) an
    award fee that is 7 percent of the negotiated contract costs, which
    will be determined through performance evaluations and paid from an
    award fee pool; and (3) DPPBI, a performance-based incentive fee
    established in 2005 and paid from an available fee pool amount that is
    2 percent of negotiated contract cost. The contract at the remaining
    neutralization site--Newport--has a base fee (3 percent), an award fee
    (3 percent), and a performance-based incentive fee (6
    percent).[Footnote 55] All five contracts had performance-based
    incentive fees added after initial contract award.

    The types and percentages of fees offered to the systems contractors to
    safely destroy the chemical weapons stockpile differ and emphasize
    different performance measures. For example, at the incineration sites,
    the award fee is the predominant fee and, from fiscal year 2003 to
    fiscal year 2007, this fee has largely emphasized safety and surety,
    environmental compliance, and management goals. Conversely, at the
    neutralization site, the performance-based incentive fee is the
    predominant fee, which has emphasized schedule goals. Figure 7 compares
    the monetary incentive fees earned by the systems contractors at the
    incineration sites and at Newport. As shown in figure 7, in fiscal year
    2006, CMA paid the systems contractors at the incineration sites a
    combined total of about $39.9 million in fees, which consisted of about
    $31.1 million (or 78 percent) in award fees and about $8.8 million (or
    22 percent) in performance-based incentive fees. In contrast, CMA paid
    the neutralization site system contractor at Newport about $14.7
    million in monetary fees, which consisted of about $2.5 million (or 17
    percent) in award fees and upwards of about $12.1 million (or 83
    percent) in performance-based incentive fees. In fiscal year 2006, the
    award fees earned at each of the incineration sites were almost three
    times higher than that earned at Newport.

    Figure 7: Comparison of the Monetary Incentive Fees Earned by Systems
    Contractors at the Incineration Sites and at Newport during Fiscal Year
    2006:

    This figure is a combination of two pie charts comparing the monetary
    incentive fees earned by systems contractors at the incineration sites
    and at Newport during fiscal year 2006.

    Incineration sites' monetary fees for fiscal year 2006L $33.9 million:

    Award fee: 78%;
    Incentive fee: 22%.

    Newport monetary fees for fiscal year 2006:

    Award fee: 83%;
    Incentive fee: 17%.

    [See PDF for image]

    Source: U.S. Army Sustainment Command.

    Note: GAO did not independently verify the award and incentive fee data
    provided by the U.S. Army Sustainment Command.

    [End of figure]

    By predominately using performance-based incentives at Newport, the
    Army has aligned more of its monetary fees with objective outcomes.
    However, Army officials said that it is difficult to determine how
    effective the usage of performance-based incentive fees has been in
    motivating the systems contractors to meet schedule and cost milestones
    because of the number of extensions made to the original incentive
    agreement. The extensions were made to account for the technical
    challenges encountered by the systems contractors during the
    neutralization process. After the terrorist attacks of September 11,
    2001, due to Army concerns about security vulnerabilities, CMA
    restructured the contracts to increase the emphasis on performance-
    based incentives at the Newport and Aberdeen neutralization sites to
    incentivize the systems contractors to destroy the remaining stockpile
    of lethal chemical agent on an accelerated schedule.[Footnote 56]

    Army Actively Managed Award Fees and May Be Able to Further Emphasize
    Schedule and Cost:

    Since 2003, the period of our analysis, the Army has actively reviewed
    and adjusted criteria for determining monetary fees--largely award
    fees--it provided to the systems contractors to incentivize aspects of
    their performance. CMA's award fee plans generally use performance
    measures that could be expected to incentivize the desired acquisition
    outcomes. CMA includes broad aspects of systems contractor performance,
    such as management performance, and CMA also views these as keys to a
    successful program. From 2003 through 2006, CMA used safety and surety,
    environmental compliance, cost, and management as the key performance
    measures. Until fiscal year 2006, however, CMA did not emphasize
    schedule as a distinct performance measure in the award fee
    plans.[Footnote 57]

    Army Emphasized Safety and Environmental Compliance in Award Fee Plans
    and Did Not Successfully Incentivize Schedule and Cost Performance:

    From fiscal year 2003 through fiscal year 2006, the incineration sites
    and Newport had not been able to maintain original production schedule
    milestones and had experienced significant cost growth. Although the
    management performance measure included some criteria for schedule,
    during the period from fiscal year 2003 through fiscal year 2005,
    schedule was not a distinct performance measure in the award fee plan,
    and safety and environmental compliance were heavily weighted. CMA
    officials acknowledged in the 2007 incentive agreement plan that while
    award fees have succeeded in motivating the systems contractors to
    maintain high levels of safety and environmental compliance at each
    site,[Footnote 58] they have not satisfactorily resulted in the control
    of schedule and cost growth. Many events and constraints contributed to
    the schedule and cost growth, both within and beyond the systems
    contractors' control. CMA officials stated that the process to
    determine the degree to which such issues cause variances to baselines
    is never clear, but generally, over the course of a year, there may be
    25 percent of the problems that are clearly outside of the control or
    responsibility of the systems contractor, 10 percent that are clearly
    within the control of the systems contractor, and 65 percent that fall
    into the "gray area" where it is difficult to determine whether the
    problem is clearly outside or within the control of the systems
    contractor.

    From fiscal year 2003 through fiscal year 2006, the Army paid award
    fees to the systems contractors between 70 percent and 96 percent of
    the available award fee pools primarily for good-to-outstanding
    performance in safety and surety, environmental compliance, and
    management, which collectively constituted most of the basis for
    determining the award fee. Table 11 shows both the actual amount and
    the percentage of the total available award fee pool each site was paid
    by the Army over the period from fiscal year 2003--when all five sites
    were either in or preparing for the operations phase--to fiscal year
    2006. Based on our analysis of this period, the Army paid the systems
    contractors at three of the four incineration sites award fees between
    86 percent and 96 percent of the total award fee pool available during
    this period. The systems contractor at Tooele was the exception,
    earning only 70 percent of the total available award fee pool,
    according to the U.S. Army Sustainment Command, because of
    unsatisfactory ratings due to the lack of real progress in planning and
    preparing for the mustard campaign and in not meeting government
    expectations for managing costs.

    Table 11: Percentage of Award Fees Earned at the Incineration Sites and
    Newport, from Fiscal Year 2003 to Fiscal Year 2006:

    Chemical demilitarization facility: Anniston;
    Amount of award fee pool (2003-2006): $24,536,587;
    Amount of award fee earned (2003-2006): $23,671,150;
    Percentage of award fee earned (2003-2006): 96.

    Chemical demilitarization facility: Pine Bluff;
    Amount of award fee pool (2003-2006): 22,564,501;
    Amount of award fee earned (2003-2006): 19,633,254;
    Percentage of award fee earned (2003-2006): 87.

    Chemical demilitarization facility: Umatilla;
    Amount of award fee pool (2003-2006): 26,235,256;
    Amount of award fee earned (2003-2006): 22,638,970;
    Percentage of award fee earned (2003-2006): 86.

    Chemical demilitarization facility: Newport;
    Amount of award fee pool (2003-2006): 10,532,717;
    Amount of award fee earned (2003-2006): 8,734,713;
    Percentage of award fee earned (2003-2006): 83.

    Chemical demilitarization facility: Tooele;
    Amount of award fee pool (2003-2006): 27,917,118;
    Amount of award fee earned (2003-2006): 19,414,301;
    Percentage of award fee earned (2003-2006): 70.

    Chemical demilitarization facility: Total;
    Amount of award fee pool (2003-2006): $111,786,179;
    Amount of award fee earned (2003-2006): $94,092,388;
    Percentage of award fee earned (2003-2006): [Empty].

    Source: U.S. Army Sustainment Command.

    [End of table]

    Anniston, for example, earned $23.7 million (or 96 percent) of the
    award fee during this period. According to an award fee evaluation
    board report, while the Anniston systems contractor worked aggressively
    to implement the schedule without compromising worker or public safety,
    the systems contractor faced challenges related to disciplined
    operations.[Footnote 59] The report noted that certain destruction
    campaigns at Anniston were characterized by schedule delays that
    resulted primarily from equipment reliability problems and, at times,
    inefficient planning and execution of repair efforts. The report
    further noted that while the efficiency of repairs was at times
    problematic, the systems contractor could not have prevented the
    equipment failure. The report also credited the systems contractor's
    risk mitigation efforts with significantly reducing the risk of a 6-
    month potential site shutdown that would have carried with it a
    corresponding cost impact of $75 million. Thus, CMA was able to justify
    paying the systems contractor nearly the full amount of available award
    fee. Also, during one period, Umatilla's systems contractor earned $2.5
    million (or 80 percent) of the available award fee pool while it had
    destroyed 36 percent of the nerve agent rockets planned for that
    period. According to an award fee evaluation board report, the
    processing shortfall was attributable to the systems contractor's lack
    of implementation of planned and emergent work in an efficient and
    effective manner, and was estimated to increase the planned operations
    schedule by 77 days.[Footnote 60] We have no basis for saying that the
    systems contractors should not have received their award fees, since
    they generally were evaluated as having met the award fee criteria,
    which at the time did not have schedule as a distinct performance
    measure.

    CMA officials have acknowledged that the existing award fee process has
    not satisfactorily resulted in the control of schedule and cost growth.
    CMA developed the schedule performance measure in response to Army
    Inspector General recommendations, and DOD guidance suggested that CMA
    focus more attention on schedule criteria. The Army's reinstatement of
    a distinct schedule performance measure coincided with the program's
    schedule extensions.[Footnote 61] Also, in fiscal year 2004, according
    to CMA, the Army Inspector General recommended that CMA develop
    quantifiable metrics and both the Army Inspector General and the
    Defense Contract Management Agency recommended that CMA place greater
    weight on cost and schedule control. CMA documents indicate that the
    schedule performance measure was developed after these recommendations
    and the Under Secretary's memorandum.

    Army Has Adjusted Award Fee Plans' Weighted Performance Measures and
    Could Further Increase the Weight of Performance Measures, Such as
    Schedule and Cost:

    The determination of award fee performance measures and related weight
    factors is part of CMA's award fee plan template development process,
    which is used as a guideline by site project managers to develop
    specific site award fee plans. The template describes specific
    criteria, standards, and procedures, and provides a range of weight for
    each performance measure to be used in the award fee plan to assess the
    systems contractors' performance and to determine the amount of award
    fee earned. The template allows the site project managers flexibility
    to select a weighted factor for each performance measure within the CMA
    template's range based upon the site project manager's discretion. For
    example, the award fee plan's fiscal year 2007 template had a range of
    10 percent to 20 percent for weighting management performance, 20
    percent to 30 percent for safety and surety, 20 percent to 25 percent
    for environmental compliance, 10 percent to 15 percent for cost
    performance, and 10 percent to 20 percent for schedule performance. The
    Army adjusted these ranges in fiscal year 2006, adding the schedule
    performance measure, increasing the template's range for management,
    and reducing the template's ranges for environmental compliance and
    safety and surety performance measures.

    Table 12 shows historical data on the award fee plans' performance
    measures and the relative weights selected by each of the incineration
    sites and Newport as well as the adjustments made from fiscal year 2003
    to fiscal year 2007.

    Table 12: Performance Measures and Weights in Award Fee Plans, Fiscal
    Years 2003-2007:

    [See PDF for image]

    Source: U.S. Army Sustainment Command.

    Notes: Data on the award fee plans included fiscal year 2003 through
    fiscal year 2007 to represent the period of time the sites were in or
    preparing for the operations phase. In addition, we evaluated the
    performance measures and related weights for those fiscal years since
    the previous APB was established in April 2003. N/A indicates that no
    weight was assigned to the particular award fee plan performance
    measure at that site for that particular fiscal year.

    [A] Site project managers can select weights outside of suggested
    ranges with CMA approval. While the award fee plan's fiscal year 2007
    template had a range of 20 percent to 30 percent for weighting safety
    and surety, Tooele's weight for safety and surety for fiscal year 2007
    was 35 percent.

    [End of table]

    As shown in table 12, government site project managers have weighted
    safety and surety and environmental compliance from a weight factor
    that ranged from 30 percent to 40 percent in fiscal year 2003 to a
    weight factor from 20 percent to 35 percent in fiscal year 2007. Also,
    the award fee plans for the incineration sites and Newport have safety
    "killer clause" provisions, whereby a systems contractor could lose all
    or part of its award fee if there is an agent release, an individual or
    worker is exposed to chemical agent, or there is a release of agent
    outside of the engineering controls.

    In fiscal year 2006, CMA added the schedule performance measure,
    increased the template's range for management, and reduced the
    template's range for environmental compliance and safety and surety. We
    found, based on our analysis, that the site project managers
    consistently placed schedule and cost performance at lower relative
    weights than other factors, including management performance. For
    example, in fiscal years 2006 and 2007, the maximum weight allowable by
    CMA for schedule performance was 20 percent, but the average weight
    assigned by the site project managers in the award fee plans was only
    13.75 percent for the incineration sites. Conversely, the site project
    managers weighted other performance measures, such as management, on
    average much closer to CMA's maximum suggested weights.

    Program officials stated that the weights assigned to the different
    performance measures are based on the relative significance of the
    criteria to the program. Due to the lethal nature of the material that
    must be processed, Congress has mandated that the program ensure
    maximum protection to the workforce, the public, and the environment.
    Accordingly, as program officials point out, safety/surety carry the
    highest weighting followed closely by environmental compliance.
    However, given the maturity of the operations phase, there may be
    opportunities, from a management perspective, for the Army to further
    adjust weights for performance measures in the award fee plan template
    to further incentivize key acquisition outcomes, such as schedule and
    cost, at the incineration sites. The five incineration sites have been
    preparing for or were in operation since at least 2003 and, along with
    Newport and Aberdeen, have eliminated over 45 percent of the chemical
    munitions stockpile. In addition, the Army has stated that it has
    reduced much of its safety risk by destroying the bulk of its most
    lethal and volatile chemical munitions. For example, the Tooele site
    has mostly ton containers with mustard agent left to destroy, which is
    considered less risky than other agents. Moreover, Army officials have
    pointed out that the systems contractors have implemented safety
    improvement plans that have incorporated appropriate protocols and
    standards into their daily operations. As a result, from a management
    perspective, there may be opportunities to further emphasize schedule
    and cost without sacrificing safety and environmental compliance.

    It Is Possible to Make Greater Use of Objective Award Fee Criteria and
    Evaluation Factors:

    Based on our review of the fiscal year 2007 award fee plan's
    performance measures and related evaluation criteria for the sites,
    most of the award fee criteria and evaluation factors are not objective
    and measurable. We determined that it is possible to make CMA's award
    fee plans more effective in achieving certain acquisition outcomes
    through greater use of objective award fee criteria. We previously
    recommended that DOD instruct the military services to move toward more
    outcome-based award fee criteria that are achievable and promote
    accountability for and are directly linked to achieving desired program
    outcomes.[Footnote 62] In March 2006, DOD issued award fee guidance
    stating that it is imperative that award fees be tied to identifiable
    interim outcomes, discrete events, or milestones as much as
    possible.[Footnote 63] We acknowledge that award fee contracts are
    expected to use subjective criteria to the extent that objective
    criteria are not feasible. However, we identified additional
    opportunities, from a management perspective, for CMA to make greater
    use of objective award fee criteria and evaluation factors to
    incentivize contractors to achieve desired acquisition outcomes, such
    as safely destroying the chemical stockpile on schedule and within
    cost.

    The award fee plans for this program make use of both subjective and
    objective criteria and our analysis of the fiscal year 2007 award fee
    plan showed that in most cases, the criteria are subjective. Three
    performance measures--safety, environmental compliance, and schedule--
    use a mix of objective and subjective performance evaluation criteria,
    although we noted these evaluations can also factor in subjective
    assessments. The award fee plan states that in evaluating performance
    in these areas, a number of subjective attributes will be considered,
    but there are no specific weights associated with the attributes.
    Rather, they are considered guides to evaluate the broad spectrum of
    potential systems contractor performance. The schedule performance
    measure was, for the most part, objective and linked to an outcome--the
    destruction of chemical munitions. For example, the 2007 schedule
    performance category used site-specific destruction tonnage goals
    related to meeting CWC deadlines and the APB schedule. On the other
    hand, three other performance measures--cost, management, and surety--
    are evaluated using all subjective criteria. While we recognize that
    using subjective criteria to assess systems contractor performance is
    acceptable, we identified opportunities for the Army to use more
    objective award fee criteria when evaluating contractor performance.

    We developed some illustrative examples to show how certain elements of
    CMA's subjective criteria taken from the 2007 award fee template could
    be revised to objectively measure systems contractor performance. We
    recognize that there are other examples that could be used to
    illustrate this point and do not suggest that these examples
    necessarily be used. To develop these examples, we used as criteria our
    analysis and prior work.[Footnote 64] For example, the current award
    fee plan contains the following subjective criterion for evaluating
    management performance: "Demonstrated ability to practice proactive
    management to identify and anticipate problems and implement effective
    countermeasures prior to adverse impact." Illustrative examples of
    potential objective criteria that could address these criteria are
    "Develop and implement an integrated risk management plan by [a
    specified date]" and "Ensure that [a specified percentage] of the risks
    identified have a countermeasure available and implemented within [a
    specified number of days] of the mustard container campaign."

    Similarly, the current award fee plan contains the following subjective
    criterion for evaluating cost performance: "Cost management systems
    (accounting, scheduling, planning, and budgeting) are all efficiently
    and effectively integrated, and produce consistent, reliable, and
    accurate data." An illustrative example of a potential objective
    criterion that could address at least a portion of these criteria is
    "Ensure that [a specified percentage] of the cost and schedule
    management systems data accurately depicts contract cost status within
    the assessment period." Moreover, the current award fee plan contains
    the following subjective criterion for evaluating surety performance:
    "Appropriate treatment and disposal of chemical agents and chemical
    agent standards." An illustrative example of a potential objective
    criterion that could address at least a portion of these criteria is
    "Ensure that [a specified percentage] of chemical agents are
    appropriately treated and disposed of under the chemical agent
    standards."

    After the March 2006 guidance on award fees was issued by the Under
    Secretary of Defense for Acquisition, Technology, and Logistics, CMA
    issued a policy on award fees in September 2006. CMA's policy stated
    that "due to the complexity of the program and the toxic nature of the
    material being processed, essential elements of performance cannot
    effectively or feasibly be measured by predetermined objective metrics.
    Therefore, award fee criteria are established that allow assessment of
    performance through the use of objective and/or subjective metrics."
    Our review of this policy indicated that CMA did not clearly or fully
    incorporate the concept of tying "award fees to identifiable interim
    outcomes, discrete events or milestones as much as possible" as called
    for in the March 2006 guidance. Although not legally required, we
    believe that tying award fees as much as possible to more objective,
    measurable evaluation criteria is a sound business and management
    practice, and that CMA could do more to fully incorporate and implement
    the March 2006 DOD policy guidance.

    Recently Established and Planned Performance-Based Incentive Fees Are
    Focused on Schedule, but Their Effectiveness Is Uncertain at This Time:

    The Army has recently offered monetary fees intended to provide
    additional incentives to improve schedule performance. In 2005, the
    Army added to its fee structure the DPPBI, which is an annual
    performance-based incentive fee that is structured to encourage the
    systems contractors at the four incineration sites to work
    collaboratively to improve various aspects of their performance. That
    is, while each individual site is evaluated and scored separately, the
    individual scores are averaged to determine a composite score that
    results in all sites earning the same percentage of the available DPPBI
    fee pool. Since the initial evaluation period in 2005, the fee amount
    available for the DPPBI has been 2 percent of the estimated fee-bearing
    contract costs--relatively small when compared to the award fee
    available to the incineration sites. The 2 percent DPPBI is in addition
    to the 10 percent available through base and award fees.

    The evaluation factors for assessing systems contractor performance for
    the DPPBI have changed over time and are divided between objective
    criteria and a subjective assessment of systems contractor performance
    during the fiscal year. Three criteria were established for the initial
    DPPBI evaluation period of January 1 through September 30, 2005: risk
    mitigation, production assessment, and lessons learned. In the initial
    evaluation period, the contractors collectively were deemed to have
    earned 51 percent of the 2005 DPPBI pool or $3,313,954 out of
    $6,497,949. In fiscal year 2006, the DPPBI criteria were changed to put
    more emphasis on performance outputs and less on processes and
    procedures. The fiscal year 2006 DPPBI criteria were established to
    encourage the contractors to collectively reduce project schedules
    while maintaining a high standard of surety, safety, and environmental
    compliance. The criteria also were established to increase CMA's
    confidence levels that the sites could achieve CWC destruction
    deadlines, although discrete goals were not specified. For the fiscal
    year 2006 period of performance, the contractors earned about 93
    percent or $8,771,200 out of the $9,411,159 in the DPPBI pool for that
    period.

    Prior to fiscal year 2007, the DPPBI was not linked to objective
    schedule performance criteria linked to the April 29, 2012, CWC treaty
    deadline. Beginning in fiscal year 2007, the DPPBI fee evaluation was
    linked to achieving discrete schedule milestones associated with
    achieving the CWC April 29, 2012, deadline. For fiscal year 2007, 70
    percent of the total DPPBI fee available to be earned (or up to a
    maximum of 1.4 percent of the total negotiated contract costs,
    depending on average collective performance) was based on specified
    schedule attainment targets for each site. The remaining 30 percent of
    the available DPPBI fee for fiscal year 2007 is based on subjective
    criteria related to the contractors' performance in the application of
    lessons learned, risk management, programmatic procurement, and other
    collaborative approaches to reduce costs and accelerate destruction of
    the stockpile. This fee would be in addition to the portion of the
    award fee targeted at schedule performance, now about 1 percent of
    total negotiated contract costs at most incineration sites. This
    additional emphasis on schedule is a positive development, but it
    remains unclear to us how effective this incentive fee will be in
    motivating individual systems contractors' schedule performance due to
    the collective manner in which the fee is earned, its relative size,
    and recent focus on objective schedule performance. It is particularly
    unclear to us how individual contractors might be motivated during
    periods when the underperformance of one or more systems contractors
    could greatly reduce the potential to earn the fees by the others.

    Finally, the Army plans to use a recently authorized incentive fee of
    up to $165 million per site in an effort to further motivate the
    systems contractors to accelerate destruction and closure activities.
    Although it had not been placed on contract at the time of our review,
    CMA drafted a plan to offer incentive fees under the authority
    contained in section 923 of the National Defense Authorization Act for
    Fiscal Year 2007.[Footnote 65],[Footnote 66] This plan would allow
    systems contractors at the four incineration sites the opportunity to
    earn fees in addition to the 12 percent available to be earned
    annually. As currently planned, systems contractors will be able to
    earn incentive payments for completion of destruction operations tied
    to dates around the final CWC deadline and for the acceleration of
    facility closure activities, but the effectiveness of this approach is
    uncertain. Under CMA's plan, it would offer the additional incentive
    fee of up to $150 million to each site--up to $100 million for
    destruction operations and up to $50 million for facility closure
    activities completed within specified target ranges. Bonuses totaling
    $15 million would also be available to each site if all sites complete
    destruction operations before the treaty deadline and accelerate
    facility closure activities.

    CMA's plan, which would have to be implemented under each of the
    contracts, has a number of positive features. For example, the plan
    recognizes, as does the statute, that the enterprise of striving to
    capture these incentives is to be conducted consistent with the
    existing obligation to provide for maximum protection for the
    environment, the general public, and the personnel who are involved in
    the destruction of the chemical agents and munitions. The potential
    effectiveness of this particular planned incentive at motivating the
    systems contractors to complete destruction operations around the
    treaty deadline is also uncertain. This incentive arrangement has not
    yet been placed on contract and thus it is too early to assess its
    potential success. Additional details about the two recently
    established incentive fees are in appendix II.

    Conclusions:

    Destroying the nation's remaining stockpile of chemical weapons in a
    safe, efficient, and timely manner is essential to improve the
    potential for the United States to meet CWC obligations and to reduce
    the risk to the public and the environment of a potential catastrophic
    event associated with lethal chemical stockpiles. We recognize that
    maintaining high levels of safety and environmental compliance are
    inextricably linked to success in meeting schedule and cost goals. We
    also recognize that it is highly important that program management,
    schedule and cost estimates, and monetary incentives be soundly
    structured, well aligned with program goals and desired acquisition
    outcomes, reasonable, and reliable. While DOD has taken specific
    actions to strengthen program management, improving the focus of annual
    performance plans beyond a single year and refining and finalizing
    planned actions in the risk management area could greatly enhance the
    usefulness of these management tools for DOD, CMA, the sites, and
    Congress. In addition, without a fully developed and implemented risk
    management approach, CMA cannot ensure that it is fully anticipating
    and mitigating risks going forward. Without such improvements, it would
    be difficult for program officials to accurately track progress,
    effectively initiate timely actions to mitigate risks, and increase the
    potential for program activities to further contribute to the overall
    program goal of safely destroying the chemical weapons stockpile in a
    safe and environmentally compliant manner.

    Similarly, it is important that DOD, CMA, and Congress have reasonable
    schedule and cost estimates with reliable supporting data and other
    assumptions, including when they change significantly, either
    increasing or decreasing funding requirements. With most of the
    chemical agent destruction sites significantly ahead of the 2005
    program office schedule, the program milestones appear to be overly
    conservative, regardless of unexpected risks, such as the rocket fires
    at Umatilla. As a result, the program schedule milestones may no longer
    be a realistic baseline for measuring program performance and warrant
    further review and independent assessment to give DOD and Congress
    greater confidence that the program is being held accountable to a
    reasonable baseline to measure the program's progress. In addition, the
    Chemical Demilitarization Program's revised APBs and its most recent
    SARs--key management and oversight documents--lack up-to-date and
    reviewed schedule milestone information. Furthermore, DOD needs
    accurate and realistic closure cost estimates and other cost inputs for
    each of DOD's chemical demilitarization sites to accurately and
    adequately budget for the program, make the EVM systems useful for
    program oversight, and arrive at reliable cost estimates.

    Finally, from a management perspective, CMA could improve its use of
    award fees to incentivize the systems contractors to try to meet the
    CWC treaty deadline in a safe and environmentally compliant manner,
    particularly at the incineration sites where award fees are
    predominant. We identified opportunities for CMA to continue and
    further advance its recent efforts to emphasize schedule and cost and
    to incorporate more objective measures in its award fee evaluation
    criteria as called for in DOD guidance on award fees. Given the
    maturity of the operations phase, the experience gained by the systems
    contractors, and the composition of the remaining stockpile, CMA may be
    able to adjust the award fee plan to provide better interim incentives
    to accelerate the systems contractors' performance, and do so in a safe
    and environmentally sound manner. Doing so would also have the benefit
    of reducing the security vulnerabilities and other risks of storing
    lethal chemical agents any longer than necessary, which creates
    additional safety and environmental hazards. We recognize that to the
    extent that objective criteria are not feasible, using subjective
    criteria to determine award fees is reasonable and expected, and DOD
    guidance directs that award fee criteria be tied to performance
    measures, such as identifiable interim outcomes, discrete events, or
    milestones, as much as possible. We identified opportunities for CMA to
    improve the effectiveness of its monetary incentives by better aligning
    its award fee policy and evaluation criteria to DOD's March 2006
    guidance.

    Recommendations for Executive Action:

    We are making 13 recommendations for executive action. They are as
    follows.

    To strengthen program management and increase the likelihood that more
    progress can be made in destroying chemical agents, we recommend that
    the Secretary of Defense direct the Secretary of the Army to take the
    following three actions:

    * incorporate baseline and trend data for past performance and
    multiyear performance goals for the future in its annual performance
    plan;

    * develop interim destruction goals, approaches, and milestones that
    are directly linked to overall program goals for meeting CWC deadlines;
    and:

    * establish a time frame for completing and implementing its risk
    management approach, including integration across sites and with DOD.

    To provide more realistic cost estimates for chemical demilitarization
    sites in light of recent experience and information gained, we
    recommend that the Secretary of Defense direct the Secretary of the
    Army to update the 2005 program schedule milestones for each of DOD's
    chemical demilitarization sites to reflect recent site processing rates
    and then adjust the cost estimate accordingly.

    To help ensure that the program baseline schedule is current and
    achievable and to improve the accuracy and reliability of program
    manager estimates for measuring the Chemical Demilitarization Program's
    progress, we recommend that the Secretary of Defense direct the CAIG to
    establish a periodic schedule to review the program's processing
    history to determine whether the schedule milestones are still
    reasonable.

    To increase the ability of decision makers to measure the Army's and
    CMA's performance against schedule growth, we recommend that Secretary
    of Defense direct the Under Secretary of Defense for Acquisition,
    Technology, and Logistics to include in the Chemical Demilitarization
    Program's APB and SAR the updated and reviewed schedule milestones for
    completing destruction operations and facility closure activities for
    each of the chemical agent destruction sites.

    To provide more realistic cost estimates for closure of chemical
    demilitarization sites and to establish, stabilize, and maintain an
    accurate, valid, and current performance management baseline, we
    recommend that the Secretary of Defense direct the Secretary of the
    Army to take the following three actions:

    * coordinate with the systems contractors in defining closure
    requirements and developing accurate and realistic closure cost
    estimates for each of DOD's chemical demilitarization sites' contract
    performance measurement baseline;

    * define all contract requirements, including authorized unpriced work
    associated with facility closure activities, and develop comprehensive
    cost estimates through closure at each chemical destruction site; and:

    * establish a time frame for completing the negotiation for placing all
    authorized work that has not been priced on contract and ensuring that
    the negotiations take place.

    After the cost estimates are complete and schedule milestones have been
    updated and reviewed, we recommend that the Secretary of Defense direct
    the CAIG to conduct an independent cost review of the cost estimates.

    To continue to further the Army's efforts to improve the potential to
    meet the CWC deadline for destroying chemical weapons safely and within
    environmental compliance and reduce the risk of storing lethal chemical
    agents any longer than necessary, we recommend that the Secretary of
    the Defense direct the Secretary of the Army to determine for each of
    the four incineration site contracts whether greater weight can be
    applied to performance measures, such as schedule and cost, in award
    fee plans to further incentivize the systems contractors to increase
    the destruction rate of chemical agents in a safe and environmentally
    sound manner.

    To further link monetary incentives and key acquisition outcomes, and
    to provide a means for objectively measuring the systems contractors'
    performance during the assessment period as much as possible, we
    recommend that the Secretary of Defense direct the Secretary of the
    Army to take the following two actions:

    * better align its award fee policy with DOD's March 2006 guidance on
    award fees and:

    * link more of its award fee criteria to performance measures that
    focus on identifiable interim outcomes, discrete events, or milestones.

    Agency Comments and Our Evaluation:

    In written comments on a draft of this report, DOD concurred or
    partially concurred with 12 of our 13 recommendations. DOD did not
    concur with our recommendation regarding defining closure requirements
    and developing accurate and realistic closure cost estimates for each
    of DOD's chemical demilitarization sites. DOD recommended that we
    delete it, since DOD officials believe that a subsequent recommendation
    defining all contract requirements, including those associated with
    facility closure activities, was more encompassing. Based on DOD's
    comments, rather than deleting the recommendation, we clarified it and
    the related recommendation to more clearly distinguish the difference
    between the two recommendations, and consolidated it with our two other
    recommendations related to defining contract requirements. DOD also
    stated that the department will continue to use acquisition management
    tools and discipline to ensure the destruction of the United States'
    chemical weapons stockpile in a safe and secure manner, while being
    economical and meeting the U.S. commitments under the Chemical Weapons
    Convention. DOD identified a number of actions that it already has
    initiated in response to our recommendations. DOD also provided us with
    technical comments, which we incorporated in the report, as
    appropriate. DOD's comments are reprinted in appendix III.

    Regarding our four recommendations related to strategic planning tools,
    DOD concurred with our recommendation that it incorporate baseline and
    trend data for past performance and multiyear performance goals for the
    future in its annual performance plan. DOD stated that CMA currently is
    incorporating these items into its fiscal year 2008 Annual Performance
    Plan and expects this effort to be completed by December 31, 2007. DOD
    partially concurred with our recommendation that it provide direction
    to develop interim destruction goals, approaches, and milestones that
    are directly linked to overall program goals for meeting CWC deadlines.
    In the spirit of our recommendation, DOD stated that it would ensure
    that all future CMA performance plans incorporate annual goals and
    approaches leading toward the April 29, 2012, destruction deadline
    through direction issued annually in support of its existing strategic
    governance process and that additional direction would not be needed.
    We support this planned action, which, if implemented properly, would
    accomplish the same effect as our recommendation. DOD concurred with
    our recommendation that the Army establish a time frame for completing
    and implementing its risk management approach, including integration
    across sites and with DOD. DOD stated that it expects an approved risk
    management approach to be completed by June 30, 2008. DOD concurred
    with our recommendation to include in the Chemical Demilitarization
    Program's APB and SAR the updated and reviewed schedule milestones for
    completing destruction operations and facility closure activities for
    each of the chemical agent destruction sites. DOD stated that it would
    request administrative changes to remove placeholder terminology from
    these documents and replace them with the CAIG's schedule estimates.

    Regarding our two recommendations related to schedule and cost
    estimates, DOD partially concurred with our recommendation that it
    update the 2005 program schedule milestones for each of DOD's chemical
    demilitarization sites to reflect recent site processing rates and then
    adjust the cost estimate accordingly. DOD stated that it has gained
    only limited information during the past year regarding processing
    rates, unknown risks, and the potential for a chemical event, and thus
    does not believe it would be prudent to adjust cost and schedule
    estimates or rebaseline at this time. However, DOD will evaluate the
    cost and schedule estimates for the entire program during the upcoming
    Fiscal Year 2010-2015 Program Objectives Memorandum build process and
    program review, and will determine whether an update is warranted. DOD
    concurred with our recommendation to establish a periodic schedule for
    the CAIG to review the program's processing history to determine
    whether the schedule milestones are still reasonable. DOD stated that
    it conducts a biannual program review of the Chemical Demilitarization
    Program and that during the next review in the spring of 2008, it would
    evaluate the program's cost and schedule estimates. If significant
    differences exist between the program's baseline and the revised
    estimates, then DOD would request a CAIG review that could result in
    new cost and schedule estimates. However, we believe that our report
    provides clear and sufficient evidence demonstrating that there are
    differences between the program's current schedule and cost estimates
    and DOD's baseline to warrant a CAIG review and update as we
    recommended.

    Regarding our four recommendations related to contract requirements,
    based on DOD's comments, we consolidated and clarified two of our
    recommendations. DOD suggested that we delete the recommendation that
    CMA coordinate with the systems contractors in defining closure
    requirements and in developing accurate and realistic closure cost
    estimates for each of DOD's chemical demilitarization sites, since the
    subsequent recommendation is more encompassing. Instead, we clarified
    this recommendation by adding "for each of DOD's chemical
    demilitarization sites' contract performance measurement baseline"
    [emphasis added], and consolidated it with our two other
    recommendations related to defining contract requirements discussed
    below. As discussed in our report, we continue to believe that CMA and
    the systems contractors must coordinate their efforts to develop
    accurate and realistic closure cost estimates. In addition, we
    consolidated our recommendation that after the cost estimates are
    complete and schedule milestones have been updated and reviewed, the
    CAIG conduct an independent cost review of the cost estimates with our
    two recommendations that are discussed below. DOD partially concurred
    with our recommendation to define all contract requirements, including
    those associated with facility closure activities, and develop
    comprehensive cost estimates through closure at each chemical
    destruction site. Based on DOD's comments, we clarified this
    recommendation by adding "define all contract requirements, including
    authorized unpriced work associated with facility closure activities"
    [emphasis added]. DOD stated that since June 2007, three of the five
    CMA destruction sites have completed contract performance baselines
    that include contract requirements for operations, secondary waste
    processing, closure, and known risk. Negotiations are under way to
    complete the contract performance baseline for the Tooele, Utah, site
    and the contract requirements and cost estimates for the Newport,
    Indiana, site by June 30, 2008. DOD concurred with our recommendation
    to establish a time frame for completing the negotiation for placing
    all authorized work that has not been priced on contract and ensuring
    that the negotiations take place. DOD stated that it expects
    negotiations to be completed by June 30, 2008. Finally, DOD partially
    concurred with our recommendation that the CAIG conduct an independent
    review of the program's cost estimates stating that the program's
    biannual review in support of the budget submission will determine if a
    CAIG independent assessment is warranted. We continue to believe that
    providing an independent review of the program's cost estimates could
    provide greater assurance that cost estimates are reasonable.

    With regard to our three recommendations related to incentive fees, DOD
    concurred with our recommendation to determine for each of the four
    incineration site contracts whether greater weight can be applied to
    performance measures, such as schedule and cost, in award fee plans to
    further incentivize the systems contractors to increase the destruction
    rate of chemical agents in a safe and environmentally sound manner. DOD
    also concurred with our recommendation to better align CMA's award fee
    policy with DOD's March 2006 guidance on award fees, stating that this
    action had been completed on September 7, 2007. However, when a copy of
    the revised policy was requested, DOD stated that its response
    incorrectly referenced that this action has been completed. As such,
    DOD will direct a revision of the CMA policy that aligns it with the
    DOD guidance on award fees, and the action is to be completed by March
    31, 2008. DOD partially concurred with our recommendation to link more
    of its award fee criteria to performance measures that focus on
    identifiable interim outcomes, discrete events, or milestones. DOD
    stated that it would conduct a review of CMA award fee plans, to be
    completed by March 31, 2008, and would then determine whether further
    direction and action is required. While this review represents a
    positive step, our report shows that there are several areas in which
    DOD could better link more of its award fee criteria with performance
    measures that focus on identifiable outcomes and we believe that DOD
    should fully implement this recommendation.

    As agreed with your offices, unless you publicly announce the contents
    of this report earlier, we plan no further distribution of it until 30
    days from the date of this letter. We will then send copies of this
    report to the Chairmen and Ranking Members of the Senate Committee on
    Armed Services; Senate Committee on Homeland Defense and Governmental
    Affairs; Subcommittee on Defense, Senate Committee on Appropriations;
    House Committee on Oversight and Government Reform; and Subcommittee on
    Defense, House Committee on Appropriations, and other interested
    congressional parties. We are also sending copies of this report to the
    Secretary of Defense; the Secretary of the Army; the Director, Office
    of Management and Budget; and the Director, U.S. Army Chemical
    Materials Agency. We also will make copies available to others on
    request. In addition, this report will be available at no charge on the
    GAO Web site at [hyperlink, http://www.gao.gov].

    If you or your staff have any questions concerning this report, please
    contact me at (202) 512-5431 or dagostinod@gao.gov. Contact points for
    our Offices of Congressional Relations and Public Affairs may be found
    on the last page of this report. GAO staff who made key contributions
    to this report are listed in appendix IV.

    Signed by:

    Davi M. D'Agostino:

    Director, Defense Capabilities and Management:

    [End of section]

    Appendix I: Scope and Methodology:

    To assess the progress the Department of Defense (DOD) and the Army
    have made in implementing our prior recommendations to strengthen
    program management, whether it has been sufficient, and whether actions
    taken to identify and mitigate the risk of future program schedule
    extensions and cost growth have been effective, we obtained and
    reviewed strategic and implementation plans and interviewed cognizant
    officials to identify whether key elements were present, such as a
    program mission statement, long-term goals and objectives, delineation
    of roles and responsibilities of DOD and Army offices, and near-term
    performance measures.

    To assess the reasonableness of the methodology used to determine
    schedule milestones as well as the reliability of cost estimates, we
    reviewed current program estimates, destruction schedules, earned value
    management (EVM) data, and other documents. We also obtained and
    reviewed the program's risk management plans and related documents. In
    addition, we also identified the issues that had caused delays and
    ascertained approaches being used to reduce the potential for delays in
    the future.

    The Army's Chemical Materials Agency (CMA) provided us with EVM reports
    submitted to the Army by the systems contractors for each of the
    chemical demilitarization facilities spanning the period from December
    2000 through February 2007. EVM reports were provided electronically in
    the form of Deltek wInsightTM transfer files for Cost Performance
    Report Formats 1 through 4 and separate Adobe�, MS Word, or MS Excel
    files for Format 5.

    Our approach in assessing the adequacy of the systems contractors' EVM
    reporting was to analyze the data provided according to EVM analysis
    tasks and DOD earned value implementation guidance. The GAO EVM
    analysis tasks are a set of 10 analysis tasks developed by GAO that are
    used to assess the adequacy of EVM. These 10 analysis tasks were
    performed on each of the EVM reports provided to us. Guidance and
    procedures outlined within the DOD Earned Value Management
    Implementation Guide were also referenced to assess EVM system and
    reporting adequacy.

    Our approach in assessing the Army's ability to assess, synthesize, and
    incorporate systems contractor historical and estimated cost,
    technical, and schedule performance into the current program estimate
    was to align historical contract performance data as documented within
    the EVM reports with program estimate and baseline data documented
    within Selected Acquisition Reports, the President's budget, and the
    Fiscal Year 2005 Army Cost Position.

    In performing this assessment, we also conducted interviews with
    government and systems contractor officials at CMA headquarters in
    Aberdeen, Maryland; Tooele Chemical Agent Disposal Facility, Deseret
    Chemical Depot (Tooele), Utah; and Umatilla Chemical Agent Disposal
    Facility, Umatilla Chemical Depot (Umatilla), Oregon.

    To analyze the effectiveness of DOD's and the Army's use of monetary
    incentives to improve systems contractors' performance, we obtained and
    reviewed contract documentation related to the implementation of the
    contracts' incentives, including the basic contract and modifications,
    award fee plans, award fee determining official decisions, award fee
    evaluation board reports, and pre-and postnegotiation memorandums, and
    the performance-based incentive fees, including the Director's
    Programmatic Performance Based Incentive (DPPBI) fee. We also
    interviewed DOD and Army officials to determine the types of award and
    incentive fees offered at the incineration sites and at Newport
    Chemical Agent Disposal Facility, Newport Chemical Depot (Newport),
    Indiana.

    To evaluate the effectiveness of the award fees in controlling costs
    and schedule, we reviewed the award fee plans from fiscal year 2003 to
    fiscal year 2007 to identify the types of performance measures and
    weight factors used to incentivize the systems contractors to safely
    destroy the chemical weapons stockpile. We also looked at the extent
    the criteria used for the performance measures were linked to
    measurable outcomes. To identify the percentage of award fees paid to
    the systems contractors at the incineration sites and Newport, we
    collected and reviewed data on the award fee payments, including the
    amount available for the contactors to earn, the amounts earned from
    fiscal year 2003 through fiscal year 2006, and the guidance for
    improving the use of these fees. We used data on the award fee plans
    from fiscal year 2003 through fiscal year 2007 because they represented
    the period of time the sites were in the operations phase.

    To evaluate the extent documentation was prepared for adjustments to
    the award fee plans, we reviewed CMA's Integrated Process Team
    documents from fiscal year 2003 to fiscal year 2007 to identify
    adjustments made by the various stakeholders, such as the CMA Director,
    U.S. Army Sustainment Command, CMA Program and Policy Office,
    incinerator site project managers, and systems contractor site general
    managers. We evaluated whether Army officials provided complete
    documentation supporting their rationale for adjusting performance
    measures and related weights throughout the process. We also evaluated
    whether CMA prepared adequate supporting documentation to enable
    reviewers to understand and evaluate the process that was used, and
    whether it contained, among other things, an adequate discussion of
    alternative performance measures or the combination of award and
    incentive fees that it considered, tested, and rejected and the reasons
    for rejecting them.

    To evaluate the effectiveness of the authority contained in section 923
    of the National Defense Authorization Act for Fiscal Year 2007, we
    obtained and reviewed the incentive agreement plan and incentive fees
    for the Aberdeen Chemical Agent Disposal Facility (Aberdeen), Maryland,
    and Newport to determine the challenges those sites faced in
    negotiating the incentive fee agreements. Also, we interviewed
    officials with the U.S. Army Sustainment Command, CMA, and CMA and
    systems contractors at Tooele and Umatilla.

    We also met with DOD and Army program officials and interviewed
    officials at two of the chemical agent destruction sites. Finally, this
    report focuses primarily on the seven CMA-managed chemical destruction
    sites--Aberdeen; Anniston Chemical Agent Disposal Facility, Anniston
    Chemical Activity, Alabama; Johnston Atoll in the Pacific Ocean;
    Newport; Pine Bluff Chemical Agent Disposal Facility, Pine Bluff
    Chemical Activity, Arkansas; Tooele; and Umatilla--because most are in
    the operational phase and represent about 82 percent of the remaining
    stockpile of chemical agents to be destroyed, and have reached a level
    of maturity in processing a variety of types of munitions and agents
    that will help meet Chemical Weapons Convention (CWC) goals. This
    report does not focus on two sites--Aberdeen and Johnston Atoll--that
    have already completed operations and have either been closed or are
    being closed. It also does not focus on the two chemical agent
    destruction sites that are part of the Assembled Chemical Weapons
    Alternatives program--Blue Grass Chemical Agent-Destruction Pilot
    Plant, Blue Grass Chemical Activity (Blue Grass), Kentucky, and Pueblo
    Chemical Agent-Destruction Pilot Plant, Pueblo Chemical Depot,
    Colorado--that currently are in the design and construction phase to
    destroy chemical agents, report directly to DOD rather than to the
    Army, and are not expected to help meet CWC goals.

    We examined the reliability of the data used in this report by
    verifying EVM system certification, applying analytical tests to the
    EVM data, and examining EVM system effectiveness in providing
    meaningful performance measurement to program management. We found that
    EVM systems at each site had been certified. The results of our
    analytical tests found the data to be reasonably free of data errors
    and anomalies. However, when we examined the effectiveness of the EVM
    system to provide meaningful performance measurement, we found
    insufficient reliability to address our objectives.

    We conducted this performance audit from May 2006 through July 2007 in
    accordance with generally accepted government auditing standards.

    [End of section]

    Appendix II: Two Recently Introduced Performance-Based Incentive Fees:

    The Army's performance-based incentive fee to collaboratively improve
    schedule and cost performance was established in 2005 because the use
    of award fees had not been successful. Also, the Army plans to use a
    recently authorized $165 million incentive fee to accelerate
    destruction and closure activities to dates linked with the treaty
    schedule.

    Army's Performance-Based Incentive Fee Begun in 2005 Was Recently
    Targeted to Objective Performance:

    In 2005, CMA established the DPPBI fee to improve schedule and cost
    performance because CMA's use of award fees had not been successful in
    controlling cost and schedule at the incineration sites. This incentive
    fee is 2 percent of the total 12 percent available fees, which includes
    the 3 percent base fee that is not performance related. DPPBI was
    structured so that the collective performance of the systems
    contractors is evaluated and rewarded through improved collaboration
    among the systems contractors that operate the four incineration sites.
    That is, while each individual site is evaluated and scored separately,
    the individual scores are averaged to determine a composite score that
    results in all sites earning the same percentage of the available DPPBI
    fee pool.

    The evaluation factors for assessing the systems contractor performance
    for DPPBI have changed over time and are divided between objective
    criteria and a subjective assessment of systems contractor performance
    during the fiscal year. Three criteria were established for the initial
    DPPBI evaluation period of January 1 through September 30, 2005: risk
    mitigation, production assessment, and lessons learned. In the initial
    evaluation period, the contractors collectively were deemed to have
    earned 51 percent of the 2005 DPPBI pool or $3,313,954 out of
    $6,497,949. In fiscal year 2006, the DPPBI criteria were changed to put
    more emphasis on performance outputs and less on processes and
    procedures. The fiscal year 2006 DPPBI criteria were established to
    encourage the contractors to reduce project schedules while maintaining
    a high standard of surety, safety, and environmental compliance. The
    criteria also were established to increase CMA's confidence levels that
    the sites could achieve CWC destruction deadlines, although discrete
    goals were not specified. For the fiscal year 2006 period of
    performance, the contractors earned about 93 percent or $8,771,200 out
    of the $9,411,159 in the DPPBI pool for that period.

    Until fiscal year 2007, the DPPBI fee evaluation was not linked to
    objective schedule performance criteria linked to the April 29, 2012,
    CWC extended treaty deadline. In fiscal year 2007, schedule performance
    attainment was added to the DPPBI objective criteria amounting to 70
    percent of the total available DPPBI fee (or up to 1.4 percent for
    average collective performance) and amounting to 30 percent for
    subjective criteria related to the contractors' performance in the
    application of lessons learned, risk management, programmatic
    procurement, and other collaborative approaches to reduce cost and
    accelerate destruction of the U.S. stockpile. The amount of the fee
    that can be earned by the systems contractors depends not just on a
    systems contractor's individual performance, but also on the
    performance of the other systems contractors. For example, if the
    systems contractor at Anniston is performing well in meeting its
    schedule-related goals during a specific period, but the systems
    contractor at Tooele is not, then the Anniston systems contractor would
    be encouraged to offer assistance to the Tooele systems contractor. If
    the Tooele systems contractor's schedule performance could not be
    improved, then the Anniston systems contractor's potential to earn this
    fee would be reduced as would the potential of systems contractors at
    the other sites.

    Recently Authorized Incentives Are Targeted to Completing Destruction
    Operations around the Treaty Deadline and Accelerating Facility Closure
    Activities:

    CMA plans to offer incentive fees under the authority contained in
    section 923 of the National Defense Authorization Act for Fiscal Year
    2007,[Footnote 67] permitting systems contractors the opportunity to
    earn incentive payments for the completion of destruction operations by
    or near the CWC deadline and the acceleration of facility closure
    activities, but the effectiveness of this approach is uncertain. The
    purpose of section 923 is to provide the systems contractor for a
    chemical demilitarization facility an incentive to accelerate the safe
    elimination of the U.S. chemical weapons stockpile and to reduce the
    total cost of the Chemical Demilitarization Program by providing
    incentive payments for the early completion of destruction operations
    and the closure of such a facility.[Footnote 68] Section 923 authorizes
    the inclusion of incentives clauses in chemical demilitarization
    contracts for the completion of both destruction operations and
    facility closure activities within target incentive ranges. Target
    incentive ranges are to be jointly agreed upon by the contracting
    officer and the systems contractor concerned and are to be specified in
    the contract's incentives clause. Under the statute, the amount of
    incentive payment a systems contractor may earn through such incentive
    clauses will be based on how early within the specified target
    incentive range the systems contractor completes destruction operations
    and facility closure activities. Section 923 sets limitations on the
    amount of incentive payments a systems contractor may receive through
    the incentives clauses. It establishes $110 million as the maximum
    incentive payment for the completion of destruction operations within
    the target incentive range specified under the contract's incentives
    clause and $55 million for the completion of facility closure
    activities within the target incentive range specified under the
    contract's incentives clause.[Footnote 69]

    Although it had not been implemented at the time of our review, CMA had
    drafted a plan for implementation of the incentives authorized under
    section 923.[Footnote 70] The plan articulates that the systems
    contractors are to assume responsibility for identifying and mitigating
    all site-specific and programmatic risks in order to successfully
    complete program objectives within budgeted cost and schedule targets.
    The plan further relates that in return, the systems contractors will
    receive incentives measured against established schedule targets for
    operations completion and facility closure.

    Under CMA's plan, it would offer the additional incentive fee of up to
    $150 million to each site--$100 million for destruction operations
    completed within target ranges by or near the April 29, 2012, treaty
    deadline, and $50 million for facility closure activities brought
    successfully to completion within target ranges. CMA's plan establishes
    the period from November 1, 2011, through October 31, 2013, as the
    target incentive range for the completion of operations at all sites,
    with the maximum amount available to be earned, exclusive of the bonus,
    at $100 million. Ninety-five percent of the incentive can be earned for
    completing operations by the CWC extended deadline of April 29, 2012.
    After the CWC deadline, a percentage of the incentives can be earned
    through October 31, 2013, on a reduced sliding scale. The CMA plan also
    offers up to $50 million per site for the completion of facility
    closure activities within a specified range of dates unique to each
    site. One hundred percent of the $50 million would be earned by a
    systems contractor for completion of facility closure activities on the
    first date of the specified range. The systems contractor would earn
    some lesser percentage of the $50 million for completion on any date
    between the first and last dates of the specified range.

    Under CMA's plan, systems contractors can earn up to $165 million at
    their respective sites through bonus potential. In addition to the
    potential $100 million fee for completing operations ahead of the
    extended CWC deadline and $50 million for meeting specified closure
    targets, bonuses can also be earned. A bonus of $10 million is offered
    to each site if all the sites complete destruction operations prior to
    the CWC deadline of April 29, 2012. A bonus of $5 million is also
    offered to each site for accelerated completion of facility closure
    activities.

    CMA's plan, which would have to be implemented under each of the
    contracts, includes language that insulates the government from having
    to make incentive payments to the systems contractors if they did not
    strictly perform under the terms CMA has set under the plan. In firmly
    establishing the target incentive ranges, the plan states that the
    target dates are not subject to extension except for catastrophic
    causes beyond the control of and without the fault or negligence of the
    systems contractor, its subcontractors, and suppliers. The plan
    establishes that the systems contractors are not entitled to submit
    claims for loss of opportunity to earn the incentives or to increase
    incentive pools due to failure of the government to provide resources
    over and above those specifically identified in the contract. It also
    stipulates that payments of incentives will be subject to adjustment
    based upon final audits of operation costs and closure costs. At the
    same time, the plan recognizes, as does the statute, that the
    enterprise of striving to capture these incentives is to be conducted
    consistent with the existing obligation to provide for maximum
    protection for the environment, the general public, and the personnel
    who are involved in the destruction of the chemical agents and
    munitions. The effectiveness of this planned incentive is uncertain
    because it had not yet been negotiated or placed on contract at the
    time of our review.

    [End of section]

    Appendix III: Comments from the Department of Defense:

    Assistant To The Secretary Of Defense:
    3050 Defense Pentagon:
    Washington, DC 20301-3050:

    Nuclear And Chemical And Biological Defense Programs:

    November 16, 2007:

    Ms. Davi M. D'Agostino:
    Director, Defense Capabilities and Management:
    United States Government Accountability Office:
    441 G Street, N.W.:
    Washington, DC 20548:

    Dear Ms. D'Agostino:

    This is the Department of Defense (DoD) response to the GAO draft
    report, GAO-08-134, `Chemical Demilitarization: Additional Management
    Actions Needed to Meet Key Performance Goals of DoD's Chemical
    Demilitarization Program,' dated October 3, 2007 (GAO Code 350799).

    The DoD concurs with most of the draft report's recommendations, except
    those pertaining to when cost and schedule estimates should be revised.
    The Department will continue to use acquisition management tools and
    discipline to ensure the destruction of the United States' chemical
    weapons stockpile in a safe and secure manner, while being economical
    and meeting our commitments under the Chemical Weapons Convention.

    The Department appreciates the opportunity to provide comments on the
    draft report. These comments, including technical comments, are
    enclosed. For further questions concerning this report, please contact
    Barbara Burgess, Senior Program Analyst for the Chemical
    Demilitarization Program, (703) 588-1983, extension 113.

    Sincerely,

    Signed by:

    Jean D. Reed:
    Special Assistant:
    Chemical and Biological Defense and Chemical Demilitarization Programs:

    Enclosures:
    As stated:

    cc:
    Deputy Assistant Secretary of the Army (Elimination of Chemical
    Weapons):

    GAO Draft Report � Dated October 3, 2007 GAO Code 350799/GAO-08-134:

    "Chemical Demilitarization: Additional Management Actions Needed to
    Meet Key Performance Goals of DoD's Chemical Demilitarization Program"

    Department Of Defense Comments To The Recommendations:

    Recommendation 1: The GAO recommends the Secretary of Defense direct
    the Director of U.S. Army Chemical Materials Agency to incorporate
    baseline and trend data for past performance and multiyear performance
    goals for the future in its annual performance plan.

    DoD Response: Concur. The Department recognizes that U.S. Army Chemical
    Materials Agency (CMA) is currently incorporating baseline and trend
    data for past performance and multiyear performance measures in the CMA
    Annual Performance Plan for 2008. This effort is expected to be
    completed and staffed by end of the first quarter of Fiscal Year 2008.

    Recommendation 2: The GAO recommends the Secretary of Defense direct
    the Director of U.S. Army Chemical Materials Agency to develop interim
    destruction goals, approaches, and milestones that are directly linked
    to overall program goals for meeting Chemical Weapons Convention
    deadlines.

    DoD Response: Partially Concur. The Department recognizes that the U.S.
    Army Chemical Materials Agency (CMA) annual destruction goals are
    interim destruction goals for meeting the Chemical Weapons Convention
    (CWC) destruction milestones. The Chemical Demilitarization Program
    Fiscal Year 2007 Annual Performance Plan provided the annual
    destruction goals for meeting the CWC 45 percent destruction milestone
    of December 31, 2007. The Department will continue to ensure all future
    CMA performance plans incorporate annual goals and approaches leading
    toward the April 2012 destruction deadline through direction issued
    annually in support of strategic governance. Therefore, no further
    direction is required from the Secretary of Defense to develop interim
    goals, approaches, and milestones.

    Recommendation 3: The GAO recommends the Secretary of Defense direct
    the Director of U.S. Army Chemical Materials Agency to establish a time
    frame for completing its risk management approach, including
    integration across sites and with DoD.

    DoD Response: Concur. The U.S. Army Chemical Materials Agency (CMA) is
    currently evaluating and updating the CMA risk management approach
    including integration of risk management across sites and within CMA
    Headquarters. The Department expects an approved risk management
    approach to be completed by the third quarter of Fiscal Year 2008.

    Recommendation 4: The GAO recommends the Secretary of Defense direct
    the Director of U.S. Army Chemical Materials Agency to update the 2005
    program schedule milestones for each of DoD's chemical demilitarization
    sites to reflect recent site processing rates and then adjust the cost
    estimate accordingly.

    DoD Response: Partially Concur. The Department recognizes successes
    within the Chemical Demilitarization Program (CDP) over the past two
    years. However, there are still some areas of the program, (i.e.,
    destruction of secondary hazardous waste and closure of the chemical
    weapons destruction facilities), with limited information to adjust
    cost and schedule estimates. As the GAO stated in their final report:
    NSIAD-00-80, Chemical Weapons Disposal: Improvements Needed in Program
    Accountability and Financial Management, May 2000: "The Army also needs
    additional time to develop a reliable baseline to estimate the costs
    for the closure and remediation of the chemical stockpile disposal
    facilities.Furthermore, because no stockpile disposal facility has yet
    to be remediated, the Army lacks real time experience on which to
    estimate these costs." The CDP now has two sites to compare closure
    activities, Johnston Atoll Chemical Agent Destruction System (JACADS)
    and Aberdeen Chemical Agent Disposal Facility (ABCDF). JACADS completed
    destruction operations in November 2000 and most closure activities in
    June 2004, with the exception of final closure of the Resource
    Conservation and Recovery Act permit, which is not expected until
    November 2008. The ABCDF completed operations in February 2006 and
    closure activities were completed in March 2007. With this limited
    information for closure, the U.S. Army Chemical Materials Agency (CMA)
    has recently completed contract performance baselines that include
    operations, secondary waste processing, closure and known risk, for
    sites at Anniston, Alabama; Pine Bluff, Arkansas; and Umatilla, Oregon.

    In addition, the Department approved the revised Acquisition Program
    Baselines for the Chemical Demilitarization Program � U.S. Army
    Chemical Materials Agency (Chem Demil- CMA) and Chem Demil-CMA Newport
    Major Defense Acquisition Programs in April 2006, with the cost and
    schedule estimates based on the Cost Analysis Improvement Group's
    (CAIG) estimates. The CAIG estimates were certified to Congress in May
    2005 as part of the Nunn- McCurdy unit cost breach. The increase in the
    unit cost was due primarily to more realistic schedule estimates.

    Therefore, the Department does not believe it is prudent to adjust cost
    and schedule estimates or rebaseline at this time due to limited
    information gained during the past year associated with processing
    rates, unknown risks, and the potential for a chemical incident. The
    Department will evaluate the cost and schedule estimates for the entire
    CDP during the upcoming Fiscal Year 2010-2015 Program Objectives
    Memorandum build process and program review, and will determine whether
    an update is warranted.

    Recommendation 5: The GAO recommends the Secretary of Defense direct
    the Chemical Analysis Improvement Group to establish a periodic
    schedule to review the program's processing history to determine
    whether the schedule milestones are still reasonable.

    DoD RESPONSE: Concur. The Department conducts a biannual program review
    of the Chemical Demilitarization Program in support of the budget
    estimate submission. During this review, the program office schedule
    and cost estimates are reviewed in comparison with the approved
    Acquisition Program Baselines to determine whether there are any
    significant differences. If there are significant differences, the
    Department will request the Cost Analysis Improvement Group conduct an
    independent assessment and possibly develop cost and schedule
    estimates. The next review will occur during the Fiscal Year 2010-2015
    Program Objectives Memorandum build process and program review in the
    spring of calendar year 2008.

    Recommendation 6: The GAO recommends the Secretary of Defense direct
    the Under Secretary of Defense for Acquisition, Technology and
    Logistics to include in the Chemical Demilitarization Program's
    Acquisition Program Baseline and Selected Acquisition Report the
    updated and reviewed scheduled milestones for completing destruction
    operations and facility closure activities for each of the chemical
    agent destruction sites.

    DoD Response: Concur. The Department will request administrative
    changes to the Chemical Demilitarization Program-U.S. Army Chemical
    Materials Agency (Chem Demil � CMA) and Chem Demil-CMA Newport
    Acquisition Program Baselines and upcoming Selected Acquisition Reports
    to remove the "To Be Determined (TBDs)," which were used as a
    placeholder for schedule milestones beyond April 2012, and replace with
    the Cost Analysis Improvement Group's schedule estimates.

    Recommendation 7: The GAO recommends the Secretary of Defense direct
    the Director of the U.S. Army Chemical Materials Agency to coordinate
    with the systems contractors in defining closure requirements and
    developing accurate and realistic closure cost estimates for each of
    DoD's chemical demilitarization sites.

    DoD Response: Non-Concur. The Department recommends GAO delete this
    recommendation since Recommendation #9 is more encompassing.

    Recommendation 8: The GAO recommends the Secretary of Defense direct
    the Cost Analysis Improvement Group to conduct an independent cost
    review of the cost estimates.

    DoD Response: Partially Concur. The Department conducts a biannual
    program review of the Chemical Demilitarization Program in support of
    the budget estimate submission. During this review, the program office
    schedule and cost estimates are reviewed in comparison with the
    approved Acquisition Program Baselines to determine whether there are
    any significant differences. If there are significant differences, the
    Department will request the Cost Analysis Improvement Group conduct an
    independent assessment and possibly develop cost and schedule
    estimates. Therefore, no direction to conduct a cost review will be
    given at this time.

    Recommendation 9: The GAO recommends the Secretary of Defense direct
    the Director of the U.S. Army Chemical Materials Agency to define all
    contract requirements, including those associated with facility closure
    activities, and develop comprehensive cost estimates through closure at
    each chemical destruction site.

    DoD Response: Partially Concur. Since June 2007, three of the five U.S.
    Army Chemical Materials Agency chemical weapons stockpile destruction
    sites (Anniston, Alabama; Pine Bluff, Arkansas; and Umatilla, Oregon)
    have completed contract performance baselines that include contract
    requirements for operations, secondary waste processing, closure and
    known risk. Negotiations are under way to complete the contract
    performance baseline by the end of the third quarter of Fiscal Year
    2008 for the Tooele, Utah, site. Negotiations are expected to be
    completed for the contract requirements and cost estimates associated
    with the closure phase for the Newport, Indiana, site by the end of the
    third quarter of Fiscal Year 2008. The Department believes no further
    direction is required.

    Recommendation 10: The GAO recommends the Secretary of Defense direct
    the Director of the U.S. Army Chemical Materials Agency to establish a
    time frame for completing the negotiation for placing all authorized
    work that has not been priced on contract and ensuring that the
    negotiations take place.

    DoD Response: Concur. The U.S. Army Chemical Materials Agency has
    negotiated almost all work presently on contract (current and future)
    resulting in low levels of authorized, unpriced work. The Department
    expects negotiations to be completed by the end of the third quarter of
    Fiscal Year 2008.

    Recommendation 11: The GAO recommends the Secretary of the Defense
    direct the Director of the U.S. Army Chemical Materials Agency to
    determine for each of the four incineration site contracts whether
    greater weight can be applied to performance measures, such as schedule
    and cost, in award fee plans to further incentivize the systems
    contractors to increase the destruction rate of chemical agents in a
    safe and environmentally sound manner.

    DoD Response: Concur. The Department will conduct a review of the award
    fee plans of the U.S. Army Chemical Materials Agency incineration site
    contracts to determine whether the weighting factors are appropriate to
    incentivize the systems contractors to reduce the schedule while
    operating in a safe, secure, and environmentally-sound manner. The
    results will be available by second quarter of FY08.

    Recommendation 12: The GAO recommends the Secretary of the Defense
    direct the Director of the U.S. Army Chemical Materials Agency better
    align its award fee policy with DoD's March 2006 guidance on award
    fees.

    DoD Response: Concur. This action was completed on September 7, 2007,
    in accordance with the March 29, 2006, Memorandum: Subject: Award Fee
    Contracts, from the Under Secretary of Defense for Acquisition,
    Technology and Logistics. CMA revised Policy Statement 29, "Procedure
    for Implementing Contractor Incentives to Include Award Fee (AF) and
    Performance Based Incentive (PBI) Fee."

    Recommendation 13: The GAO recommends the Secretary of the Defense
    direct the Director of the U.S. Army Chemical Materials Agency to link
    more of its award fee criteria to performance measures that focus on
    identifiable interim outcomes, discrete events, or milestones.

    DoD Response: Partially Concur. The Department will conduct a review of
    the U.S. Army Chemical Materials Agency (CMA) award fee plans to
    determine whether there are performance measures that focus on outcomes
    or outputs. This review will be completed along with the review of the
    award fee weighting factors by the end of the second quarter of Fiscal
    Year 2008. Based on the findings of the review, the Department will
    determine whether further direction and action is required. Currently,
    the CMA Fiscal Year 2006 and Fiscal Year 2007 Award Fee plans included
    objective schedule criteria that focused on U.S. tons of chemical agent
    destroyed to complete destruction by April 2012.

    [End of section]

    Appendix IV: GAO Contact and Staff Acknowledgments:

    GAO Contact:

    Davi M. D'Agostino, (202) 512-5431 or dagostinod@gao.gov:

    Acknowledgments:

    Mark A. Pross, Assistant Director; Bonita P. Anderson; Rodell B.
    Anderson; Susan C. Ditto; Jennifer K. Echard; Neil D. Feldman; James C.
    Lawson; Lonnie J. McAllister; Brian L. Octeau; Charles W. Perdue; Karen
    A. Richey; John W. Van Schaik; Cheryl A. Weissman; and Alex M. Winograd
    made key contributions to this report.

    [End of section]

    Related GAO Products:

    Cost Assessment Guide: Best Practices for Estimating and Managing
    Program Costs. GAO-07-1134SP. Washington, D.C.: July 2007 (Exposure
    Draft).

    Chemical Demilitarization: Actions Needed to Improve the Reliability of
    the Army's Cost Comparison Analysis for Treatment and Disposal Options
    for Newport's VX Hydrolysate. GAO-07-240R. Washington, D.C.: January
    26, 2007.

    Chemical Weapons: Sustained Leadership, Along with Key Strategic
    Management Tools, Is Needed to Guide DOD's Destruction Program. GAO-03-
    1031. Washington, D.C.: September 5, 2003.

    Chemical Weapons: Lessons Learned Program Generally Effective but Could
    Be Improved and Expanded. GAO-02-890. Washington, D.C.: September 10,
    2002.

    Chemical Weapons: FEMA and Army Must Be Proactive in Preparing States
    for Emergencies. GAO-01-850. Washington, D.C.: August 13, 2001.

    Chemical Weapons Disposal: Improvements Needed in Program
    Accountability and Financial Management. GAO/NSIAD-00-80. Washington,
    D.C.: May 8, 2000.

    Chemical Demilitarization: Funding Status of the Chemical
    Demilitarization Program. GAO/NSIAD-99-232R. Washington, D.C.: July 29,
    1999.

    Chemical Weapons: DOD Does Not Have a Strategy to Address Low-Level
    Exposures. GAO/NSIAD-98-228. Washington, D.C.: September 23, 1998.

    Chemical Weapons Stockpile: Changes Needed in the Management of the
    Emergency Preparedness Program. GAO/NSIAD-97-91. Washington, D.C.: June
    11, 1997.

    Chemical Weapons and Materiel: Key Factors Affecting Disposal Costs and
    Schedule. GAO/T-NSIAD-97-118. Washington, D.C.: March 11, 1997.

    Chemical Weapons Stockpile: Emergency Preparedness in Alabama Is
    Hampered by Management Weaknesses. GAO/NSIAD-96-150. Washington, D.C.:
    July 23, 1996.

    Chemical Weapons Disposal: Issues Related to DOD's Management. GAO/T-
    NSIAD-95-185. Washington, D.C.: July 13, 1995.

    Chemical Weapons: Army's Emergency Preparedness Program Has Financial
    Management Weaknesses. GAO/NSIAD-95-94. Washington, D.C.: March 15,
    1995.

    Chemical Stockpile Disposal Program Review. GAO/NSIAD-95-66R.
    Washington, D.C.: January 12, 1995.

    Chemical Weapons: Stability of the U.S. Stockpile. GAO/NSIAD-95-67.
    Washington, D.C.: December 22, 1994.

    Chemical Weapons Disposal: Plans for Nonstockpile Chemical Warfare
    Materiel Can Be Improved. GAO/NSIAD-95-55. Washington, D.C.: December
    20, 1994.

    Chemical Weapons: Issues Involving Destruction Technologies. GAO/T-
    NSIAD-94-159. Washington, D.C.: April 26, 1994.

    Chemical Weapons Destruction: Advantages and Disadvantages of
    Alternatives to Incineration. GAO/NSIAD-94-123. Washington, D.C.: March
    18, 1994.

    Arms Control: Status of U.S.-Russian Agreements and the Chemical
    Weapons Convention. GAO/NSIAD-94-136. Washington, D.C.: March 15, 1994.

    Chemical Weapon Stockpile: Army's Emergency Preparedness Program Has
    Been Slow to Achieve Results. GAO/NSIAD-94-91. Washington, D.C.:
    February 22, 1994.

    Chemical Weapons Storage: Communities Are Not Prepared to Respond to
    Emergencies. GAO/T-NSIAD-93-18. Washington, D.C.: July 16, 1993.

    Chemical Weapons Destruction: Issues Affecting Program Cost, Schedule,
    and Performance. GAO/NSIAD-93-50. Washington, D.C.: January 21, 1993.

    Chemical Weapons Destruction: Issues Related to Environmental
    Permitting and Testing Experience. GAO/T-NSIAD-92-43. Washington, D.C.:
    June 16, 1992.

    Chemical Weapons Disposal. GAO/NSIAD-92-219R. Washington, D.C.: May 14,
    1992.

    Chemical Weapons: Stockpile Destruction Cost Growth and Schedule
    Slippages Are Likely to Continue. GAO/NSIAD-92-18. Washington, D.C.:
    November 20, 1991.

    Chemical Weapons: Physical Security for the U.S. Chemical Stockpile.
    GAO/NSIAD-91-200. Washington, D.C.: May 15, 1991.

    Chemical Warfare: DOD's Effort to Remove U.S. Chemical Weapons from
    Germany. GAO/NSIAD-91-105. Washington, D.C.: February 13, 1991.

    Chemical Weapons: Status of the Army's M687 Binary Program. GAO/NSIAD-
    90-295. Washington, D.C.: September 28, 1990.

    Chemical Weapons: Stockpile Destruction Delayed at the Army's Prototype
    Disposal Facility. GAO/NSIAD-90-222. Washington, D.C.: July 30, 1990.

    Chemical Weapons: Obstacles to the Army's Plan to Destroy Obsolete U.S.
    Stockpile. GAO/NSIAD-90-155. Washington, D.C.: May 24, 1990.

    [End of section]

    Footnotes:

    [1] The Convention on the Prohibition of the Development, Production,
    Stockpiling and Use of Chemical Weapons and on Their Destruction is
    commonly known as the Chemical Weapons Convention.

    [2] In 1996, Congress directed DOD to evaluate the use of alternative
    technologies and suspended incineration planning activities at two of
    the nine storage sites with assembled weapons--Blue Grass Chemical
    Agent-Destruction Pilot Plant, Blue Grass Chemical Activity, Kentucky,
    and Pueblo Chemical Agent-Destruction Pilot Plant, Pueblo Chemical
    Depot, Colorado. See Pub. L. No. 104-201, � 142 (1996) and Pub. L. No.
    104-208, � 8065 (1996). In addition, Congress directed that those two
    sites be managed in a program independent from the Army's Chemical
    Demilitarization Program and report directly to DOD instead of to the
    Army. See Pub. L. No. 105-261, � 142 (1998). The two sites are managed
    under a separate program known as the Assembled Chemical Weapons
    Alternatives program, and are not the focus of this report.

    [3] The U.S. deadline for 100 percent destruction of the unitary
    stockpile was extended from April 29, 2007, to April 29, 2012. The CWC
    allows an extension of up to 5 years, and an extension was granted to
    the United States at its request in December 2006. For purposes of this
    report, we refer to the extended CWC deadline of April 29, 2012, as the
    U.S. treaty deadline, unless otherwise indicated.

    [4] A Nunn-McCurdy unit cost breach occurs when a major defense
    acquisition program experiences an increase of at least 15 percent in
    program acquisition unit cost or procurement unit cost above the unit
    costs in the current APB. Unit cost reporting is required by 10 U.S.C.
    � 2433. For programs with unit cost increases of at least 25 percent, a
    certification by the Secretary of Defense is required. Certification
    responsibility has been delegated to the Under Secretary of Defense for
    Acquisition, Technology, and Logistics, who, by law, must certify that
    (1) such an acquisition program is essential to national security; (2)
    there are no alternatives to such an acquisition program, which will
    provide equal or greater military capability at less cost; (3) the new
    estimates of the program acquisition unit cost or procurement unit cost
    are reasonable; and (4) the management structure for the acquisition
    program is adequate to manage and control program acquisition unit cost
    or procurement unit cost. See 10 U.S.C. � 2433 (b)(2)(A) (2007).

    [5] Total program cost estimates exceeded APBs in 2001 and again in
    2005. On May 2, 2002, the Under Secretary of Defense for Acquisition,
    Technology, and Logistics--the Defense Acquisition Executive--
    submitted a Nunn-McCurdy breach certification to Congress for the
    Chemical Demilitarization Program as required by 10 U.S.C. � 2433. The
    unit cost breach was due primarily to (1) revised processing rates; (2)
    schedule extensions; (3) new environmental regulations; (4) worse-than-
    expected condition of the stockpile; (5) increase in equipment, labor
    rates, and construction costs; and (6) higher emergency preparedness
    costs. On May 6, 2005, the Under Secretary of Defense for Acquisition,
    Technology, and Logistics again certified to Congress that the program
    acquisition unit costs for the Chemical Demilitarization Program had
    increased 41 percent (in then-year dollars) due primarily to more
    realistic schedule estimates associated with operations and closure
    phases, environmental regulatory compliance, and concept/design
    maturation. Program acquisition unit costs for the Newport Chemical
    Agent Disposal Facility, Newport Chemical Depot, Indiana, program
    increased by 52 percent (in then-year dollars) due primarily to more
    realistic schedule estimates associated with the nerve agent
    neutralization process and secondary hazardous waste generation.

    [6] The Chemical Stockpile Elimination Project includes the program's
    four operating incineration sites and two bulk-agent-only
    neutralization sites.

    [7] These four incineration sites; the neutralization site at Newport
    Chemical Agent Disposal Facility, Newport Chemical Depot, Indiana; and
    the two closed sites that were at Johnston Atoll in the Pacific Ocean
    and Aberdeen Chemical Agent Disposal Facility, Maryland, are managed
    for the Army by the Chemical Materials Agency, which was formerly known
    as the Program Manager for Chemical Demilitarization.

    [8] GAO, Chemical Weapons: Sustained Leadership, Along with Key
    Strategic Management Tools, Is Needed to Guide DOD's Destruction
    Program, GAO-03-1031 (Washington, D.C.: Sept. 5, 2003).

    [9] GAO, Defense Acquisitions: DOD Wastes Billions of Dollars through
    Poorly Structured Incentives, GAO-06-409T (Washington, D.C.: Apr. 5,
    2006).

    [10] GAO, Defense Acquisitions: DOD Has Paid Billions in Award and
    Incentive Fees Regardless of Acquisition Outcomes, GAO-06-66
    (Washington, D.C.: Dec. 19, 2005).

    [11] The Office of Management and Budget (OMB) defines EVM as a project
    management tool that effectively integrates the project scope of work
    with cost, schedule, and technical performance elements for optimal
    project planning and control. Beginning in August 2005, OMB
    specifically required federal agencies and contractors to use EVM
    systems on all new capital assets. See Office of Management and Budget,
    Capital Programming Guide (Washington, D.C.: 2006). Also, DOD requires
    the use of EVM on all programs greater than $20 million.

    [12] GAO's prior recommendations are found in GAO-03-1031, p. 26.

    [13] GAO, Cost Assessment Guide: Best Practices for Estimating and
    Managing Program Costs,GAO-07-1134SP (Washington, D.C.: July 2007,
    Exposure Draft).

    [14] The Department of Defense Authorization Act for Fiscal Year 1986,
    Pub. L. No. 99-145, � 1412(a) (1985).

    [15] Nerve agents are the most toxic and rapidly acting of known
    chemical warfare agents. GB, also known as sarin, and VX are human-made
    lethal chemical warfare nerve agents that affect the nervous system by
    interfering with the signals sent from the brain to vital organs.

    [16] Blister agents include mustard, which is a type of chemical
    warfare agent and powerful irritant that causes blistering of the skin
    and mucous membranes on contact.

    [17] Pub. L. No. 104-201, � 142 (1996).

    [18] Since the 1930s, the Army and the chemical industry have used ton
    containers to store and ship bulk chemicals, including chemical agent.
    Equivalent in length and diameter to two stacked 55-gallon drums, a ton
    container weighs approximately 1,600 pounds and measures nearly 7 feet
    in length.

    [19] The stockpile at the Newport Chemical Depot, Indiana, is the first
    U.S. stockpile containing VX that will be destroyed by using
    neutralization--a process that mixes hot water and sodium hydroxide (a
    caustic chemical) with VX to change the chemical composition to a less
    toxic form. The resulting by-product is a liquid wastewater commonly
    referred to as hydrolysate that consists mostly of water but also has a
    caustic component and organic salts that need further treatment to meet
    CWC requirements and to meet federal and state environmental
    requirements for disposal. The hydrolysate is being stored in
    intermodal storage containers on-site until a post-treatment plan can
    be implemented. For additional information on the disposal of
    hydrolysate, see GAO, Chemical Demilitarization: Actions Needed to
    Improve the Reliability of the Army's Cost Comparison Analysis for
    Treatment and Disposal Options for Newport VX Hydrolysate, GAO-07-240R
    (Washington, D.C.: Jan. 26, 2007).

    [20] Pub. L. No. 104-201, � 142 (1996).

    [21] Pub. L. No. 105-261, � 142 (1998).

    [22] The CWC implementing legislation provides the statutory authority
    for domestic compliance with the convention's provisions. See Pub. L.
    No. 105-277, Division I (1998).

    [23] This report solely focuses on the weapons the convention defines
    as category 1, which are the most dangerous chemicals in the stockpile.

    [24] In 2004, DOD split the Chemical Demilitarization Program into
    three major defense acquisition programs: (1) all CMA chemical agent
    destruction sites, except for the Newport site; (2) the Newport site;
    and (3) the ACWA program.

    [25] GAO-03-1031.

    [26] Pub. L. No. 108-375 � 931 (2004) directed the Under Secretary of
    Defense for Acquisition, Technology, and Logistics and the Secretary of
    the Army to jointly prepare a strategic plan for future activities for
    the destruction of the U.S. stockpile of lethal chemical agents and
    munitions. Section 931 further directed that the plan include, at a
    minimum, realistic budgeting for stockpile destruction and related
    support programs, contingency planning for foreseeable or anticipated
    problems, and a management approach and associated actions that address
    compliance with the obligations of the United States under the CWC
    treaty and that take full advantage of opportunities to accelerate
    destruction of the stockpile.

    [27] The performance plan applies to both CMA and the ACWA portions of
    the demilitarization program. It was the program's first plan and was
    issued jointly by both organizations.

    [28] GAO, Agency Performance Plans: Examples of Practices That Can
    Improve Usefulness to Decisionmakers, GAO/GGD/AIMD-99-69 (Washington,
    D.C.: Feb. 26, 1999).

    [29] The Probability of Program Success Report is designed to improve
    the Army's ability to accurately assess the program's probability of
    success, and clearly and concisely represent that success probability
    to leadership.

    [30] See, for instance, GAO, Executive Guide: Effectively Implementing
    the Government Performance Results Act, GAO/GGD-96-118 (Washington,
    D.C.: June 1996).

    [31] OMB's PART was developed to assess and improve program performance
    so that the federal government can achieve better results. A review
    using this tool helps identify a program's strengths and weaknesses to
    inform funding and management decisions aimed at making the program
    more effective. The tool, therefore, looks at all factors that affect
    and reflect program performance, including program purpose and design;
    performance measurement, evaluations, and strategic planning; program
    management; and program results. Because the tool includes a consistent
    series of analytical questions, it allows programs to show improvements
    over time, and allows comparisons between similar programs. See Office
    of Management and Budget, Program Assessment Rating Tool (PART),
    [hyperlink, http://www.whitehouse.gov/omb/part/].

    [32] GAO/GGD/AIMD-99-69.

    [33] GAO-07-1134SP.

    [34] A histogram is a graphical display of tabulated frequencies.

    [35] The OSD CAIG has specific responsibility for providing independent
    cost estimates when requested by the Under Secretary of Defense for
    Acquisition, Technology, and Logistics in support of major milestone
    decision points (Milestone B, Milestone C, or the full-rate production
    decision review). Overall, CAIG serves as the principal advisory body
    to the Milestone Decision Authority on all matters concerning an
    acquisition program's life cycle cost.

    [36] Defense Contract Management Agency, Department of Defense Earned
    Value Management Implementation Guide (Washington, D.C.: Apr. 7, 2005).
    Also see DOD Memorandum, Revision to DOD Earned Value Management Policy
    (Washington, D.C.: Mar. 7, 2005), and Office of Management and Budget,
    Capital Programming Guide (Washington, D.C.: 2006).

    [37] EVM is a process required by OMB that can help program managers
    track program progress by using the value of work done as a basis for
    estimating the cost to compete the project.

    [38] Defense Contract Management Agency, Department of Defense Earned
    Value Management Implementation Guide.

    [39] GAO-07-1134SP.

    [40] Defense Contract Management Agency, Department of Defense Earned
    Value Management Implementation Guide.

    [41] In its technical comments on a draft of this report, DOD expressed
    concern about our statement that the EVM system used at each site may
    not be reliable. As a result, the department stated that it will
    request the Defense Contract Management Agency to conduct an audit of
    the CMA EVM system beginning no later than June 30, 2008.

    [42] The systems contractor and government have identified work that
    (authorized but unpriced) is negotiated and added to the contract price
    annually.

    [43] Defense Contract Management Agency, Department of Defense Earned
    Value Management Implementation Guide. Also, see GAO-07-1134SP.

    [44] In its technical comments on a draft of this report, DOD expressed
    concern about our statement that shortcomings in the underlying (EVM)
    data include invalid and unstable performance baselines. DOD stated
    that the issues with the contract baselines are issues of program
    management or risk, not issues of EVM data or process reliability. DOD
    stated that this problem has been ameliorated by the recent completion
    of life cycle contract negotiations at three of the five sites covered
    by our report (Anniston, Pine Bluff, and Umatilla). The contract
    negotiations for Newport and Toole are ongoing and should be completed
    by June 30, 2008. The department stated that it will request the
    Defense Contract Management Agency to conduct an audit of the CMA EVM
    system beginning no later than June 30, 2008.

    [45] "Comprehensive" means that the cost estimate is complete and
    accounts for all possible costs. Comprehensive cost estimates
    completely define the program, reflect the current schedule, and are
    technically reasonable. See GAO-07-1134SP.

    [46] Johnston Atoll was closed in December 2003 and Aberdeen was closed
    in May 2007.

    [47] The systems contractor and government have identified work that
    (authorized but unpriced) is negotiated and added to the contract price
    annually.

    [48] The contracts use Operating and Maintenance funds, which are
    available for obligation for 1 fiscal year. This creates the need to
    negotiate the contracts on a yearly basis, since each fiscal year's
    funds are appropriated.

    [49] GAO-07-1134SP.

    [50] Defense Contract Management Agency, Department of Defense Earned
    Value Management Implementation Guide.

    [51] GAO-07-1134SP.

    [52] In its technical comments on a draft of this report, DOD stated
    that the Army agrees that there is usually more work that is being
    considered and not yet in the contract baseline. The Army did not agree
    that this situation affects the EVM data. DOD stated that the Army uses
    the data to determine current variance and trend information. The data
    are also used by the systems contractors and CMA program office along
    with estimates of additional future work to evaluate the life cycle
    schedule and costs. The department stated that it will request the
    Defense Contract Management Agency to conduct an audit of the CMA EVM
    system beginning no later than June 30, 2008.

    [53] CMA had previously incentivized schedule as a distinct performance
    measure in fiscal year 2002 at all four incineration sites and
    additionally in fiscal years 2000 and 2001 at Tooele.

    [54] GAO-06-66.

    [55] The initial actual percentages of the base and award fees for each
    of these contracts were originally established through the full and
    open competitions conducted for each of these contracts. Each
    contractor's proposal included a percentage for both the base fee and
    the award fee, which the government evaluated as part of the contract
    selection process. With the exception of one systems contractor, all
    the contractors proposed a total fee of 10 percent with up to 3 percent
    proposed for the base fee and the remaining 7 percent allocated to
    award fee. The one exception is Newport where the contractor proposed a
    base fee of 3 percent and an award fee of 3 percent.

    [56] However, these incentives did not keep the Newport contract on
    schedule. The systems contractor missed the target completion date for
    the third of its four performance incentive milestones and the program
    was delayed by over a year. According to DOD, the failure to meet this
    milestone was due to unforeseen technical difficulties and could not
    have been ultimately influenced by any type of contractual language.

    [57] Schedule was a distinct performance measure in the Tooele award
    fee plan for fiscal years 2000 and 2001. In 2002, it was a distinct
    performance measure in the award fee plans at each of the incineration
    sites.

    [58] CMA officials noted they were acting in accordance with Congress's
    mandate that the program shall provide for the maximum protection of
    the workforce, the public, and the environment. See 50 U.S.C. 1521
    (c)(1)(A) (2007).

    [59] See the Award Fee Review Board Report for the Anniston Chemical
    Agent Disposal Facility, Evaluation Number 17, for the evaluation
    period from April 1, 2005, to September 30, 2005.

    [60] See the Award Fee Evaluation Board Report for the Umatilla
    Chemical Agent Disposal Facility, First Period, Fiscal Year 2005, for
    the period of performance from October 1, 2004, to March 31, 2005.

    [61] Memorandum from the Under Secretary of Defense for Acquisition,
    Technology, and Logistics on Chemical Demilitarization Program
    Strategic Governance (Washington, D.C.: Jan. 6, 2006).

    [62] GAO-06-66 and GAO, NASA Procurement: Use of Award Fees for
    Achieving Program Outcomes Should Be Improved, GAO-07-58 (Washington,
    D.C.: Jan. 17, 2007).

    [63] Memorandum from the Under Secretary of Defense for Acquisition,
    Technology, and Logistics on Award Fee Contracts (FAR 16, DFARS 215,
    DFARS 216) (Washington, D.C.: Mar. 29, 2006). In addition, the National
    Defense Authorization Act for Fiscal Year 2007 directed DOD to issue
    guidance, with detailed implementation instructions, to ensure that all
    new contracts using award fees link such fees to acquisition outcomes,
    which are defined in terms of program cost, schedule, and performance.
    See Pub. L. No. 109-364, � 814 (2006). Furthermore, in an April 2007
    memorandum, DOD issued further award fee guidance stating that it is
    the policy of the department that for new solicitations issued on or
    after August 1, 2007, objective criteria be utilized, whenever
    possible, to measure contract performance. If it is determined that
    objective criteria do not exist, then the head of the contracting
    activity must sign a determination and finding that "the work to be
    performed is such that it is neither feasible nor effective to devise
    predetermined objective incentive targets applicable to cost, technical
    performance, or schedule."

    [64] See GAO-06-66 and GAO-07-58.

    [65] Pub. L. No. 109-364 (2006).

    [66] Chemical Materials Agency, Incentive Agreement for Completion of
    Destruction Operations and Facility Closure Activities at Chemical
    Stockpile Disposal Facilities (Washington, D.C.: May 19, 2007).

    [67] Pub. L. No. 109-364 (2006).

    [68] Pub. L. No. 109-364, � 923 (a)(2) (2006).

    [69] Pub. L. No. 109-364, � 923 (b)(2) (2006).

    [70] Chemical Materials Agency, Incentive Agreement for Completion of
    Destruction Operations and Facility Closure Activities at Chemical
    Stockpile Disposal Facilities (Washington, D.C.: May 19, 2007).

    GAO's Mission:

    The Government Accountability Office, the audit, evaluation and
    investigative arm of Congress, exists to support Congress in meeting
    its constitutional responsibilities and to help improve the performance
    and accountability of the federal government for the American people.
    GAO examines the use of public funds; evaluates federal programs and
    policies; and provides analyses, recommendations, and other assistance
    to help Congress make informed oversight, policy, and funding
    decisions. GAO's commitment to good government is reflected in its core
    values of accountability, integrity, and reliability.

    Obtaining Copies of GAO Reports and Testimony:

    The fastest and easiest way to obtain copies of GAO documents at no
    cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
    weekday, GAO posts newly released reports, testimony, and
    correspondence on its Web site. To have GAO e-mail you a list of newly
    posted products every afternoon, go to [hyperlink, http://www.gao.gov]
    and select "Subscribe to Updates."

    Order by Mail or Phone:

    The first copy of each printed report is free. Additional copies are $2
    each. A check or money order should be made out to the Superintendent
    of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
    more copies mailed to a single address are discounted 25 percent.
    Orders should be sent to:

    U.S. Government Accountability Office:
    441 G Street NW, Room LM:
    Washington, D.C. 20548:

    To order by Phone:
    Voice: (202) 512-6000:
    TDD: (202) 512-2537:
    Fax: (202) 512-6061:

    To Report Fraud, Waste, and Abuse in Federal Programs:

    Contact:

    Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
    E-mail: fraudnet@gao.gov:
    Automated answering system: (800) 424-5454 or (202) 512-7470:

    Congressional Relations:

    Gloria Jarmon, Managing Director, JarmonG@gao.gov:
    (202) 512-4400:
    U.S. Government Accountability Office:
    441 G Street NW, Room 7125:
    Washington, D.C. 20548:

    Public Affairs:

    Chuck Young, Managing Director, youngc1@gao.gov:
    (202) 512-4800:
    U.S. Government Accountability Office:
    441 G Street NW, Room 7149:
    Washington, D.C. 20548:

    *** End of document. ***