Chemical Demilitarization: Additional Management Actions Needed  
to Meet Key Performance Goals of DOD's Chemical Demilitarization 
Program (06-DEC-07, GAO-08-134).				 
                                                                 
Destruction of the nation's remaining stockpile of chemical	 
weapons in a safe, efficient, and timely manner is essential to  
meet Chemical Weapons Convention treaty obligations and to reduce
the risk of a potential catastrophic event. The Department of	 
Defense (DOD) established the Chemical Demilitarization Program  
to manage the destruction of the remaining stockpile. GAO was	 
asked to evaluate the (1) progress DOD and the Army have made in 
addressing GAO's prior recommendations to strengthen program	 
management, (2) reasonableness of schedule milestones, (3)	 
reliability of cost estimates, and (4) effectiveness of efforts  
to provide monetary incentives to the systems contractors. GAO	 
reviewed relevant planning documents, schedules, cost estimates, 
and contracts; interviewed program and contractor officials; and 
visited chemical agent destruction sites.			 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-08-134 					        
    ACCNO:   A78748						        
  TITLE:     Chemical Demilitarization: Additional Management Actions 
Needed to Meet Key Performance Goals of DOD's Chemical		 
Demilitarization Program					 
     DATE:   12/06/2007 
  SUBJECT:   Chemical regulation				 
	     Chemical weapons					 
	     Chemical weapons disposal				 
	     Cost analysis					 
	     Cost control					 
	     Defense cost control				 
	     Evaluation criteria				 
	     Performance measures				 
	     Program evaluation 				 
	     Program management 				 
	     Rates						 
	     Risk assessment					 
	     Risk management					 
	     Schedule slippages 				 
	     Strategic planning 				 
	     Program goals or objectives			 
	     Program implementation				 
	     Chemical Weapons Convention			 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-08-134

   

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to [email protected]. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Requesters: 

United States Government Accountability Office: 

GAO: 

December 2007: 

Chemical Demilitarization: 

Additional Management Actions Needed to Meet Key Performance Goals of 
DOD's Chemical Demilitarization Program: 

Chemical Demilitarization: 

GAO-08-134: 

GAO Highlights: 

Highlights of GAO-08-134, report to congressional requesters. 

Why GAO Did This Study: 

Destruction of the nationï¿½s remaining stockpile of chemical weapons in 
a safe, efficient, and timely manner is essential to meet Chemical 
Weapons Convention treaty obligations and to reduce the risk of a 
potential catastrophic event. The Department of Defense (DOD) 
established the Chemical Demilitarization Program to manage the 
destruction of the remaining stockpile. GAO was asked to evaluate the 
(1) progress DOD and the Army have made in addressing GAOï¿½s prior 
recommendations to strengthen program management, (2) reasonableness of 
schedule milestones, (3) reliability of cost estimates, and (4) 
effectiveness of efforts to provide monetary incentives to the systems 
contractors. GAO reviewed relevant planning documents, schedules, cost 
estimates, and contracts; interviewed program and contractor officials; 
and visited chemical agent destruction sites. 

What GAO Found: 

DOD and the Army have taken steps in addressing GAOï¿½s prior 
recommendations to strengthen program management by establishing an 
overall strategy and supporting implementation plan, but some key 
elements, such as annual performance measures for some key goals, 
including interim destruction goals, are not fully developed. Moreover, 
actions DOD and the Army have taken to identify and mitigate the risk 
of future program schedule extensions and cost growth have not been 
effective because the Chemical Materials Agencyï¿½s risk management 
process has not been fully developed or integrated with DODï¿½s risk 
management process. As a result, managers lack an integrated and 
systematic approach to evaluate and manage risk. 

Recently achieved destruction rates may indicate that adjusted schedule 
milestones are overly conservative. Moreover, the programï¿½s schedule 
was extended in 2005 in response to the slower-than-anticipated 
destruction rates experienced since 2003, when more optimistic 
milestones were adopted. GAOï¿½s review of processing rates achieved at 
the sites since the revised schedule determined that most sites are 
significantly ahead of the revised, extended program estimates for 
recently completed munitions campaigns. For example, Umatilla, Oregon, 
completed destruction of one of its rocket munitions by August 2007, 
about 12 months earlier than forecasted. 

GAO was not able to verify the accuracy of the programï¿½s cost estimates 
because of shortcomings in the underlying cost data, such as undefined 
facility closure requirements and unstable baseline costs. GAO 
determined that the usefulness of the earned value management tool to 
provide visibility over the programï¿½s cost performance is limited 
because of these shortcomings. The programï¿½s projected cost growth is 
largely attributable to longer schedules and increased costs associated 
with facility closure estimates. GAO estimated that a 1-month schedule 
extension would cost an additional $9.5 million. Also, schedules used 
for the closure cost estimates for destruction sites have increased, 
from 12 months to about 30 months at some sites, but they have not been 
finalized or independently reviewed. 

Since 2003, the Army has actively managed criteria for determining 
monetary feesï¿½largely award feesï¿½it provided systems contractors to 
incentivize their performance. These fees had emphasized safety and 
environmental compliance, and Army officials acknowledged that award 
fees did not successfully control schedule and cost growth. In 2006, 
after audit results and DOD direction, and while extending the 
schedule, the Army reinstated schedule as a performance measure. The 
award fee plan allows site project managers flexibility to weigh 
performance measures within a range, and GAO determined that the high 
end of the range for management was generally selected, rather than for 
schedule and cost. Most criteria in the programï¿½s award fee plans are 
subjective and are not linked to objective outcomes as much as 
possible, as called for in DOD guidance. Given the programï¿½s maturity, 
GAO found that the Army could apply more weight to performance 
measures, such as schedule and cost, and add objective criteria in 
these plans 

What GAO Recommends: 

GAO recommends that DOD and the Army develop interim destruction goals, 
approaches, and milestones; establish time frames to complete its risk 
management approach; develop realistic schedule and closure cost 
estimates; finalize and independently review cost estimates; and 
determine whether a greater emphasis can be placed on schedule and 
cost, and develop more objective award fee performance evaluation 
criteria. DOD concurred or partially concurred with 12 of GAOï¿½s 13 
recommendations. 

To view the full product, including the scope and methodology, click on 
[hyperlink, http://www.GAO-08-134]. For more information, contact Davi 
D'Agostino at (202) 512-5431 or [email protected]. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

DOD and the Army Have Taken Steps to Strengthen Program Management, but 
Have Not Fully Developed an Integrated Approach for Measuring 
Performance and Managing Risk: 

Recently Improved Destruction Rates for Chemical Munitions May Indicate 
Overly Conservative Schedule Milestones at the Four Incineration Sites: 

Shortcomings in Underlying Cost Data Make Earned Value Management 
Results Unreliable and Limited Our Ability to Determine Cost Estimate 
Reliability: 

The Army Has Actively Managed Award Fees and Could Further Emphasize 
Schedule and Cost and Link Award Fees to More Objective Performance 
Measures: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Scope and Methodology: 

Appendix II: Two Recently Introduced Performance-Based Incentive Fees: 

Appendix III: Comments from the Department of Defense: 

Appendix IV: GAO Contact and Staff Acknowledgments: 

Related GAO Products: 

Tables: 

Table 1: Status of DOD's Chemical Agent Stockpile Sites and Chemical 
Agent Destroyed as of September 30, 2007: 

Table 2: Comparison of Some Key Destruction Objectives Identified in 
DOD's Strategic Plan and CMA's Strategic Plan: 

Table 3: Comparison of Performance Measures Identified in the PART, OSD 
Strategic Plan, CMA Strategic Plan, and Selected Acquisition Report: 

Table 4: Comparison of Estimated Operation Completion Dates: 

Table 5: Number of Days Saved per Site since 2005 Program Office 
Schedule Estimates: 

Table 6: Potential Operational Cost Savings Associated with Days Saved 
since 2005 Schedule Estimate: 

Table 7: Number of Months of Improvement in 2007 Program Current 
Estimate for Operations Completion versus 2005 Army Program Office 
Estimate and 2005 DOD Independent Estimate: 

Table 8: CMA Assessed Confidence in Achieving DOD Schedule Milestone 
and April 29, 2012, Treaty Deadline: 

Table 9: Changes in Estimated Total Program Cost: 

Table 10: Comparison of GAO's Independent Estimates to Complete 
Operations and Closure Activities with the Systems Contractor's and 
Program Manager's Estimates: 

Table 11: Percentage of Award Fees Earned at the Incineration Sites and 
Newport, from Fiscal Year 2003 to Fiscal Year 2006: 

Table 12: Performance Measures and Weights in Award Fee Plans, Fiscal 
Years 2003-2007: 

Figures: 

Figure 1: U.S. Stockpile of Chemical Agents and Munitions: 

Figure 2: Umatilla Chemical Agent Disposal Facility, Umatilla Chemical 
Depot, Oregon: 

Figure 3: Delivery of a Ton Container to the Processing Building at the 
Newport Chemical Agent Disposal Facility, Newport Chemical Depot, 
Indiana: 

Figure 4: Intermodal Storage Containers at the Newport Chemical Agent 
Disposal Facility, Newport Chemical Depot, Indiana: 

Figure 5: 2005 Program Office Schedule Estimates by Processing Rates, 
Downtime, and Risk: 

Figure 6: Comparison of the Performance Baseline with the Planned 
Budget and Systems Contractors' Estimate at Completion: 

Figure 7: Comparison of the Monetary Incentive Fees Earned by Systems 
Contractors at the Incineration Sites and at Newport during Fiscal Year 
2006: 

Abbreviations: 

ACWA: Assembled Chemical Weapons Alternatives: 

APB: Acquisition Program Baseline: 

CAIG: Cost Analysis Improvement Group: 

CMA: Chemical Materials Agency: 

CWC: Chemical Weapons Convention: 

DOD: Department of Defense: 

DPPBI: Director's Programmatic Performance Based Incentive: 

EAC: estimate at completion: 

EVM: earned value management: 

OMB: Office of Management and Budget: 

OSD: Office of the Secretary of Defense: 

PART: Program Assessment Rating Tool: 

SAR: Selected Acquisition Report: 

United States Government Accountability Office: 

Washington, DC 20548: 

December 6, 2007: 

The Honorable Adam Smith: 
Chairman: 
The Honorable Mac Thornberry: 
Ranking Member: 
Subcommittee on Terrorism and Unconventional Threats and Capabilities:  
Committee on Armed Services: 
House of Representatives: 

The Honorable Jim Saxton: 
House of Representatives: 

Destruction of the nation's remaining stockpile of lethal chemical 
weapons in a safe, efficient, and timely manner is essential to meeting 
U.S. treaty obligations under the Chemical Weapons Convention 
(CWC)[Footnote 1] and to reducing the risk to the public and the 
environment of a potential catastrophic event. In 1985, Congress 
directed the Department of Defense (DOD) to destroy the U.S. stockpile 
of lethal chemical agents and munitions in the United States with the 
requirement that DOD was to ensure maximum protection for the 
environment, the general public, and the personnel involved in the 
destruction efforts. DOD designated the Department of the Army as its 
executive agent, and the Army established the Chemical Demilitarization 
Program to destroy the stockpile at nine storage sites.[Footnote 2] The 
United States and more than 180 other countries are signatories to the 
CWC, which went into effect in 1997. The CWC prohibits the use of 
chemical weapons and mandates a deadline of April 29, 2012, for the 
United States to destroy the existing stockpiles.[Footnote 3] 

DOD's Chemical Demilitarization Program has not experienced a chemical 
event since its inception in 1985 that resulted in an environmental 
catastrophe or death, but the program has experienced some potentially 
serious events and has been plagued by numerous schedule revisions, 
cost growth, and long-standing management weaknesses. The Chemical 
Demilitarization Program is managed as a major defense acquisition 
program and, as such, has cost and schedule milestones that are 
contained in an Acquisition Program Baseline (APB). As of September 30, 
2006, Congress had appropriated about $15.8 billion for the program. In 
2005, DOD reported that the program had exceeded its milestones under 
the 2003 APB in excess of Nunn-McCurdy thresholds for cost and schedule 
performance[Footnote 4] for the second time in 3 years,[Footnote 5] and 
new schedule milestones and cost estimates were being developed. These 
revisions increased DOD's estimated costs for the entire Chemical 
Demilitarization Program by 33 percent from about $24 billion in 2003 
to about $32 billion in 2005. Moreover, the Army's revised cost 
estimates for the portion of the program responsible for destroying the 
stockpile, the Chemical Stockpile Elimination Project,[Footnote 6] 
increased by more than 53 percent from about $15.8 billion in 2003 to 
about $24.2 billion in 2005. A large percentage of this increase was 
attributable to revised government cost estimates for the destruction 
facility contracts. In April 2006, DOD announced that the program will 
not meet the extended CWC deadline of April 29, 2012, for total 
destruction of its stockpile, and it estimated that only 66 percent of 
the stockpile will be destroyed by then. DOD also announced that it 
would seek resources to complete destruction as close to the CWC 
deadline as possible and separately requested authority to provide 
additional monetary incentives if systems contractors operating the 
incineration sites could accelerate their destruction activities to 
safely exceed, meet, or finish destruction near the CWC deadline. 
Moreover, the program's remaining four incineration sites--Anniston 
Chemical Agent Disposal Facility, Anniston Chemical Activity 
(Anniston), Alabama; Pine Bluff Chemical Agent Disposal Facility, Pine 
Bluff Chemical Activity (Pine Bluff), Arkansas; Tooele Chemical Agent 
Disposal Facility, Deseret Chemical Depot (Tooele), Utah; and Umatilla 
Chemical Agent Disposal Facility, Umatilla Chemical Depot (Umatilla), 
Oregon--were forecasted to miss the CWC deadline for the first 
time.[Footnote 7] 

In the past, the program has had long-term problems in meeting all of 
the schedule milestones for destroying the stockpile, in part because 
of the lengthy environmental permitting processes prior to operation 
start-up and unexpected conditions of the aging munitions. The 
stockpile destruction program managed by the Chemical Materials Agency 
(CMA) has now reached a more mature phase, and CMA and its systems 
contractors have more experience in incinerating each type of chemical 
agent and munition in its stockpile as well as neutralizing bulk nerve 
and blister agents. 

DOD, the Army, CMA, and the systems contractors that operate the 
destruction facilities face a difficult and hazardous task, as well as 
the need to overcome some significant technical challenges, in 
destroying the U.S. stockpile of lethal chemical agents while ensuring 
the safety of workers, the public, and the environment. Recognizing 
these difficulties and the unique program risks, DOD uses a contract 
that reimburses the systems contractors for all reasonable, allowable, 
and allocable costs. To encourage defense contractors to perform in an 
innovative, efficient, and effective way in areas deemed important to 
the program's success, DOD gives its systems contractors the 
opportunity to earn millions of dollars through monetary incentives 
known as award and incentive fees. 

We reported in 2003 that DOD and the Army had not developed an overall 
strategy and risk management approach for the chemical demilitarization 
program to guide the program with a set of principles and a means to 
manage risk, understand its evolution and implementation, and determine 
whether initiatives are achieving their desired results.[Footnote 8] 
The absence of an overall strategy and risk management approach left 
the program without a clear road map to monitor program performance and 
meet the goal of destroying the chemical stockpile in a safe and timely 
manner. We recommended that DOD and the Army adopt practices that 
leading organizations embrace for effectively implementing and managing 
programs, to include promulgating a comprehensive mission statement, 
long-term and annual performance goals, measurable performance 
indicators, and evaluation and corrective action plans. DOD concurred 
with our recommendations and stated that it would establish these 
management tools. A list of related GAO products regarding chemical 
demilitarization and storage and disposal of the chemical weapons 
stockpile is included at the end of this report. 

We also reported that overall DOD programs have paid contractors large 
amounts of award fees on acquisitions that are falling behind schedule, 
overrunning costs, and experiencing significant technical 
problems.[Footnote 9] In addition, we recommended, among other things, 
that DOD instruct the military services to move toward more outcome- 
based award fee criteria that are both achievable and promote 
accountability for acquisition outcomes.[Footnote 10] DOD concurred 
with the recommendation and, in a March 2006 policy memorandum, 
addressed desired outcomes and the role the award fee should play in 
the overall acquisition strategy. The memorandum stated that "while 
award fee contracts are used when it is neither feasible nor effective 
to devise predetermined objective performance targets, it is imperative 
that award fees be tied to identifiable interim outcomes, discrete 
events or milestones, as much as possible." 

This report responds to a request from the House Committee on Armed 
Services' Subcommittee on Terrorism and Unconventional Threats and 
Capabilities that we conduct a review of DOD's Chemical 
Demilitarization Program. Specifically, this report evaluates the (1) 
progress DOD and the Army have made in addressing our prior 
recommendations to strengthen program management, whether this progress 
has been sufficient to correct past program weaknesses, and the 
effectiveness of actions DOD and the Army have taken to identify and 
mitigate the risk of future program schedule extensions and cost 
growth; (2) reasonableness of the methodology used to determine 
schedule milestones; (3) reliability of its cost estimates; and (4) 
effectiveness of DOD's and the Army's use of monetary incentives to 
improve systems contractors' performance. 

To assess the progress DOD and the Army have made in implementing GAO's 
prior recommendations to strengthen program management, whether it has 
been sufficient, and the effectiveness of actions taken to identify and 
mitigate the risk of future program schedule extensions and cost 
growth, we obtained and reviewed strategic and implementation plans and 
interviewed cognizant officials to identify whether key elements were 
present, such as a program mission statement, long-term goals and 
objectives, delineation of roles and responsibilities of DOD and Army 
offices, and near-term performance measures. To assess the 
reasonableness of the methodology used to determine schedule milestones 
as well as the reliability of cost estimates, we reviewed current 
program estimates, destruction schedules, earned value management 
(EVM)[Footnote 11] data, and other documents. We also obtained and 
reviewed the program's risk management plans and related documents. In 
addition, we identified the issues that had caused delays and 
ascertained approaches being used to reduce the potential for delays in 
the future. To analyze the effectiveness of DOD's and the Army's use of 
monetary incentives to improve systems contractors' performance, we 
obtained and reviewed contract documents on the award fees offered to 
the systems contractors, and interviewed DOD and Army officials as well 
as systems contractors' representatives. We obtained and reviewed the 
incentive agreement plan and incentive fees for the Aberdeen Chemical 
Agent Disposal Facility (Aberdeen), Maryland, and Newport Chemical 
Agent Disposal Facility, Newport Chemical Depot (Newport), Indiana, to 
determine the challenges those sites faced in negotiating the incentive 
fee agreements. Also, we interviewed officials with the U.S. Army 
Sustainment Command, CMA, and CMA and systems contractors at Tooele and 
Umatilla. In addition, we met with DOD and Army program officials and 
interviewed officials at two of the chemical agent destruction sites. 

This report focuses primarily on the seven CMA-managed chemical 
destruction sites--Aberdeen, Anniston, Johnston Atoll in the Pacific 
Ocean, Newport, Pine Bluff, Tooele, and Umatilla. We selected these 
sites for our review because most are in the operational phase, 
represent about 82 percent of the remaining stockpile of chemical 
agents to be destroyed, and have reached a level of maturity in 
processing a variety of types of munitions and agents that will help 
meet CWC goals. Two sites--Aberdeen and Johnston Atoll--have already 
completed operations and have either been closed or are being closed. 
This report does not focus on the two chemical agent destruction sites 
that are part of the Assembled Chemical Weapons Alternatives (ACWA) 
program--Blue Grass Chemical Agent-Destruction Pilot Plant, Blue Grass 
Chemical Activity (Blue Grass), Kentucky, and Pueblo Chemical Agent- 
Destruction Pilot Plant, Pueblo Chemical Depot (Pueblo), Colorado-- 
because they are currently in the design and construction phase to 
destroy chemical agents, report directly to DOD rather than to the 
Army, and because these facilities are not forecasted to begin 
destruction operations before the CWC deadline, although the stockpiles 
at these two sites are required to be destroyed by the deadline as 
well. 

We examined the reliability of the data used in this report by 
verifying EVM system certification, applying analytical tests to the 
EVM data, and examining EVM system effectiveness in providing 
meaningful performance measurement to program management. We determined 
that the EVM system as used at each site was not reliable, which is 
discussed more fully in the report. We conducted this performance audit 
from May 2006 through July 2007 in accordance with generally accepted 
government auditing standards. A more thorough description of our scope 
and methodology is provided in appendix I. 

Results in Brief: 

While DOD and the Army have taken steps toward addressing our prior 
recommendations to strengthen program management by establishing 
strategic and supporting implementation plans, they have not fully 
developed an integrated approach for measuring performance or for 
managing risk.[Footnote 12] DOD issued a strategic plan in June 2005, 
and CMA issued a strategic plan in July 2005 and a risk management plan 
in May 2006. While these plans identify challenges and contain 
performance measures to monitor progress, we found that the plans lack 
schedule goals for key phases. Neither DOD's 2006 baseline nor its 
April 2006 and April 2007 Selected Acquisition Reports (SAR) to 
Congress include schedule milestones extending beyond 2012. The program 
further lacks annual performance goals linked to long-term strategic 
goals, such as meeting the extended April 29, 2012, CWC deadline. 
Without this linkage, program officials will lack the information on 
interim progress needed to determine whether corrective action is 
needed to safely destroy the stockpile by the treaty deadline. 
Furthermore, while CMA's integrated risk management plan identified a 
process for conducting risk management, the agency has not developed 
the tools needed to implement its approach, such as a risk trade-off 
analysis. For example, the program's plans identified contamination of 
the mustard agent stockpile at one site as a key risk but lacked a 
method for analyzing that risk or managing its potential effects on 
cost and schedule. The later discovery of the extent of the 
contamination delayed planned operations by more than 12 months. Also, 
our analysis showed that DOD's and the Army's processes for identifying 
risk are not integrated across sites or organizations, and CMA's 
identified risks have not been fully aligned with those identified in 
DOD's strategic plan. Consequently, program managers are hampered in 
their information sharing and their progress in safely destroying 
chemical agents, as well as in their ability to systematically evaluate 
and manage risk. Finally, unlike more useful annual performance plans 
we have reviewed, which include historical performance trends and 
multiyear future goals, CMA's annual performance plan has a single-year 
focus. We are recommending that the Secretary of the Army incorporate 
historical trends and multiyear future goals in its annual performance 
plan; develop interim destruction goals, approaches, and milestones; 
and establish a time frame for implementing its risk management 
approach, including integration across sites and with DOD. 

Recently achieved processing rates have considerably exceeded 
forecasted rates at the operational sites and indicate that DOD's 
revised schedule milestones may be too conservative to be a realistic 
measure. DOD extended its Chemical Demilitarization Program schedule in 
2005 in response to the slower actual destruction rates experienced 
than had been estimated when more optimistic milestones were adopted in 
2003. As we have previously reported,[Footnote 13] best practices for 
schedule development call for assumptions that are both up-to-date and 
reasonable. Using a new schedule methodology, and as a result of 
factoring in slower processing rates and greater risk, CMA extended 
schedules at each site between 62 months and 80 months beyond the 
baselines established in 2003. The extensions prompted budget and 
acquisition reviews, including an independent cost and schedule 
assessment, which in turn led DOD to further extend the program's 
schedule milestones for completing destruction operations and closure 
activities. Our analysis of recently achieved processing rates showed 
that at the time of our review, the sites were between 2 months and 39 
months ahead of the revised estimates for munitions destruction. 
Revised schedule estimates published in the April 2007 SAR further 
reflect processing rates that considerably exceed forecasts. 
Nevertheless, DOD has no current plans to reassess the program schedule 
milestones, as such reassessments are usually prompted by schedule 
slippages. Higher destruction rates than scheduled indicate that an 
independent reassessment of the program's scheduled milestones may be 
warranted, thus assuring DOD and Congress of the reasonableness of the 
baseline against which its progress is being measured and of its costs 
as the current schedule is not providing DOD and Congress with a 
reasonable baseline against which to measure progress and cost. We are 
recommending that the Secretary of Defense direct the Cost Analysis 
Improvement Group to assess the reasonableness of schedule milestones, 
and that DOD periodically review the program's processing rates to 
determine whether the milestones are still reasonable and include 
schedule milestones for completing destruction operations and facility 
closure activities in the program's baseline and acquisition report. 

Shortcomings in the underlying data used in the program's EVM showed 
that the cost estimate is unreliable. Our review showed that 
performance baselines lacked validity and had not been stable because 
all requirements, such as closure costs, are not fully identified and 
included in program cost estimates and because large amounts of 
additional costs are added to the performance baseline annually. In 
addition, systems contractors' cost estimates were significantly lower 
than the Army's cost estimates, due largely to the Army's expectations 
of longer operational periods and greater closure requirements. The 
program's cost increases are largely attributable to extended schedules 
and facility closure estimates. Best practices indicate that cost 
estimates should be comprehensive, and DOD guidance further reflects 
the need for valid and stable performance baselines from which to make 
projections and perform trend analysis. DOD and Congress currently lack 
accurate estimates for budgeting for and funding all activities through 
facility closure. Similarly, given the significant differences between 
the contractors' and Army's cost estimates, an independent cost review 
would provide DOD and Congress assurance that cost estimates are 
realistic. We are recommending that the Army define closure 
requirements and develop accurate and realistic closure cost estimates; 
define all contract requirements, including authorized unpriced work 
associated with facility closure activities, and establish a time frame 
for placing these requirements on contract; and that an independent 
review of the cost estimates be done and linked with reasonable 
schedule milestones. 

According to the Army and based on our analysis, award fees--the 
predominant fees--offered to contractors have thus far not 
satisfactorily resulted in the control of schedule performance outcomes 
and cost growth. Army officials indicate that the majority of the award 
fee incentives are dedicated to achieving high standards for safety, 
surety, and environmental compliance in recognition of the lethal 
nature of the chemical agent stockpile and the congressional mandate 
for maximum protection of the workforce, the public, and the 
environment. The Army has actively reviewed and adjusted criteria for 
determining award fees it has provided to the systems contractors to 
incentivize aspects of their performance. Based on our analysis, from 
2003 through 2006, the award fee plans have heavily emphasized safety 
and environmental compliance and did not have a separate performance 
measure for schedule until 2006. Army officials acknowledged that award 
fees have not satisfactorily resulted in the control of schedule and 
cost growth. In response to our previous recommendations, in March 
2006, DOD issued policy guidance directing that award fees be tied to 
identifiable interim outcomes, discrete events, or milestones as much 
as possible. CMA issued its award fee policy in September 2006, and our 
review of this policy indicated that CMA did not clearly or fully 
incorporate the concept of tying "award fees to identifiable interim 
outcomes, discrete events or milestones as much as possible," as 
directed in the March 2006 DOD guidance. In addition, the current award 
fee plan for the five chemical demilitarization sites we reviewed 
allows government site project managers flexibility in weighting each 
performance measure within a set range. We found that managers have 
selected the high end of the range for management, and the low end of 
the range for schedule and cost, when determining award fee 
percentages. Moreover, many performance measures in the award fee plans 
contain subjective award fee criteria, and award fees for the most part 
are not linked to objective or measurable criteria. Our analysis showed 
that it is possible for CMA to further link award fee criteria in this 
program to objective or measurable outcomes in at least three 
performance measures. Given the maturity of the operations phase at the 
incineration sites and Newport, although not legally required, we 
identified opportunities, from a management perspective, to apply more 
weight to performance measures, such as schedule and cost, to better 
align CMA's award fee policy with DOD's policy, and to make greater use 
of objective, measurable criteria in award fee plans to increase the 
likelihood of meeting the April 29, 2012, treaty deadline. Two recently 
established and planned performance-based incentive fees for the 
incineration sites are focused on schedule. One fee was established in 
2005 to encourage and reward collaboration among the incineration site 
systems contractors with the goal of improving the overall program 
schedule and cost performance. This fee is awarded based on sites' 
collective performance and, in fiscal year 2007, this incentive fee 
focused mostly on schedule performance associated with achieving the 
CWC treaty deadline. The other planned incentive fee, as authorized by 
Congress, is structured to accelerate destruction operations and 
facility closure at the incineration sites. We could not determine how 
effective these incentives might be due to the collective manner in 
which the fee to encourage and reward collaboration is earned, its 
relative size, and recent focus on objective schedule performance, and 
because the other planned incentive fee has not yet been implemented. 
We are recommending that the Army (1) determine whether greater weight 
can be applied to key performance measures, such as schedule and cost, 
in award fee plans; (2) better align its award fee policy with DOD's 
March 2006 guidance on award fees; and (3) link award fee criteria, as 
much as possible, to performance measures that focus on identifiable 
interim outcomes, discrete events, or milestones. 

We provided a draft of this report to DOD in October 2007 for its 
review and comment. In written comments on a draft of this report, DOD 
concurred or partially concurred with 12 of our 13 recommendations. DOD 
did not concur with our recommendation regarding defining closure 
requirements and developing accurate and realistic closure cost 
estimates for each of DOD's chemical demilitarization sites. DOD 
recommended that we delete it, since DOD officials believe that a 
subsequent recommendation defining all contract requirements, including 
those associated with facility closure activities, was more 
encompassing. Based on DOD's comments, rather than deleting the 
recommendation, we clarified it and the related recommendation to more 
clearly distinguish the difference between the two recommendations, and 
consolidated it with our two other recommendations related to defining 
contract requirements. DOD also stated that the department will 
continue to use acquisition management tools and discipline to ensure 
the destruction of the United States' chemical weapons stockpile in a 
safe and secure manner, while being economical and meeting the U.S. 
commitments under the CWC. DOD identified a number of actions that it 
already has initiated in response to our recommendations. DOD also 
provided us with technical comments, which we incorporated in the 
report as appropriate. DOD's response is reprinted in appendix III. 

Background: 

In 1985, Congress directed DOD to destroy the U.S. stockpile of lethal 
chemical agents and munitions in the United States.[Footnote 14] DOD 
designated the Department of the Army as its executive agent for the 
program, and the Army established the Chemical Demilitarization 
Program, which was charged with the destruction of the original 
stockpile at nine storage sites. Figure 1 shows the locations of the 
nine storage sites; the number of items, such as projectiles, rockets, 
mortars, land mines, and bombs, originally stored at each site; and the 
number of items remaining to be destroyed as of September 30, 2007. 

Figure 1: U.S. Stockpile of Chemical Agents and Munitions: 

This figure is a map showing stockpile of chemical agents and 
munitions. 

Umatilla Chemical Depot, Oregon; 
Number of items: 220,603; 
Remaining items: 65,060. 

Deseret Chemical Depot, Utah; 
Number of items: 1,138,785; 
Remaining items: 122,361. 

Newport Chemical Depot, Indiana; 
Number of items: 1,690; 
Remaining items: 579. 

Aberdeen Proving Ground, Maryland; 
Number of items: 1,817; 
Remaining items: 0. 

Blue Grass Army Depot, Kentucky; 
Number of items: 101,764; 
Remaining items: 101,764. 

Anniston Army Depot, Alabama; 
Number of items: 661,531; 
Remaining items: 446,712. 

Pine Bluff Arsenal, Arkansas; 
Number of items: 123,095; 
Remaining items: 32,684. 

Pueblo Chemical Depot, Colorado; 
Number of items: 780,078; 
Remaining items: 780,078. 

Johnston Atoll, Pacific Ocean; 
Number of items: 412,798; 
Remaining items: 60. 

[See PDF for image] 

Source: GAO analysis of CMA data. 

[End of figure] 

The remaining U.S. stockpile, as of June 2007, consists of two nerve 
agents--GB and VX[Footnote 15]--and a blister agent--mustard[Footnote 
16]--which currently are stored at seven sites in the continental 
United States. Incineration was initially selected as the method to 
destroy the stockpile at all nine sites, and destruction began in 1990 
at Johnston Atoll. This location was selected to be the operational 
prototype facility to test the destruction process prior to it being 
conducted in the continental United States. Figure 2 shows an example 
of a chemical agent disposal facility like those used at the four 
operational incineration sites (Anniston, Pine Bluff, Tooele, and 
Umatilla). 

Figure 2: Umatilla Chemical Agent Disposal Facility, Umatilla Chemical 
Depot, Oregon: 

This figure is a photograph of Umatilla Chemical Agent Disposal 
Facility, in Umatilla Chemical Depot, Oregon. 

[See PDF for image] 

Source: Umatilla Chemical Agent Disposal Facility. 

[End of figure] 

Subsequent to the decision to use incineration to destroy the remaining 
stockpile, Congress directed DOD to evaluate the possibility of using 
alternative technologies to incineration.[Footnote 17] In 1994, the 
Army initiated a project to develop nonincineration technologies for 
use at the two bulk-agent-only sites at Aberdeen and Newport. These 
sites were selected in part because their stockpiles were relatively 
simple--each site had only one type of agent and this agent was stored 
in bulk-agent (ton) containers, which are used to store various 
chemical agents at the chemical agent destruction sites.[Footnote 18] 
Figure 3 shows a ton container being transported to the processing 
building at Newport. 

Figure 3: Delivery of a Ton Container to the Processing Building at the 
Newport Chemical Agent Disposal Facility, Newport Chemical Depot, 
Indiana: 

This figure is a photograph of the delivery of a ton container to the 
processing building at Newport Chemical Agent Disposal Facility, in 
Newport, Indiana. 

[See PDF for image] 

Source: Newport Chemical Agent Disposal Facility. 

[End of figure] 

In 1997, DOD approved pilot testing of a neutralization technology at 
these two sites. Figure 4 shows the type of hydrolysate-filled 
intermodal storage containers used at Newport. The hydrolysate is 
awaiting final disposition.[Footnote 19] 

Figure 4: Intermodal Storage Containers at the Newport Chemical Agent 
Disposal Facility, Newport Chemical Depot, Indiana: 

This figure is a photograph of intermodal storage containers at the 
Newport Chemical Agent Disposal Facility in Newport Chemical Depot, 
Indiana. 

[See PDF for image] 

Source: Newport Chemical Agent Disposal Facility. 

[End of figure] 

Also in 1996, Congress directed DOD to evaluate the use of alternative 
technologies and suspended incineration planning activities at Blue 
Grass and Pueblo, two sites with assembled weapons.[Footnote 20] 
Furthermore, Congress directed that these two sites be managed in a 
program independent of the Army's Chemical Demilitarization Program and 
report to DOD directly instead of the Army.[Footnote 21] Thus, the ACWA 
program was established. 

Table 1 shows DOD's nine chemical agent stockpile sites; the primary 
destruction method used (i.e., incineration or neutralization) and the 
secondary disposal method used for the treatment of the resulting 
wastewater, if applicable; the types of munitions destroyed, such as 
projectiles, rockets, mortars, land mines, and bombs in addition to ton 
containers; the types of agents (GB, VX, and mustard) and their status 
(e.g., destroyed, ongoing destruction, or planned destruction); the 
original agent tonnage; the tonnage destroyed; the percentage of the 
stockpile destroyed; the remaining agent tonnage; and the status of 
each facility (closed, operational, or under construction) as of 
September 30, 2007. 

Table 1: Status of DOD's Chemical Agent Stockpile Sites and Chemical 
Agent Destroyed as of September 30, 2007: 

Site: Johnston Atoll; 
Destruction method: Incineration; 
Munition types destroyed: Projectiles, land mines, mortars, and ton 
containers; 
Type of agent and status[A]: GB, VX, and mustard (destroyed); 
Original agent tonnage: 705[B]; 
Tonnage destroyed: 705; 
Percentage of stockpile destroyed: 100; 
Remaining agent tonnage (as of September 2007): 0; 
Facility status: Closed. 

Site: Tooele; 
Destruction method: Incineration; 
Munition types destroyed: Ton containers, mines, rockets, projectiles, 
spray tanks, and mortars; 
Type of agent and status[A]: GB and VX (destroyed) and mustard 
(ongoing); 
Original agent tonnage: 13,617; 
Tonnage destroyed: 9,292; 
Percentage of stockpile destroyed: 68; 
Remaining agent tonnage (as of September 2007): 4,325; 
Facility status: Operational. 

Site: Anniston; 
Destruction method: Incineration; 
Munition types destroyed: Rockets and projectiles; 
Type of agent and status[A]: GB (destroyed), VX (ongoing), and mustard 
(planned); 
Original agent tonnage: 2,254; 
Tonnage destroyed: 726; 
Percentage of stockpile destroyed: 32; 
Remaining agent tonnage (as of September 2007): 1,528; 
Facility status: Operational. 

Site: Umatilla; 
Destruction method: Incineration; 
Munition types destroyed: Rockets, bombs, and projectiles; 
Type of agent and status[A]: GB (ongoing) and VX and mustard (planned); 
Original agent tonnage: 3,719; 
Tonnage destroyed: 1,016; 
Percentage of stockpile destroyed: 27; 
Remaining agent tonnage (as of September 2007): 2,703; 
Facility status: Operational. 

Site: Pine Bluff; 
Destruction method: Incineration; 
Munition types destroyed: Rockets and ton containers; 
Type of agent and status[A]: GB (destroyed) and VX and mustard 
(planned); 
Original agent tonnage: 3,850; 
Tonnage destroyed: 484; 
Percentage of stockpile destroyed: 12; 
Remaining agent tonnage (as of September 2007): 3,366; 
Facility status: Operational. 

Site: Aberdeen; 
Destruction method: Neutralization and off-site treatment of 
wastewater; 
Munition types destroyed: Ton containers; 
Type of agent and status[A]: Mustard (destroyed) and off-site 
wastewater treatment (complete); 
Original agent tonnage: 1,622; 
Tonnage destroyed: 1,622; 
Percentage of stockpile destroyed: 100; 
Remaining agent tonnage (as of September 2007): 0; 
Facility status: Closed. 

Site: Newport; 
Destruction method: Neutralization and off-site treatment of 
wastewater; 
Munition types destroyed: Ton containers; 
Type of agent and status[A]: VX (ongoing) and off-site wastewater 
treatment (ongoing)[C]; 
Original agent tonnage: 1,269; 
Tonnage destroyed: 587; 
Percentage of stockpile destroyed: 46; 
Remaining agent tonnage (as of September 2007): 682; 
Facility status: Operational. 

Site: Pueblo; 
Destruction method: Neutralization and on-site biotreatment of 
wastewater; 
Munition types destroyed: Projectiles and mortars; 
Type of agent and status[A]: Mustard (planned); 
Original agent tonnage: 2,611; 
Tonnage destroyed: 0; 
Percentage of stockpile destroyed: 0; 
Remaining agent tonnage (as of September 2007): 2,611; 
Facility status: Destruction pilot plant under construction. 

Site: Blue Grass; 
Destruction method: Neutralization and on-site water oxidation of 
wastewater; 
Munition types destroyed: Rockets and projectiles; 
Type of agent and status[A]: GB, VX, and mustard (planned); 
Original agent tonnage: 523; 
Tonnage destroyed: 0; 
Percentage of stockpile destroyed: 0; 
Remaining agent tonnage (as of September 2007): 523; 
Facility status: Destruction pilot plant under construction. 

Total; 
Destruction method: [Empty]; 
Munition types destroyed: [Empty]; 
Type of agent and status[A]: [Empty]; 

Original agent tonnage: 30,170; 
Tonnage destroyed: 14,432; 
Percentage of stockpile destroyed: 48; 
Remaining agent tonnage (as of September 2007): 15,738; 
Facility status: [Empty]. 

Source: DOD. 

Note: The Non-Stockpile Chemical Materiel Project is the CMA 
organization responsible for all chemical warfare materiel that resides 
outside of a declared stockpile. This materiel consists of chemical 
agent identification sets, which consist of glass ampoules, vials and 
bottles of chemical agent, and actual chemical weapons. The project 
provides centralized management and direction to DOD for the disposal 
of nonstockpile chemical warfare materiel in a safe, environmentally 
sound, and cost-effective manner. It is responsible for destroying the 
nerve agents GB and VX in projectiles, vials, bottles, glass ampoules, 
and ton containers, As of June 2007, the Non-Stockpile Chemical 
Materiel Project had destroyed 563 tons of chemical agent, which is 
counted toward meeting the CWC 45 percent destruction milestone. 

[A] The stockpile includes two nerve agents (GB and VX) and a blister 
agent (mustard). 

[B] The site at Johnston Atoll destroyed 1,326 tons of agent before the 
CWC entry into force in 1997. Overall, the site at Johnston Atoll has 
destroyed a total of 2,031 tons of agent. 

[C] On January 5, 2007, CMA began shipping Newport VX hydrolysate to a 
commercial wastewater treatment facility in Port Arthur, Texas. Because 
the VX hydrolysate is being transported to Texas for final treatment, 
the United States receives credit for destruction of the Newport 
stockpile under CWC. Of an estimated 1.8 million gallons of VX 
hydrolysate at Newport, as of June 2007, approximately 33 percent has 
already been shipped to Texas for incineration. 

[End of table] 

In 1997, the United States ratified the CWC, which prohibits the use of 
these weapons and mandates the elimination of existing stockpiles by 
April 29, 2012, which represents a 5-year extension from the initial 
deadline.[Footnote 22] The CWC also contains a series of interim 
deadlines applicable to the U.S. stockpile[Footnote 23] for destroying 
1 percent, 20 percent, and 45 percent of the stockpile. The United 
States met the 1 percent and 20 percent destruction deadlines, but did 
not meet the original 45 percent destruction deadline. The United 
States received an extension from April 2004 to December 2007 for the 
45 percent deadline. As of June 2007, program officials reported that 
45 percent of the U.S stockpile had been destroyed, 6 months earlier 
than the interim CWC deadline for destroying that amount, and the 
entire stockpile at two of the nine sites has been eliminated. 

DOD designated the Chemical Demilitarization Program as a major 
acquisition program.[Footnote 24] As such, the overall program cost and 
schedule--and its milestones for major phases of the program--are 
identified in the APB and are reported to Congress in the SAR. 
Additionally, the Army, through its CMA, developed program estimates 
that include schedule milestones to support its day-to-day management 
responsibilities of the program. Finally, systems contractors that 
operate the chemical agent destruction facilities have established 
schedule milestones for completing major phases identified in their 
individual contracts with the Army. 

From 1998 through 2005, the total program cost estimates increased by 
113 percent, resulting in the program breaching approved program 
baselines in 2001 and 2005. As a result, the program was required to 
certify to Congress in each instance that new cost estimates were 
reasonable and that the management structure is adequate to manage and 
control costs. The bulk of these program costs will go to systems 
contractors that construct and operate the facilities that destroy the 
chemical weapons. 

DOD and the Army Have Taken Steps to Strengthen Program Management, but 
Have Not Fully Developed an Integrated Approach for Measuring 
Performance and Managing Risk: 


DOD and the Army have taken steps in addressing our prior 
recommendations to strengthen program management by establishing an 
overall strategy and supporting implementation plans, but some key 
elements related to performance measures and risk management are not 
fully developed or implemented. DOD and the Army have identified a 
number of risks across the program that could affect the program's cost 
and schedule, and CMA has developed a risk management plan, but the 
program's risk management approach has not been effective because it 
has not been fully implemented or integrated across the program. 
Moreover, while DOD and the Army have established key program goals, 
performance measures have not always been linked to these goals. 

DOD Addressed GAO's Prior Recommendations by Establishing Strategic and 
Supporting Implementation Plans: 

DOD has taken a number of actions to address our past recommendations 
that it establish strategic and supporting implementation plans. We 
recommended in 2003 that DOD and the Army develop an overall strategy 
and implementation plan for the chemical demilitarization program that 
would articulate a program mission statement, identify the program's 
long-term goals and objectives, delineate the roles and 
responsibilities of all DOD and Army offices, and establish near-term 
performance measures.[Footnote 25] We also recommended that DOD and the 
Army implement a risk management approach that anticipates and 
influences internal and external factors that could adversely affect 
program performance. DOD concurred with our recommendations and has 
strengthened program management by establishing an overall strategy and 
supporting implementation plans. 

In response to our 2003 recommendations, DOD and the Army have 
developed various strategic and supporting plans. For example, in June 
2005, DOD published the Strategic Plan for Destruction of Lethal 
Chemical Agents and Munitions.[Footnote 26] The purpose of this plan is 
to serve as an overarching plan for destruction of the U.S. stockpile 
of lethal chemical agents and munitions and nonstockpile items. It 
applies to both portions of the program implemented by CMA as well as 
the ACWA program. (In this report, we only address the parts of the 
plan pertaining to the CMA stockpile program.) This plan included many 
of the elements that we recommended, such as a program mission 
statement to enhance national security by eliminating chemical warfare 
munitions while protecting the workforce, the public, and the 
environment and meeting requirements specified in the CWC. It also 
included overall goals and objectives and performance measures. 

Also, in July 2005, CMA published the U.S. Army's Chemical Materials 
Agency Strategic Plan (2005-2010). This plan encompasses strategies for 
all CMA organizational elements and programs--including the chemical 
stockpile storage program, the Non-Stockpile Chemical Materiel Project, 
the Elimination of Chemical Weapons program, and the Pine Bluff Arsenal 
industrial base--together with those of the AWCA program. According to 
the CMA strategic plan, its purpose is to be a road map for the safe 
elimination of the U.S. chemical weapons stockpile. This strategic plan 
articulated program goals and objectives and also described a number of 
supporting implementation plans that we recommended should be 
developed, including annual performance plans that would provide 
performance measures and an integrated risk management plan that would 
address the program's challenges. 

In its July 2005 strategic plan, CMA identified 11 supporting plans 
that were directly or indirectly related to the stockpile program. 
According to CMA officials, 9 of the 11 supporting plans had been 
developed, as of July 2007. These plans include the CMA resource 
management plan, a risk management plan, and a risk management 
directorate plan. 

The DOD strategic plan articulated four key goals for the Chemical 
Demilitarization Program: (1) eliminate the stockpile while protecting 
the workforce, the public, and the environment, and comply with U.S 
obligations under the CWC; (2) employ sound business practices to 
ensure efficient life cycle management of program resources; (3) 
institute a program of continuous improvement regarding safety, 
environment protection, efficient operations and maintenance, and 
facility closure to mitigate risks and to ensure compliance with 
statutory, regulatory, and policy requirements and decisions; and (4) 
maintain communications with the public and local and national 
officials. Performance objectives with the associated performance 
metrics were developed to address these goals. Table 2 contains some 
key objectives to meet goals contained in the DOD and CMA strategic 
plans. 

Table 2: Comparison of Some Key Destruction Objectives Identified in 
DOD's Strategic Plan and CMA's Strategic Plan: 

DOD's Strategic Plan: Achieve CWC extended 45 percent destruction 
milestone by December 31, 2007, while ensuring the safety of the 
workers, the public, and the environment; 
CMA's Strategic Plan: Destroy the U.S. stockpile of lethal chemical 
agents and munitions while meeting the CWC deadlines and containing 
total life cycle cost by using an integrated risk management approach. 

DOD's Strategic Plan: Achieve CWC extended 100 percent destruction 
deadline by April 29, 2012, while ensuring the safety of the workers, 
the public, and the environment; 
CMA's Strategic Plan: No objective linked to meeting CWC extended 
deadline by April 29, 2012. 

DOD's Strategic Plan: Manage life cycle costs to the program within 
fiscal resources and consistent with the APB; 
CMA's Strategic Plan: No objective linked to meeting the APB. 

DOD's Strategic Plan: Develop and institute a risk management culture 
that ensures that no chemical agent releases, exposures, and accidents 
occur across the entire demilitarization community; 
CMA's Strategic Plan: Meet and maintain Army, state, and federal safety 
and security standards and environmental laws, regulations, and permit 
conditions. 

Source: GAO analysis of DOD and CMA data. 

[End of table] 

CMA also jointly issued the program's Fiscal Year 2007 Annual 
Performance Plan in September 2006.[Footnote 27] The performance plan 
contains, among other things, annual goals for meeting the CWC's 
December 2007 interim destruction deadline. However, Tooele is the only 
site that has approved annual plans, which it is implementing. CMA 
officials stated that they will request annual performance plans from 
all of the sites for fiscal year 2008. Moreover, the plans lacked some 
attributes that would make them more useful to decision makers and to 
Congress. We have previously found that performance plans can be more 
effective if they (1) use intermediate goals and measures to show 
progress or contribution to intended results, particularly if it may 
take years to meet the intended results; (2) show baseline and trend 
data for past performance; and (3) permit an agency to convey what it 
expects to achieve in the long term by including multiyear performance 
goals in its performance plan.[Footnote 28] With baseline and trend 
data, the more useful performance plans provided a context for drawing 
conclusions about whether performance goals are reasonable and 
appropriate. By including multiyear performance goals, a performance 
plan can provide congressional and other decision makers with an 
indication of the incremental progress the agency expects to make in 
achieving results. CMA's performance plan only contains goals related 
to the specified fiscal year and does not show previous years' trends 
or show how achievement of the specified goals would contribute to the 
program's long-term goal. 

Risk Management Approach Is Not Fully Developed or Integrated, and Some 
Key Program Goals Lack Clearly Linked Performance Measures: 

DOD has partially implemented our recommendations to establish 
performance measures and implement a risk management approach that 
anticipates and influences internal and external factors that could 
adversely affect program performance. Specifically, CMA has not yet 
completed all the risk management tools identified in its risk 
management plan or integrated its risk management efforts across sites 
and organizations. Further, while DOD did establish performance 
measures as we recommended, some key program goals still lack clearly 
linked performance measures. Moreover, the program is using four 
different performance measurement approaches and measurements that are 
not well-aligned with one another. Without annual performance goals and 
measures linked with long-term strategic goals, program officials will 
lack information on the interim progress being achieved and whether 
corrective action is needed to safely destroy the stockpile by the 
treaty deadline. 

CMA Has Made Progress in Developing an Integrated Risk Management 
Approach but Has Not Completed Its Risk Management Tools and 
Integration Efforts: 

As identified in its U.S. Army Chemical Materials Agency Integrated 
Risk Management Plan, CMA has developed a process to identify program- 
level risks and reports them as part of the Army's Probability of 
Program Success Report.[Footnote 29] The program office, in conjunction 
with the site project managers, also has developed a "risk landscape," 
which is used to capture risks that could affect the operating and 
closure schedules. However, no programwide approach exists to integrate 
the risk management efforts at the site, CMA, and DOD levels to ensure 
that all risks are being addressed in an integrated manner. For 
example, CMA officials acknowledged that risks that are included in the 
Probability of Program Success Reports and the site risk landscapes 
were developed independently of each other and thus may not be fully 
integrated. Also, while the sites were developing risk plans that are 
at different stages of maturity, CMA has not yet developed a single set 
of risk management procedures to be used by the site-level project 
managers and site systems contractors when developing these plans. 
Lastly, the risks identified in DOD plans and those identified by CMA 
were not identified as part of an integrated process. CMA officials 
stated that the risks it identified are an aggregate of some site risks 
and CMA program-level risks, whereas the risk identified by DOD in its 
strategic plan represents a general analysis of the major factors that 
the senior leadership has selected as appropriate for long-term 
attention. CMA officials stated that CMA is working to establish a 
formal process for aligning risks among all planning and reporting 
documents. 

Other steps that are needed to fully implement this approach have not 
been completed. For instance, CMA identified other tools to aid in 
implementing an overarching risk management program to complete the 
steps in this process, but these tools have not been fully developed. 
These tools include a risk trade-off analysis and a risk management 
information system database. A risk trade-off analysis is normally 
included in this risk planning step. According to CMA officials, while 
risk trade-off analyses are discussed in their risk management plan, 
the program has not formally used this analysis in assessing program 
risk, and thus this analysis has not been available to help decision 
makers consider how to address program challenges. For example, while 
the condition of the stockpile of mustard agent at Tooele was 
identified as a key program risk, a process was not in place to perform 
a comprehensive analysis to consider the risks to the program's cost 
and schedule if assumptions about the level of mercury contamination 
were understated. Because the extent of the problem was not discovered 
sooner, needed facility modifications were delayed, extending the 
planned operations schedule by 12 months. 

CMA has also not completed the development of its integrated risk 
information management system, which would involve the collection, 
documentation, and reporting of risk data in support of the steps in 
the overall risk management process. According to CMA, an initial 
requirement analysis for the development of an integrated risk 
information system determined that initially considered in-house 
solutions would require considerable additional development. A market 
survey will be conducted to consider external solutions. While a 
comprehensive risk management approach could manage risks, and CMA is 
still in the process of developing such an approach, CMA has not 
established time frames for the completion and implementation of its 
risk management approach. Without clear expectations for fully 
developing and implementing its risk management approach within a set 
time frame, CMA could continue to be limited in its efforts to control 
some of the issues that have been affecting the program. 

Some Key Program Goals Lack Clearly Linked Performance Measures: 

While DOD and the Army have made progress in establishing program 
goals, we found that some program goals were not linked to one another 
or to key performance measures, and making such linkages is a practice 
of leading organizations.[Footnote 30] For example, in its fiscal year 
2007 annual performance plan, CMA had developed an annual performance 
measure linked to the interim CWC destruction goal of 45 percent by 
December 2007. However, neither DOD nor CMA nor the Army has developed 
annual performance measures that are linked to achieving the ultimate 
goal of destroying the total stockpile at each remaining site by the 
April 29, 2012, treaty deadline. Additionally, while the Army has 
recently established annual site goals for chemical agent tons 
destroyed, these targets have been linked either to Army or DOD 
schedules that far exceed the extended April 29, 2012, treaty deadline. 
Without this linkage, it is difficult for program officials to track 
the progress that the program is making toward its goals, and program 
officials have little assurance that program activities are linked to 
program goals. 

The program is using four different performance measurement approaches: 
the SAR, the Office of Management and Budget's (OMB) Program Assessment 
Rating Tool (PART),[Footnote 31] and DOD's and CMA's strategic plans. 
As this is a major acquisition program, some performance measures 
related to cost and schedule are contained in the SAR. However, while 
DOD and CMA officials cited the schedule milestones in the APB as the 
schedule the program is accountable to, neither the 2006 approved APB, 
the April 2006 SAR, nor the April 2007 SAR contained schedule milestone 
dates beyond April 2012, but rather stated that schedule milestones are 
"to be determined." DOD officials stated that the dates are not 
included in the APB because it was approved about 1 week before DOD 
officially announced to Congress that all sites would not meet the 
extended April 29, 2012, treaty deadline. Although 18 months (as of 
October 2007) have elapsed since DOD made that announcement, the dates 
have still not been included in these documents. DOD and CMA officials 
stated that the milestone dates are still being used for performance 
tracking despite not being included in the most recent SARs. While the 
program may have a measurement for tracking progress, its exclusion 
from the SAR limits program oversight by Congress. 

The program also developed additional performance measures beyond those 
used in the acquisition reporting system. For example, the program 
developed performance measures in (1) the Office of the Secretary of 
Defense's (OSD) strategic plan, (2) OMB's PART, and (3) CMA's strategic 
plan. However, these measures have not always been linked to each other 
and have not always been linked to key program goals. For example, 
while the achievement of destruction of the stockpile by the treaty 
deadline is an overall program goal in both the DOD and CMA strategic 
planning documents, neither DOD nor CMA has developed annual 
performance measures that are linked to achieving that goal even though 
they had established goals linked to the interim 45 percent destruction 
goal. 

Leading organizations seek to establish clear hierarchies of 
performance goals and measures and link these goals and measures for 
each successive organizational level to the overall program 
goal.[Footnote 32] Our analysis of the various performance measures 
also determined that they were not always linked to one another or tied 
to key program goals. For example, while OSD identified 14 performance 
measures that support the four goals contained in the strategic plan 
that directly relate to the Chemical Demilitarization Program's 
stockpile program, CMA's performance measures are not always aligned 
with OSD's performance measures. Also, while OSD identified performance 
measures that are similar to those tracked by OMB in the PART, two 
performance measures in the PART are not included in OSD's strategic 
plan. These two measures were the planned cost per ton of agent 
destroyed against actual cost and the statistical confidence of meeting 
the 45 percent CWC deadline. The SAR contained performance measures 
related to safety and environmental performance, such as those related 
to chemical exposures and releases, but did not have performance 
measures related CWC tonnage goals. Table 3 compares measures 
identified in the PART, the OSD strategic plan, the CMA strategic plan, 
and the SAR. 

Table 3: Comparison of Performance Measures Identified in the PART, OSD 
Strategic Plan, CMA Strategic Plan, and Selected Acquisition Report: 

OMB PART measures: Number of chemical agent exposures; 
OSD strategic plan: No chemical agent exposures as defined in the APB; 
CMA strategic plan: Number of chemical exposures as defined in the APB; 
SAR (CMA and Newport): Number of chemical agent exposures. 

OMB PART measures: Number of chemical agent releases; 
OSD strategic plan: No chemical releases as defined in the APB; 
CMA strategic plan: Number of chemical agent releases as defined in the 
APB; 
SAR (CMA and Newport): Number of chemical agent releases. 

OMB PART measures: Tons of chemical agent destroyed against CWC 45 
percent destruction deadline; 
OSD strategic plan: Percentage of actual amount of stockpiled chemical 
materiel destroyed against CWC 45 percent destruction deadline; 
CMA strategic plan: No tonnage destroyed performance measure against 
CWC 45 percent destruction deadline; 
SAR (CMA and Newport): No CWC tonnage destroyed performance measure. 

OMB PART measures: Tons of chemical agent destroyed against CWC 100 
percent destruction deadline; 
OSD strategic plan: Percentage of actual amount of stockpiled chemical 
materiel destroyed against CWC 100 percent destruction deadline; 
CMA strategic plan: No tonnage destroyed performance measure against 
CWC 100 percent destruction deadline; 
SAR (CMA and Newport): No CWC tonnage destroyed performance measure. 

OMB PART measures: Recordable incidence rate; 
OSD strategic plan: Recordable incidence rate; 
CMA strategic plan: Recordable incidence rate; 
SAR (CMA and Newport): Meet Army, state, and/or federal requirements. 

OMB PART measures: Percentage of stockpile destroyed against CWC 45 
percent destruction deadline; 
OSD strategic plan: No related measure; 
CMA strategic plan: Statistical confidence in achieving CWC 45 percent 
destruction deadline; 
SAR (CMA and Newport): No related measure. 

OMB PART measures: Planned cost per ton of agent destroyed against 
actual cost; 
OSD strategic plan: No related measure; 
CMA strategic plan: Plan cost per ton of agent destroyed against actual 
cost; 
SAR (CMA and Newport): No related measure. 

Source: GAO analysis of DOD and CMA data. 

[End of table] 

This lack of integration between key program goals and performance 
measures hampers program officials' ability to effectively track the 
progress that the program is making because the performance plan lacks 
destruction goals, approaches, and milestones that are directly linked 
to overall program goals for safely meeting CWC deadlines. 

Recently Improved Destruction Rates for Chemical Munitions May Indicate 
Overly Conservative Schedule Milestones at the Four Incineration Sites: 

Since DOD extended the program office schedules and DOD schedule 
milestones in 2005, actual processing rates have considerably exceeded 
the forecasted rates at the four incineration sites, indicating that 
schedule milestones being used to measure the program's progress may be 
too conservative to be realistic. DOD's Chemical Demilitarization 
Program schedule has been extended in recent years largely because of 
slower-than-expected processing rates for destroying chemical agent- 
filled munitions. Program office schedules were extended in 2005 
because of a greater recognition of slower processing rates for 
destroying chemical munitions than schedule milestones adopted in the 
more optimistic 2003 APB. In addition, the program office employed a 
new schedule forecasting methodology that also contributed to the 
schedule extensions because it accounted for more equipment downtime 
and for the risks that have delayed processing in the past. The 
schedule extensions exceeded the existing 2003 baseline and led DOD to 
adopt new schedule milestones in 2006. Finally, program confidence 
levels for meeting the April 29, 2012, CWC milestone are low. 

Slower-Than-Anticipated Processing Rates and Time Frames Led to 
Schedule Extensions: 

The program office schedules have been extended since 2003 largely 
because forecasted processing rates for destroying chemical agent- 
filled munitions were reduced based on the program's analysis of 
historical processing rates being slower than anticipated. Our review 
of best practices indicates that estimating assumptions should be 
realistic and backed by historical data to minimize uncertainty and 
risk.[Footnote 33] Best practices also stipulate that estimates are 
accurate when they are not overly conservative or optimistic. Best 
practices also recommend that when underlying data or other assumptions 
change, cost and schedule estimates should be revised to reflect the 
current status. According to program officials, previous program office 
schedule estimates developed in the early 2000s were based on 
assumptions about processing rates derived largely from peak processing 
experiences at Johnston Atoll, which began operations in 1990, and 
early munitions processing campaigns at Tooele--the first two sites in 
operation. According to CMA officials, program office schedule 
estimates used from 2001 to 2003 were derived from using the 5 best 
weeks of these two sites' production rates. The program's assumption 
was that the follow-on sites, which were similarly designed, would be 
able to achieve these rates because of the program's years of 
experience with the various munitions and agents. Program officials 
said that in practice those rates were achievable only for short 
periods and were not sustainable because of problems with individual 
munitions, such as difficulty in draining the agent (much slower than 
assumed), equipment jams, and processing restrictions specified in 
facility operating permits. 

New Schedule Methodology Includes More Time for Downtime and Processing 
Risks: 

The program office adopted a new schedule development methodology in 
2004 that not only used modeling to determine processing rates, but 
gave greater recognition to the issues that cause delays, such as plant 
downtime days for equipment maintenance and the inclusion of risks, 
such as those associated with the unexpected condition of the stored 
munitions that further extended the schedule. While previous schedule 
estimates were point estimates, in that a single processing rate was 
developed and used for each campaign and the individual campaigns were 
added together, statistical modeling was used to develop a distribution 
of different schedule outcomes. Processing rates were evaluated by 
munitions type, such as projectiles, rockets, and mines, to develop a 
histogram of daily processing outcomes.[Footnote 34] These results were 
used as part of the modeling inputs. For current estimates, unlike past 
estimates, schedule estimators also collected data on issues that had 
caused processing delays in the past and identified possible impacts on 
the schedule and probabilities of occurrence. 

Data were also collected on past program incidents that had caused 
sites to remain idle for extended periods, and the estimators also 
developed probabilities for these "downtime events." The model was run 
to produce a distribution of 5,000 schedule outcomes. The median 
result--referred to as the 50 percent confidence level--was selected as 
the program office schedule estimate. Selection of the median result 
was also considered conservative, since the previous program office 
schedules have used the mode, which is the outcome that happens most 
frequently. According to schedule estimators, the median result is 
typically to the right of or longer than the mode. 

Thus, when the schedule estimates were updated in 2005 using the median 
schedule outcomes, they were not only more conservative, but also 
considered more realistic. Figure 5 shows how processing rates, 
downtime, and risk contributed to the schedule estimate. 

Figure 5: 2005 Program Office Schedule Estimates by Processing Rates, 
Downtime, and Risk: 

This figure is a bar graph showing 2005 program office schedule 
estimates by processing rates, downtime, and risk. 

[See PDF for image] 

Source: CMA. 

[End of figure] 

Program Office Schedule Extensions Led to Revised Program Baseline: 

When these schedule estimates exceeded the schedule milestones 
contained in the April 2003 APB and resulted in major changes to the 
life cycle cost estimates, an acquisition-related process was initiated 
to develop new schedule milestones. The new schedule estimate was 
approved by the Army Cost Review Board in May 2005 and resulted in the 
new Army Cost Position. Concurrently, DOD was also conducting a program 
review through its Defense Acquisition Board process that would result 
in new schedule estimates to be included in the program's Approved 
Acquisition Program Baseline. As part of this process, the DOD's Cost 
Analysis Improvement Group (CAIG) developed independent schedule 
estimates.[Footnote 35] CAIG's schedule forecasts were also based on 
modeling to select the median outcome. Unlike those based on the 
program rebaselining that occurred from 2001 to 2003, the program 
office schedule estimates and those developed by CAIG were relatively 
close in most cases. The schedule milestones in the 2003 APB are 
compared with those in the 2005 Army and DOD schedule estimates in 
table 4. 

Table 4: Comparison of Estimated Operation Completion Dates: 

Site: Anniston; 
2003 Acquisition Program Baseline: May 2011; 
2005 Army program office estimate: September 2016; 
Difference from 2003 Acquisition Program Baseline: 64 months; 
2005 DOD independent estimate: June 2016; 
Difference from 2003 Acquisition Program Baseline: 61 months; 
Program current estimate: May 2015. 

Site: Newport; 
2003 Acquisition Program Baseline: January 2006; 
2005 Army program office estimate: October 2011; 
Difference from 2003 Acquisition Program Baseline: 75 months; 
2005 DOD independent estimate: February 2012; 
Difference from 2003 Acquisition Program Baseline: 73 months; 
Program current estimate: October 2009. 

Site: Pine Bluff; 
2003 Acquisition Program Baseline: November 2009; 
2005 Army program office estimate: January 2015; 
Difference from 2003 Acquisition Program Baseline: 62 months; 
2005 DOD independent estimate: November 2015; 
Difference from 2003 Acquisition Program Baseline: 72 months; 
Program current estimate: June 2013. 

Site: Tooele; 
2003 Acquisition Program Baseline: February 2008; 
2005 Army program office estimate: October 2014; 
Difference from 2003 Acquisition Program Baseline: 80 months; 
2005 DOD independent estimate: September 2015; 
Difference from 2003 Acquisition Program Baseline: 91 months; 
Program current estimate: November 2014. 

Site: Umatilla; 
2003 Acquisition Program Baseline: January 2011; 
2005 Army program office estimate: June 2017; 
Difference from 2003 Acquisition Program Baseline: 77 months; 
2005 DOD independent estimate: March 2017; 
Difference from 2003 Acquisition Program Baseline: 74 months; 
Program current estimate: December 2014. 

Source: GAO analysis of DOD and CMA data. 

[End of table] 

Recently Improved Processing Rates May Make the Current Schedule 
Milestones Overly Conservative, but Program Confidence Levels for 
Meeting Treaty Milestones Are Low: 

Our review of processing rates that have been achieved at the sites 
since the program office estimates were developed in 2005 determined 
that most sites are significantly ahead of the program office estimate 
for recently completed munition campaigns, indicating that schedule 
milestones being used to measure the program's progress may be too 
conservative to be realistic. For example, the 2005 program office 
schedule forecasted that Umatilla would complete destruction of its 
91,442 sarin nerve agent rocket munitions by August 2007. Those 
operations were actually completed in August 2006--about 12 months 
ahead of the program office 2005 estimate. This processing rate was 
achieved despite delays to the site operator internal schedule related 
to the unanticipated fires associated with some of the rockets. 
Additional progress made through May 2007 has placed Umatilla about 
1,184 days ahead of 2005 program office schedule estimate. We found 
similar occurrences at the other incineration sites, and actual and 
near-term projected performance by sites reveals that significant 
schedule progress has been made since the 2005 schedule-estimating 
process. Table 5 shows actual and projected schedule savings as 
compared to the 2005 program office estimated schedule for campaigns 
recently completed or under way. 

Table 5: Number of Days Saved per Site since 2005 Program Office 
Schedule Estimates: 

Site: Anniston; 
Cumulative days saved: 443; 
Actual or projected days saved: Actual; 
End date of last completed campaign completed to date: March 8, 2007; 
Campaigns ongoing and completed or started and completed from May 2005 
through July 2007: GB 8-inch projectiles; 
GB 155-mm projectiles; 
GB M360 105-mm projectiles; 
VX M55 rockets/warheads. 

Site: Newport; 
Cumulative days saved: 658; 
Actual or projected days saved: Projected; 
End date of last completed campaign completed to date: December 2, 
2009[A]; 
Campaigns ongoing and completed or started and completed from May 2005 
through July 2007: VX agent neutralization. 

Site: Pine Bluff; 
Cumulative days saved: 375; 
Actual or projected days saved: Actual; 
End date of last completed campaign completed to date: May 19, 2007; 
Campaigns ongoing and completed or started and completed from May 2005 
through July 2007: GB M55 rockets/ warheads. 

Site: Tooele; 
Cumulative days saved: 61; 
Actual or projected days saved: Actual; 
End date of last completed campaign completed to date: August 17, 2006; 
Campaigns ongoing and completed or started and completed from May 2005 
through July 2007: VX land mines; 
Agent changeover. 

Site: Umatilla; 
Cumulative days saved: 1,184; 
Actual or projected days saved: Actual; 
End date of last completed campaign completed to date: January 24, 
2007; 
Campaigns ongoing and completed or started and completed from May 2005 
through July 2007: GB M55 rockets/ warheads; 
GB MC-1 bombs; 
GB MK-166 bombs; 
GB 8-inch projectiles; 
GB 155-mm projectiles. 

Site: Aberdeen; 
Cumulative days saved: 750; 
Actual or projected days saved: Actual; 
End date of last completed campaign completed to date: February 13, 
2006; 
Campaigns ongoing and completed or started and completed from May 2005 
through July 2007: Ton container cleanout. 

Total; 
Cumulative days saved: 3,471; 
Actual or projected days saved: [Empty]; 
End date of last completed campaign completed to date: [Empty]; 
Campaigns ongoing and completed or started and completed from May 2005 
through July 2007: [Empty]. 

Source: GAO analysis of Army data. 

[A] Figures listed above for Anniston, Pine Bluff, Tooele, and Umatilla 
represent actual savings. Figures for Newport are projected savings for 
current ongoing campaigns based on the historical processing rates at 
those sites. The Newport forecast for completing the current campaign 
is December 2009. 

[End of table] 

We calculate that to date, almost $1.3 billion in cost savings has 
accrued based on schedule savings since the 2005 program office 
estimates were established. Table 6 shows actual and projected schedule 
savings as compared to the 2005 estimated schedule savings for 
campaigns recently completed or under way. 

Table 6: Potential Operational Cost Savings Associated with Days Saved 
since 2005 Schedule Estimate: 

Dollars in millions. 

Anniston; 
Cumulative days saved: 443; 
Potential cost savings: $178. 

Newport; 
Cumulative days saved: 658; 
Potential cost savings: 227[A]. 

Pine Bluff; 
Cumulative days saved: 375; 
Potential cost savings: 144. 

Tooele; 
Cumulative days saved: 61; 
Potential cost savings: 25. 

Umatilla; 
Cumulative days saved: 1,184; 
Potential cost savings: 452. 

Aberdeen; 
Cumulative days saved: 750; 
Potential cost savings: 236. 

Total; 
Cumulative days saved: 3,471; 
Potential cost savings: $1,262. 

Source: GAO analysis of Army data. 

Note: Time savings were converted to a cost by using the average fiscal 
year 2005 and fiscal year 2006 budgets expressed in dollars per day. 

[A] Figures listed above for Anniston, Pine Bluff, Tooele, and Umatilla 
represent actual savings. Figures for Newport are projected savings for 
current ongoing campaigns based on the historical processing rates at 
those sites. The Newport forecast for completing the current campaign 
is December 2009. 

[End of table] 

The April 2007 SAR for the CMA sites (including Newport) reported new 
program office schedule estimates, which were a significant improvement 
from the Army's 2005 program office schedule estimates that generated 
the program's rebaselining and also were significantly ahead of the 
independent DOD schedule estimates that are used as the program's 
unofficial schedule baseline and were developed by CAIG. Table 7 shows 
the number of months that the estimated schedule for completing 
operations has improved from 2005 to 2007. 

Table 7: Number of Months of Improvement in 2007 Program Current 
Estimate for Operations Completion versus 2005 Army Program Office 
Estimate and 2005 DOD Independent Estimate: 

Site: Anniston; 
2005 Army program office estimate: September 2016; 
2005 DOD independent estimate: June 2016; 
2007 program current estimate: May 2015; 
Improvement from 2005 program office estimate: 16 months; 
Improvement from 2005 independent DOD estimate: 13 months. 

Site: Newport; 
2005 Army program office estimate: October 2011; 
2005 DOD independent estimate: February 2012; 
2007 program current estimate: October 2009; 
Improvement from 2005 program office estimate: 24 months; 
Improvement from 2005 independent DOD estimate: 28 months. 

Site: Pine Bluff; 
2005 Army program office estimate: January 2015; 
2005 DOD independent estimate: November 2015; 
2007 program current estimate: June 2013; 
Improvement from 2005 program office estimate: 19 months; 
Improvement from 2005 independent DOD estimate: 29 months. 

Site: Tooele; 
2005 Army program office estimate: October 2014; 
2005 DOD independent estimate: September 2015; 
2007 program current estimate: November 2014; 
Improvement from 2005 program office estimate: (1 month); 
Improvement from 2005 independent DOD estimate: 10 months. 

Site: Umatilla; 
2005 Army program office estimate: June 2017; 
2005 DOD independent estimate: March 2017; 
2007 program current estimate: December 2014; 
Improvement from 2005 program office estimate: 30 months; 
Improvement from 2005 independent DOD estimate: 27 months. 

Source: GAO analysis of DOD and CMA data. 

[End of table] 

In addition to its annual schedule update process, CMA tracks on a 
monthly basis the confidence levels of achieving various program 
schedule estimates. CMA tracks these levels after updating its model to 
incorporate updated processing rates and any changes to estimates for 
downtime or risk. These updates have shown upward trends in the 
confidence level, which is the percentage of schedule outcomes that 
meet or exceed a given date. As previously discussed, the 2005 program 
office estimate reflected a 50 percent confidence level. By the end of 
fiscal year 2006, which was about 18 months after the 2005 program 
office estimates were developed, only Tooele, at 48 percent, was 
slightly below the 50 percent confidence level. The other three sites' 
confidence levels for achieving the 2005 program office estimate for 
completing operations had increased significantly, ranging from 71 
percent at Anniston to 83 percent at Umatilla. Moreover, confidence 
levels based on monthly updates conducted in April 2007 showed that the 
program office estimates for meeting the DOD schedule milestones were 
as high as 98 percent for completing operations, but confidence levels 
were very low for meeting the April 29, 2012, treaty deadline, as shown 
in table 8. 

Table 8: CMA Assessed Confidence in Achieving DOD Schedule Milestone 
and April 29, 2012, Treaty Deadline: 

Percentages: Site: Anniston; 
Program's assessed confidence in achieving operations schedule 
milestone: 73; 
Program's assessed confidence in sites completing destruction by April 
29, 2012, CWC milestone: 5. 

Percentages: Site: Newport; 
Program's assessed confidence in achieving operations schedule 
milestone: 98; 
Program's assessed confidence in sites completing destruction by April 
29, 2012, CWC milestone: 98. 

Percentages: Site: Pine Bluff; 
Program's assessed confidence in achieving operations schedule 
milestone: 81; 
Program's assessed confidence in sites completing destruction by April 
29, 2012, CWC milestone: 19. 

Percentages: Site: Tooele; 
Program's assessed confidence in achieving operations schedule 
milestone: 42; 
Program's assessed confidence in sites completing destruction by April 
29, 2012, CWC milestone: 1. 

Percentages: Site: Umatilla; 
Program's assessed confidence in achieving operations schedule 
milestone: 84; 
Program's assessed confidence in sites completing destruction by April 
29, 2012, CWC milestone: 2. 

Source: CMA. 

[End of table] 

While the program office has updated its schedule estimates to reflect 
the latest processing rates and changes to other assumptions, resulting 
in schedule improvements in excess of 18 months in most cases, 
independent schedule estimates have not been developed since the CAIG 
2005 estimates because these estimates are usually performed in 
conjunction with program rebaselinings because of schedule or cost 
growth. Also, when schedules or other assumptions change, cost and 
schedule estimates should be revised to reflect current status. Because 
these steps have not been taken, DOD and Congress can have little 
confidence that the program is being held accountable to a reasonable 
baseline to measure program progress and that program costs are 
reasonable. 

Shortcomings in Underlying Cost Data Make Earned Value Management 
Results Unreliable and Limited Our Ability to Determine Cost Estimate 
Reliability: 

Because of shortcomings in the underlying data, the usefulness of the 
EVM data to make cost projections is limited, which makes results 
unreliable, and our ability to independently verify and validate the 
reliability of the cost estimates was restricted. The Chemical 
Demilitarization Program's projected cost growth is largely 
attributable to longer schedules and increased costs associated with 
facility closure estimates. Shortcomings in the underlying data include 
invalid and unstable performance baselines. The performance baselines 
are problematic because all requirements, such as closure costs, are 
not fully identified and included in program cost estimates and because 
large amounts of additional costs are added to the performance baseline 
annually. Moreover, there were significant differences between the 
systems contractor cost estimates through completion and the government 
cost estimates through completion because the government anticipates 
longer operational periods and greater closure requirements. 

Projected Cost Growth Is Largely Attributable to Longer Schedules and 
Increased Facility Closure Estimates: 

The Chemical Demilitarization Program's continuing growth in cost 
estimates is largely attributable to longer schedules and the increased 
costs associated with plant closure. Longer schedules led to increases 
in cost estimates for sites currently in the operations phase because 
of additional labor costs, which are a high percentage of site costs. 
In addition, according to CMA officials, labor costs are very expensive 
for this program because of the need for highly specialized labor that 
cannot be easily replaced. According to these officials, each staff 
member, for example, must pass the Personal Reliability Program, which 
provides specialized training and security clearances for working at 
the chemical deconstruction plants. Program officials have stated that 
for this reason, it is often more cost-effective during plant shutdowns 
or extended delays to retain this specialized workforce than to lay off 
and rehire new staff. This workforce costs approximately $9.5 million 
per month based on an average monthly spend rate across the five sites 
we examined, which means that a 1-month schedule extension could 
increase the life cycle cost estimates by about that amount. Table 9 
shows the increase in program cost as the schedule was extended. 

Table 9: Changes in Estimated Total Program Cost: 

Then-year dollars in billions. 

Total life cycle cost estimate; 
2003 APB: $19.6; 
2005 Army cost position: $29.2; 
Increase: $9.6; 
April 2007 SAR: $28.6. 

Source: U.S. Army. 

[End of table] 

While the Chemical Demilitarization Program Is Using Earned Value 
Management as a Tool, Some Underlying Data Make Results Unreliable: 

EVM is an important program management tool required by DOD and OMB 
that measures performance by comparing the value of work accomplished 
with the value of work planned and the cost of the work performed. 
Using EVM data, early warning of schedule delays and cost overruns can 
be determined. EVM uses contractor-reported data to examine variances 
reported in contractor performance reports between actual cost and time 
of performing work tasks and the budgeted or estimated cost and time. 
As a result, DOD and OMB policy and guidance require the use of EVM to 
measure program performance.[Footnote 36] 

The Chemical Demilitarization Program uses EVM[Footnote 37] as a tool 
that allows both government and systems contractor managers to have 
visibility over cost performance on their contracts. However, the 
ability of the program's EVM system to reliably manage and forecast the 
program's cost estimates may be affected by large variances between the 
systems contractors' estimated cost to complete operations and closure 
versus the government's estimate; systems contractors' baselines that 
have not been carefully controlled, leading to high percentages of 
authorized work that is unpriced for long periods of time; and limited 
identification of requirements for closure costs at individual sites. 

Because of the shortcomings in the underlying cost data, the usefulness 
of the EVM data to make cost projections is limited and our ability to 
independently verify and validate the reliability of the cost estimates 
was restricted. In other words, the mechanics of the EVM system are 
working properly; however, we found that the performance management 
baseline does not accurately reflect the remaining life cycle costs of 
the Chemical Demilitarization Program. DOD's EVM implementation guide 
states that "the "to-go" plan should reflect a realistic schedule of 
how the remaining work actually is to be done and the new budget should 
be adequate and reflect a realistic estimate, remaining program risks, 
and contain an appropriate amount of management reserve."[Footnote 38] 
According to best practices, an EVM system should be devoid of data 
errors and anomalies that may distort the analysis.[Footnote 39] In 
addition, any cost or schedule variances should be explained and 
corrective actions identified. Furthermore, the EVM system should have 
a valid and stable performance baseline from which to make projections 
and do trend analysis.[Footnote 40] In the case of the Chemical 
Demilitarization Program, the cost account managers are inserting their 
costs into the system properly, system certification verifies that the 
EVM reported costs match the costs in the accounting system, the EVM 
reports are being prepared with minimal errors, and cost account 
managers are explaining any variances. However, the actual data are 
measured against a baseline that does not reflect all of the work to be 
performed, such as facility closure activities, which distorts cost and 
schedule performance metrics. This undermines the effectiveness of the 
EVM system in providing management with accurate and realistic metrics 
for making informed management decisions.[Footnote 41] 

Performance Baselines Lacked Validity and Have Been Unstable: 

Our review showed that the performance baselines lack validity and have 
not been stable because CMA and site systems contractors have not fully 
identified all requirements, such as closure costs, and included them 
in program cost estimates and because large amounts of additional 
costs[Footnote 42] are added to the performance baseline annually. EVM 
guidelines state that once a contract is awarded, the contractor should 
establish a performance baseline that will serve as a time-phased 
budget plan for accomplishing work and measuring contract 
performance.[Footnote 43] This is essentially the spending plan for the 
program and should closely equate to the planned budget, which is the 
value of work planned through the project's completion. The performance 
baseline should include all costs associated with completing the 
program, including direct and indirect labor, material, and other 
direct costs associated with the authorized work. However, we found 
that each of the sites was lacking a stable and reliable EVM 
performance baseline, which hindered our ability to perform an 
independent EVM analysis.[Footnote 44] 

While the planned budget and the performance baseline should closely 
equate, we found that the performance baseline is significantly less 
than the planned budget (close to $2 billion less) because most sites 
have significant amounts of work that are authorized by the systems 
contractors and the Army but are not yet placed on contract. For 
example, the April 2007 SAR reports that while the contract value at 
Umatilla is $1.2 billion through December 2006, an additional $700 
million has been authorized but is pending negotiation. Also, the 
Estimate at Completion (EAC) cost estimate, which represents all 
estimated costs through the end of the contracts, is $4 billion more 
than the performance baseline. Figure 6 shows the extent that planned 
budget and the EAC exceed the performance baseline for all sites 
combined over a 19-month period from August 2005 to February 2007. 

Figure 6: Comparison of the Performance Baseline with the Planned 
Budget and Systems Contractors' Estimate at Completion: 

This figure is a combination line graph comparing the performance 
baseline with the planned budget and systems contractors' estimate at 
completion. The X axis is the year and month, and the Y axis represents 
dollars in millions. One line represents planned budget, another 
represents estimate at completion, and another represents performance 
baseline. 

[See PDF for image] 

Source: GAO analysis of Army data. 

[End of figure] 

The performance baseline does not include all requirements, such as 
closure costs, which are not fully identified and included in program 
cost estimates. Best practices state that a cost estimate should be 
comprehensive in order to be considered credible.[Footnote 45] Systems 
contractor officials stated that in many instances what was included in 
their estimated completion costs were placeholder amounts, which they 
believed to be unrealistic. According to program office officials, the 
government and the systems contractors have not yet completely defined 
the requirements for closure, so a realistic closure cost estimate is 
not currently available. However, since two sites--Johnston Atoll and 
Aberdeen--have already been closed historical data are now available to 
develop more accurate closure estimates at the remaining 
sites.[Footnote 46] For example, based on the Army's experience of 
destroying blister and nerve agents at Johnston Atoll, closure costs 
for other destruction sites are increasing from initial estimates. As a 
result of incorporating these actual data, assumptions for conducting 
closure activities have increased from 12 months to about 30 months at 
some sites. 

The performance baseline also is unstable because large amounts of 
additional costs are added to the performance baseline 
annually.[Footnote 47] Each fiscal year, a portion of the unpriced work 
that is part of the planned budget is negotiated (priced), and it is 
moved into the performance baseline. The negotiated funding is often 
greater than what was used as a "placeholder" budget, increasing the 
contract baseline to more than what was included as unpriced work. For 
example, the fiscal year 2006 negotiated increase in the contract value 
at Anniston was $6 million more than the reduction in authorized 
unpriced work. Program office officials explained that the performance 
baseline at each site is constantly changing because of yearly contract 
renegotiations[Footnote 48] for future operations and closure costs. 
Program officials attribute the yearly changes to the contracts using 
Operating and Maintenance funds, which expire within 1 fiscal year. 
This situation created the need to put on contract a year's worth of 
work in conjunction with each fiscal year's appropriation. 

CMA officials stated that they are making efforts to stabilize the 
baseline by placing the full life cycle costs associated with all 
activities at each site. In its technical comments on a draft of this 
report, DOD stated that three of the five CMA sites (Anniston, Pine 
Bluff, and Umatilla) had completed contract performance baselines and 
negotiations were ongoing at the two remaining sites (Newport and 
Tooele). DOD expected that the negotiations for these two sites would 
be completed by June 30, 2008. Once this happens, CMA officials believe 
that there will be less of a need for annual negotiations and more 
realistic performance baselines should result. Until program management 
has a performance baseline that reflects the current plan to complete 
remaining work and follows a disciplined process for incorporating 
changes, management will not be able to benefit from the EVM system's 
ability to reveal actual program performance and detect program risks. 

Program Manager-Estimated Completion Costs Are Significantly Higher 
Than Completion Costs Generated from EVM Data, Suggesting Data Are 
Incomplete: 

It is a best practice to develop a range of independent EACs using 
commonly accepted EVM equations because doing so informs management of 
the likely range of costs for completing the program so appropriate 
action can be taken.[Footnote 49] In addition, DOD guidance states that 
remaining work should reflect a realistic schedule and budget that also 
reflect remaining risks.[Footnote 50] Finally, EVM guidelines state 
that program managers should, at least on a monthly basis, rely on EVM 
data to generate EACs that are derived based on the cost of work 
completed to date along with an estimate of what it will cost to 
complete the rest of the program.[Footnote 51] By relying on past 
performance and using well-defined EVM indices, a variety of EACs can 
be generated. However the EACs generated are only as good as the 
underlying data in the EVM system. 

Based on our analysis, program cost estimates often exceeded the worst 
case of our range of independently developed completion costs estimates 
because of shortcomings in the underlying data of the EVM systems 
maintained at the chemical demilitarization sites. It is a best 
practice to develop a range of independent cost estimates using 
commonly accepted EVM equations. Doing so informs management of the 
likely range of costs for completing the program so that appropriate 
action can be taken. We found that when we developed independent EACs 
using the systems contractor's EVM data and commonly accepted EVM 
forecasting equations in all cases the systems contractors' EACs are 
either at the upper end of or higher than our estimated EAC range based 
on our analysis shown in table 10. In addition, the program manager's 
EAC was far outside of our projected range and, in many cases, was 
almost twice as high as our EAC. For example, our estimated cost range 
for Anniston based on the EVM data was from $1.6 billion to $1.7 
billion. The systems contractor's most likely cost estimate was $100 
million above the upper limit of our range, and the program manager's 
estimate was $1.4 billion higher than the upper range limit. We 
attribute this difference to an understated performance baseline, which 
is critical to the estimating process. Table 10 compares our 
independent EAC to that of the systems contractor's and program 
manager's EACs. 

Table 10: Comparison of GAO's Independent Estimates to Complete 
Operations and Closure Activities with the Systems Contractor's and 
Program Manager's Estimates: 

Then-year dollars in billions. 

Anniston; 
Original contract cost: $0.6; 
Current planned budget: $1.7; 
Systems contractor's most likely EAC: $1.8; 
GAO's independent EAC range: $1.6-$1.7; 
2006 Selected Acquisition Report program manager's EAC: $3.1. 

Newport; 
Original contract cost: 0.3; 
Current planned budget: 1.0; 
Systems contractor's most likely EAC: 1.1; 
GAO's independent EAC range: 0.9-1.1; 
2006 Selected Acquisition Report program manager's EAC: 1.8. 

Pine Bluff; 
Original contract cost: 0.5; 
Current planned budget: 1.5; 
Systems contractor's most likely EAC: 1.9; 
GAO's independent EAC range: 1.4-1.5; 
2006 Selected Acquisition Report program manager's EAC: 2.4. 

Tooele; 
Original contract cost: 0.2; 
Current planned budget: 1.6; 
Systems contractor's most likely EAC: 2.4; 
GAO's independent EAC range: 1.4-1.6; 
2006 Selected Acquisition Report program manager's EAC: 3.3. 

Umatilla; 
Original contract cost: 0.5; 
Current planned budget: 1.8; 
Systems contractor's most likely EAC: 1.9; 
GAO's independent EAC range: 1.6-1.8; 
2006 Selected Acquisition Report program manager's EAC: 3.8. 

Source: GAO analysis of Army data. 

[End of table] 

In addition, the large difference between the program manager's EAC and 
our independent EACs indicates that there is more work that is being 
considered by the program manager. More work, in turn, will affect 
future costs and requirements than is predicted by the EVM data and, 
therefore, calls into question the usefulness of the EVM data to 
effectively manage the program.[Footnote 52] The ability to respond 
quickly to program problems depends on having early visibility into 
what is causing them. Thus, if the EVM performance baseline is missing 
some of the remaining work, then accurate progress assessments will not 
be available to provide management with a better picture of program 
status. This lack of insight may hinder decision making and program 
success because problems may go undetected. 

Without accurate and realistic closure requirements used to develop 
completion cost estimates, DOD and Congress will not be able to 
accurately identify and adequately budget for all activities through 
facility closure. Similarly, without an independent cost review once 
the cost estimates are complete and schedule milestones have been 
updated and reviewed, DOD and Congress will not have assurance that 
cost estimates for the closure of chemical demilitarization sites are 
realistic. 

The Army Has Actively Managed Award Fees and Could Further Emphasize 
Schedule and Cost and Link Award Fees to More Objective Performance 
Measures: 

Our review of the Army's use of monetary fees to incentivize certain 
aspects of the systems contractors' performance at the five sites in or 
near beginning operations in 2003 showed that the contract monetary fee 
structures at the incineration sites differed from those at Newport's 
neutralization site. Award fees are predominant at the incineration 
sites while incentive fees are predominant at Newport. We found that 
while the Army has actively reviewed and adjusted the types of monetary 
fees used and the criteria and weighting for determining monetary fees, 
there be may be opportunities to further emphasize schedule and cost. 
For example, during the period we reviewed, award fees have emphasized 
safety and environmental compliance at each site, but have not resulted 
in the control of schedule and cost growth. In fiscal year 2006, after 
several external organizations recommended that it do so, the Army 
reinstituted the use of a distinct award fee performance measure for 
schedule.[Footnote 53] However, site project managers often weighted 
schedule at the lower end of the specified range. Also, while DOD has 
published guidance that award fees should be tied to identifiable 
interim outcomes, discrete events, or milestones as much as possible, 
we found that several performance measures, such as management and 
cost, had evaluation factors that were all or mostly all subjective and 
were not clearly linked to a measurable outcome. In addition to award 
fees, in 2005, the Army established the Director's Programmatic 
Performance Based Incentive (DPPBI), a performance-based incentive fee 
to encourage the incineration site systems contractors to establish a 
collaborative relationship with the goal to share lessons learned in 
areas that would result in schedule reduction across all sites. Before 
fiscal year 2007, this fee was based on subjective schedule performance 
measures. By fiscal year 2007, DPPBI had evolved to focus mostly on the 
sites' collective schedule performance linked to the CWC treaty 
deadline. It is unclear how effective this incentive fee will be in 
motivating individual systems contractors' schedule performance due to 
the collective manner in which the fee is earned, its relative size, 
and recent focus on objective schedule performance. An additional 
incentive fee, authorized by section 923 of the National Defense 
Authorization Act for Fiscal Year 2007, provides up to $165 million at 
each incineration site for the completion of destruction and facility 
closure by or close to the CWC deadline. This incentive fee, as 
planned, is structured to accelerate destruction operations and 
facility closure at the incineration sites. Because this fee has not 
yet been implemented, we have no basis at this time to determine how 
effective this incentive might be at controlling schedule. 

Contract Monetary Fee Structures at the Incineration Sites Differ from 
Those at Newport's Neutralization Site, and Award Fees Are Predominant 
at the Incineration Sites: 

CMA uses cost-plus-award-fee contracts with incentive fees to encourage 
and reward the systems contractors to perform in an innovative, 
efficient, and effective way in areas deemed important to the program's 
success. Under these contracts, the systems contractors' allowable, 
incurred costs are reimbursable to the extent prescribed in the 
contract. According to CMA officials, the Chemical Demilitarization 
Program systems contracts are cost reimbursable, based on the fact that 
it is not possible to accurately describe the requirements in 
sufficient detail to address the myriad risks and uncertainties 
associated with destroying aging and deteriorating munitions that are 
filled with chemical agent. As a result of this high-risk environment, 
the government accepts most of the cost risk. Thus, when problems 
arise, such as rocket fires or mercury contamination, government and 
systems contractor resources are focused on fixing the problems or 
mitigating the risk rather than fixing blame. 

The base fee for this program is a fixed amount of money negotiated at 
the beginning of each year of contract operations, and is usually paid 
on a monthly basis to the systems contractor for its performance, 
though this fee is not tied to specific performance criteria as are the 
award and incentive fees. According to program officials, the rationale 
for providing a base fee is due to the program's complexity and to 
provide the systems contractors with a level cash flow for financial 
viability. Award fees are intended to motivate systems contractor 
performance in areas that are susceptible to judgmental and qualitative 
measurement and evaluation and, as a result, award fee criteria and 
evaluations tend to be subjective. Typically, award fee contracts 
emphasize multiple aspects of contractor performance in a wide variety 
of areas, such as quality, timeliness, technical ingenuity, and cost- 
effective management.[Footnote 54] Finally, incentive fee contracts use 
what is considered an objective evaluation of the systems contractor's 
performance to adjust the fee paid, which usually involves applying a 
fee determination formula specified in the contract. 

The contracts at each of the four operational incineration sites-- 
Anniston, Pine Bluff, Tooele, and Umatilla--include (1) a base fee 
(representing 3 percent of the negotiated total cost of the contract), 
which is a fixed amount not subject to performance evaluation; (2) an 
award fee that is 7 percent of the negotiated contract costs, which 
will be determined through performance evaluations and paid from an 
award fee pool; and (3) DPPBI, a performance-based incentive fee 
established in 2005 and paid from an available fee pool amount that is 
2 percent of negotiated contract cost. The contract at the remaining 
neutralization site--Newport--has a base fee (3 percent), an award fee 
(3 percent), and a performance-based incentive fee (6 
percent).[Footnote 55] All five contracts had performance-based 
incentive fees added after initial contract award. 

The types and percentages of fees offered to the systems contractors to 
safely destroy the chemical weapons stockpile differ and emphasize 
different performance measures. For example, at the incineration sites, 
the award fee is the predominant fee and, from fiscal year 2003 to 
fiscal year 2007, this fee has largely emphasized safety and surety, 
environmental compliance, and management goals. Conversely, at the 
neutralization site, the performance-based incentive fee is the 
predominant fee, which has emphasized schedule goals. Figure 7 compares 
the monetary incentive fees earned by the systems contractors at the 
incineration sites and at Newport. As shown in figure 7, in fiscal year 
2006, CMA paid the systems contractors at the incineration sites a 
combined total of about $39.9 million in fees, which consisted of about 
$31.1 million (or 78 percent) in award fees and about $8.8 million (or 
22 percent) in performance-based incentive fees. In contrast, CMA paid 
the neutralization site system contractor at Newport about $14.7 
million in monetary fees, which consisted of about $2.5 million (or 17 
percent) in award fees and upwards of about $12.1 million (or 83 
percent) in performance-based incentive fees. In fiscal year 2006, the 
award fees earned at each of the incineration sites were almost three 
times higher than that earned at Newport. 

Figure 7: Comparison of the Monetary Incentive Fees Earned by Systems 
Contractors at the Incineration Sites and at Newport during Fiscal Year 
2006: 

This figure is a combination of two pie charts comparing the monetary 
incentive fees earned by systems contractors at the incineration sites 
and at Newport during fiscal year 2006. 

Incineration sites' monetary fees for fiscal year 2006L $33.9 million: 

Award fee: 78%; 
Incentive fee: 22%. 

Newport monetary fees for fiscal year 2006: 

Award fee: 83%; 
Incentive fee: 17%. 

[See PDF for image] 

Source: U.S. Army Sustainment Command. 

Note: GAO did not independently verify the award and incentive fee data 
provided by the U.S. Army Sustainment Command. 

[End of figure] 

By predominately using performance-based incentives at Newport, the 
Army has aligned more of its monetary fees with objective outcomes. 
However, Army officials said that it is difficult to determine how 
effective the usage of performance-based incentive fees has been in 
motivating the systems contractors to meet schedule and cost milestones 
because of the number of extensions made to the original incentive 
agreement. The extensions were made to account for the technical 
challenges encountered by the systems contractors during the 
neutralization process. After the terrorist attacks of September 11, 
2001, due to Army concerns about security vulnerabilities, CMA 
restructured the contracts to increase the emphasis on performance- 
based incentives at the Newport and Aberdeen neutralization sites to 
incentivize the systems contractors to destroy the remaining stockpile 
of lethal chemical agent on an accelerated schedule.[Footnote 56] 

Army Actively Managed Award Fees and May Be Able to Further Emphasize 
Schedule and Cost: 

Since 2003, the period of our analysis, the Army has actively reviewed 
and adjusted criteria for determining monetary fees--largely award 
fees--it provided to the systems contractors to incentivize aspects of 
their performance. CMA's award fee plans generally use performance 
measures that could be expected to incentivize the desired acquisition 
outcomes. CMA includes broad aspects of systems contractor performance, 
such as management performance, and CMA also views these as keys to a 
successful program. From 2003 through 2006, CMA used safety and surety, 
environmental compliance, cost, and management as the key performance 
measures. Until fiscal year 2006, however, CMA did not emphasize 
schedule as a distinct performance measure in the award fee 
plans.[Footnote 57] 

Army Emphasized Safety and Environmental Compliance in Award Fee Plans 
and Did Not Successfully Incentivize Schedule and Cost Performance: 

From fiscal year 2003 through fiscal year 2006, the incineration sites 
and Newport had not been able to maintain original production schedule 
milestones and had experienced significant cost growth. Although the 
management performance measure included some criteria for schedule, 
during the period from fiscal year 2003 through fiscal year 2005, 
schedule was not a distinct performance measure in the award fee plan, 
and safety and environmental compliance were heavily weighted. CMA 
officials acknowledged in the 2007 incentive agreement plan that while 
award fees have succeeded in motivating the systems contractors to 
maintain high levels of safety and environmental compliance at each 
site,[Footnote 58] they have not satisfactorily resulted in the control 
of schedule and cost growth. Many events and constraints contributed to 
the schedule and cost growth, both within and beyond the systems 
contractors' control. CMA officials stated that the process to 
determine the degree to which such issues cause variances to baselines 
is never clear, but generally, over the course of a year, there may be 
25 percent of the problems that are clearly outside of the control or 
responsibility of the systems contractor, 10 percent that are clearly 
within the control of the systems contractor, and 65 percent that fall 
into the "gray area" where it is difficult to determine whether the 
problem is clearly outside or within the control of the systems 
contractor. 

From fiscal year 2003 through fiscal year 2006, the Army paid award 
fees to the systems contractors between 70 percent and 96 percent of 
the available award fee pools primarily for good-to-outstanding 
performance in safety and surety, environmental compliance, and 
management, which collectively constituted most of the basis for 
determining the award fee. Table 11 shows both the actual amount and 
the percentage of the total available award fee pool each site was paid 
by the Army over the period from fiscal year 2003--when all five sites 
were either in or preparing for the operations phase--to fiscal year 
2006. Based on our analysis of this period, the Army paid the systems 
contractors at three of the four incineration sites award fees between 
86 percent and 96 percent of the total award fee pool available during 
this period. The systems contractor at Tooele was the exception, 
earning only 70 percent of the total available award fee pool, 
according to the U.S. Army Sustainment Command, because of 
unsatisfactory ratings due to the lack of real progress in planning and 
preparing for the mustard campaign and in not meeting government 
expectations for managing costs. 

Table 11: Percentage of Award Fees Earned at the Incineration Sites and 
Newport, from Fiscal Year 2003 to Fiscal Year 2006: 

Chemical demilitarization facility: Anniston; 
Amount of award fee pool (2003-2006): $24,536,587; 
Amount of award fee earned (2003-2006): $23,671,150; 
Percentage of award fee earned (2003-2006): 96. 

Chemical demilitarization facility: Pine Bluff; 
Amount of award fee pool (2003-2006): 22,564,501; 
Amount of award fee earned (2003-2006): 19,633,254; 
Percentage of award fee earned (2003-2006): 87. 

Chemical demilitarization facility: Umatilla; 
Amount of award fee pool (2003-2006): 26,235,256; 
Amount of award fee earned (2003-2006): 22,638,970; 
Percentage of award fee earned (2003-2006): 86. 

Chemical demilitarization facility: Newport; 
Amount of award fee pool (2003-2006): 10,532,717; 
Amount of award fee earned (2003-2006): 8,734,713; 
Percentage of award fee earned (2003-2006): 83. 

Chemical demilitarization facility: Tooele; 
Amount of award fee pool (2003-2006): 27,917,118; 
Amount of award fee earned (2003-2006): 19,414,301; 
Percentage of award fee earned (2003-2006): 70. 

Chemical demilitarization facility: Total; 
Amount of award fee pool (2003-2006): $111,786,179; 
Amount of award fee earned (2003-2006): $94,092,388; 
Percentage of award fee earned (2003-2006): [Empty]. 

Source: U.S. Army Sustainment Command. 

[End of table] 

Anniston, for example, earned $23.7 million (or 96 percent) of the 
award fee during this period. According to an award fee evaluation 
board report, while the Anniston systems contractor worked aggressively 
to implement the schedule without compromising worker or public safety, 
the systems contractor faced challenges related to disciplined 
operations.[Footnote 59] The report noted that certain destruction 
campaigns at Anniston were characterized by schedule delays that 
resulted primarily from equipment reliability problems and, at times, 
inefficient planning and execution of repair efforts. The report 
further noted that while the efficiency of repairs was at times 
problematic, the systems contractor could not have prevented the 
equipment failure. The report also credited the systems contractor's 
risk mitigation efforts with significantly reducing the risk of a 6- 
month potential site shutdown that would have carried with it a 
corresponding cost impact of $75 million. Thus, CMA was able to justify 
paying the systems contractor nearly the full amount of available award 
fee. Also, during one period, Umatilla's systems contractor earned $2.5 
million (or 80 percent) of the available award fee pool while it had 
destroyed 36 percent of the nerve agent rockets planned for that 
period. According to an award fee evaluation board report, the 
processing shortfall was attributable to the systems contractor's lack 
of implementation of planned and emergent work in an efficient and 
effective manner, and was estimated to increase the planned operations 
schedule by 77 days.[Footnote 60] We have no basis for saying that the 
systems contractors should not have received their award fees, since 
they generally were evaluated as having met the award fee criteria, 
which at the time did not have schedule as a distinct performance 
measure. 

CMA officials have acknowledged that the existing award fee process has 
not satisfactorily resulted in the control of schedule and cost growth. 
CMA developed the schedule performance measure in response to Army 
Inspector General recommendations, and DOD guidance suggested that CMA 
focus more attention on schedule criteria. The Army's reinstatement of 
a distinct schedule performance measure coincided with the program's 
schedule extensions.[Footnote 61] Also, in fiscal year 2004, according 
to CMA, the Army Inspector General recommended that CMA develop 
quantifiable metrics and both the Army Inspector General and the 
Defense Contract Management Agency recommended that CMA place greater 
weight on cost and schedule control. CMA documents indicate that the 
schedule performance measure was developed after these recommendations 
and the Under Secretary's memorandum. 

Army Has Adjusted Award Fee Plans' Weighted Performance Measures and 
Could Further Increase the Weight of Performance Measures, Such as 
Schedule and Cost: 

The determination of award fee performance measures and related weight 
factors is part of CMA's award fee plan template development process, 
which is used as a guideline by site project managers to develop 
specific site award fee plans. The template describes specific 
criteria, standards, and procedures, and provides a range of weight for 
each performance measure to be used in the award fee plan to assess the 
systems contractors' performance and to determine the amount of award 
fee earned. The template allows the site project managers flexibility 
to select a weighted factor for each performance measure within the CMA 
template's range based upon the site project manager's discretion. For 
example, the award fee plan's fiscal year 2007 template had a range of 
10 percent to 20 percent for weighting management performance, 20 
percent to 30 percent for safety and surety, 20 percent to 25 percent 
for environmental compliance, 10 percent to 15 percent for cost 
performance, and 10 percent to 20 percent for schedule performance. The 
Army adjusted these ranges in fiscal year 2006, adding the schedule 
performance measure, increasing the template's range for management, 
and reducing the template's ranges for environmental compliance and 
safety and surety performance measures. 

Table 12 shows historical data on the award fee plans' performance 
measures and the relative weights selected by each of the incineration 
sites and Newport as well as the adjustments made from fiscal year 2003 
to fiscal year 2007. 

Table 12: Performance Measures and Weights in Award Fee Plans, Fiscal 
Years 2003-2007: 

[See PDF for image] 

Source: U.S. Army Sustainment Command. 

Notes: Data on the award fee plans included fiscal year 2003 through 
fiscal year 2007 to represent the period of time the sites were in or 
preparing for the operations phase. In addition, we evaluated the 
performance measures and related weights for those fiscal years since 
the previous APB was established in April 2003. N/A indicates that no 
weight was assigned to the particular award fee plan performance 
measure at that site for that particular fiscal year. 

[A] Site project managers can select weights outside of suggested 
ranges with CMA approval. While the award fee plan's fiscal year 2007 
template had a range of 20 percent to 30 percent for weighting safety 
and surety, Tooele's weight for safety and surety for fiscal year 2007 
was 35 percent. 

[End of table] 

As shown in table 12, government site project managers have weighted 
safety and surety and environmental compliance from a weight factor 
that ranged from 30 percent to 40 percent in fiscal year 2003 to a 
weight factor from 20 percent to 35 percent in fiscal year 2007. Also, 
the award fee plans for the incineration sites and Newport have safety 
"killer clause" provisions, whereby a systems contractor could lose all 
or part of its award fee if there is an agent release, an individual or 
worker is exposed to chemical agent, or there is a release of agent 
outside of the engineering controls. 

In fiscal year 2006, CMA added the schedule performance measure, 
increased the template's range for management, and reduced the 
template's range for environmental compliance and safety and surety. We 
found, based on our analysis, that the site project managers 
consistently placed schedule and cost performance at lower relative 
weights than other factors, including management performance. For 
example, in fiscal years 2006 and 2007, the maximum weight allowable by 
CMA for schedule performance was 20 percent, but the average weight 
assigned by the site project managers in the award fee plans was only 
13.75 percent for the incineration sites. Conversely, the site project 
managers weighted other performance measures, such as management, on 
average much closer to CMA's maximum suggested weights. 

Program officials stated that the weights assigned to the different 
performance measures are based on the relative significance of the 
criteria to the program. Due to the lethal nature of the material that 
must be processed, Congress has mandated that the program ensure 
maximum protection to the workforce, the public, and the environment. 
Accordingly, as program officials point out, safety/surety carry the 
highest weighting followed closely by environmental compliance. 
However, given the maturity of the operations phase, there may be 
opportunities, from a management perspective, for the Army to further 
adjust weights for performance measures in the award fee plan template 
to further incentivize key acquisition outcomes, such as schedule and 
cost, at the incineration sites. The five incineration sites have been 
preparing for or were in operation since at least 2003 and, along with 
Newport and Aberdeen, have eliminated over 45 percent of the chemical 
munitions stockpile. In addition, the Army has stated that it has 
reduced much of its safety risk by destroying the bulk of its most 
lethal and volatile chemical munitions. For example, the Tooele site 
has mostly ton containers with mustard agent left to destroy, which is 
considered less risky than other agents. Moreover, Army officials have 
pointed out that the systems contractors have implemented safety 
improvement plans that have incorporated appropriate protocols and 
standards into their daily operations. As a result, from a management 
perspective, there may be opportunities to further emphasize schedule 
and cost without sacrificing safety and environmental compliance. 

It Is Possible to Make Greater Use of Objective Award Fee Criteria and 
Evaluation Factors: 

Based on our review of the fiscal year 2007 award fee plan's 
performance measures and related evaluation criteria for the sites, 
most of the award fee criteria and evaluation factors are not objective 
and measurable. We determined that it is possible to make CMA's award 
fee plans more effective in achieving certain acquisition outcomes 
through greater use of objective award fee criteria. We previously 
recommended that DOD instruct the military services to move toward more 
outcome-based award fee criteria that are achievable and promote 
accountability for and are directly linked to achieving desired program 
outcomes.[Footnote 62] In March 2006, DOD issued award fee guidance 
stating that it is imperative that award fees be tied to identifiable 
interim outcomes, discrete events, or milestones as much as 
possible.[Footnote 63] We acknowledge that award fee contracts are 
expected to use subjective criteria to the extent that objective 
criteria are not feasible. However, we identified additional 
opportunities, from a management perspective, for CMA to make greater 
use of objective award fee criteria and evaluation factors to 
incentivize contractors to achieve desired acquisition outcomes, such 
as safely destroying the chemical stockpile on schedule and within 
cost. 

The award fee plans for this program make use of both subjective and 
objective criteria and our analysis of the fiscal year 2007 award fee 
plan showed that in most cases, the criteria are subjective. Three 
performance measures--safety, environmental compliance, and schedule-- 
use a mix of objective and subjective performance evaluation criteria, 
although we noted these evaluations can also factor in subjective 
assessments. The award fee plan states that in evaluating performance 
in these areas, a number of subjective attributes will be considered, 
but there are no specific weights associated with the attributes. 
Rather, they are considered guides to evaluate the broad spectrum of 
potential systems contractor performance. The schedule performance 
measure was, for the most part, objective and linked to an outcome--the 
destruction of chemical munitions. For example, the 2007 schedule 
performance category used site-specific destruction tonnage goals 
related to meeting CWC deadlines and the APB schedule. On the other 
hand, three other performance measures--cost, management, and surety-- 
are evaluated using all subjective criteria. While we recognize that 
using subjective criteria to assess systems contractor performance is 
acceptable, we identified opportunities for the Army to use more 
objective award fee criteria when evaluating contractor performance. 

We developed some illustrative examples to show how certain elements of 
CMA's subjective criteria taken from the 2007 award fee template could 
be revised to objectively measure systems contractor performance. We 
recognize that there are other examples that could be used to 
illustrate this point and do not suggest that these examples 
necessarily be used. To develop these examples, we used as criteria our 
analysis and prior work.[Footnote 64] For example, the current award 
fee plan contains the following subjective criterion for evaluating 
management performance: "Demonstrated ability to practice proactive 
management to identify and anticipate problems and implement effective 
countermeasures prior to adverse impact." Illustrative examples of 
potential objective criteria that could address these criteria are 
"Develop and implement an integrated risk management plan by [a 
specified date]" and "Ensure that [a specified percentage] of the risks 
identified have a countermeasure available and implemented within [a 
specified number of days] of the mustard container campaign." 

Similarly, the current award fee plan contains the following subjective 
criterion for evaluating cost performance: "Cost management systems 
(accounting, scheduling, planning, and budgeting) are all efficiently 
and effectively integrated, and produce consistent, reliable, and 
accurate data." An illustrative example of a potential objective 
criterion that could address at least a portion of these criteria is 
"Ensure that [a specified percentage] of the cost and schedule 
management systems data accurately depicts contract cost status within 
the assessment period." Moreover, the current award fee plan contains 
the following subjective criterion for evaluating surety performance: 
"Appropriate treatment and disposal of chemical agents and chemical 
agent standards." An illustrative example of a potential objective 
criterion that could address at least a portion of these criteria is 
"Ensure that [a specified percentage] of chemical agents are 
appropriately treated and disposed of under the chemical agent 
standards." 

After the March 2006 guidance on award fees was issued by the Under 
Secretary of Defense for Acquisition, Technology, and Logistics, CMA 
issued a policy on award fees in September 2006. CMA's policy stated 
that "due to the complexity of the program and the toxic nature of the 
material being processed, essential elements of performance cannot 
effectively or feasibly be measured by predetermined objective metrics. 
Therefore, award fee criteria are established that allow assessment of 
performance through the use of objective and/or subjective metrics." 
Our review of this policy indicated that CMA did not clearly or fully 
incorporate the concept of tying "award fees to identifiable interim 
outcomes, discrete events or milestones as much as possible" as called 
for in the March 2006 guidance. Although not legally required, we 
believe that tying award fees as much as possible to more objective, 
measurable evaluation criteria is a sound business and management 
practice, and that CMA could do more to fully incorporate and implement 
the March 2006 DOD policy guidance. 

Recently Established and Planned Performance-Based Incentive Fees Are 
Focused on Schedule, but Their Effectiveness Is Uncertain at This Time: 

The Army has recently offered monetary fees intended to provide 
additional incentives to improve schedule performance. In 2005, the 
Army added to its fee structure the DPPBI, which is an annual 
performance-based incentive fee that is structured to encourage the 
systems contractors at the four incineration sites to work 
collaboratively to improve various aspects of their performance. That 
is, while each individual site is evaluated and scored separately, the 
individual scores are averaged to determine a composite score that 
results in all sites earning the same percentage of the available DPPBI 
fee pool. Since the initial evaluation period in 2005, the fee amount 
available for the DPPBI has been 2 percent of the estimated fee-bearing 
contract costs--relatively small when compared to the award fee 
available to the incineration sites. The 2 percent DPPBI is in addition 
to the 10 percent available through base and award fees. 

The evaluation factors for assessing systems contractor performance for 
the DPPBI have changed over time and are divided between objective 
criteria and a subjective assessment of systems contractor performance 
during the fiscal year. Three criteria were established for the initial 
DPPBI evaluation period of January 1 through September 30, 2005: risk 
mitigation, production assessment, and lessons learned. In the initial 
evaluation period, the contractors collectively were deemed to have 
earned 51 percent of the 2005 DPPBI pool or $3,313,954 out of 
$6,497,949. In fiscal year 2006, the DPPBI criteria were changed to put 
more emphasis on performance outputs and less on processes and 
procedures. The fiscal year 2006 DPPBI criteria were established to 
encourage the contractors to collectively reduce project schedules 
while maintaining a high standard of surety, safety, and environmental 
compliance. The criteria also were established to increase CMA's 
confidence levels that the sites could achieve CWC destruction 
deadlines, although discrete goals were not specified. For the fiscal 
year 2006 period of performance, the contractors earned about 93 
percent or $8,771,200 out of the $9,411,159 in the DPPBI pool for that 
period. 

Prior to fiscal year 2007, the DPPBI was not linked to objective 
schedule performance criteria linked to the April 29, 2012, CWC treaty 
deadline. Beginning in fiscal year 2007, the DPPBI fee evaluation was 
linked to achieving discrete schedule milestones associated with 
achieving the CWC April 29, 2012, deadline. For fiscal year 2007, 70 
percent of the total DPPBI fee available to be earned (or up to a 
maximum of 1.4 percent of the total negotiated contract costs, 
depending on average collective performance) was based on specified 
schedule attainment targets for each site. The remaining 30 percent of 
the available DPPBI fee for fiscal year 2007 is based on subjective 
criteria related to the contractors' performance in the application of 
lessons learned, risk management, programmatic procurement, and other 
collaborative approaches to reduce costs and accelerate destruction of 
the stockpile. This fee would be in addition to the portion of the 
award fee targeted at schedule performance, now about 1 percent of 
total negotiated contract costs at most incineration sites. This 
additional emphasis on schedule is a positive development, but it 
remains unclear to us how effective this incentive fee will be in 
motivating individual systems contractors' schedule performance due to 
the collective manner in which the fee is earned, its relative size, 
and recent focus on objective schedule performance. It is particularly 
unclear to us how individual contractors might be motivated during 
periods when the underperformance of one or more systems contractors 
could greatly reduce the potential to earn the fees by the others. 

Finally, the Army plans to use a recently authorized incentive fee of 
up to $165 million per site in an effort to further motivate the 
systems contractors to accelerate destruction and closure activities. 
Although it had not been placed on contract at the time of our review, 
CMA drafted a plan to offer incentive fees under the authority 
contained in section 923 of the National Defense Authorization Act for 
Fiscal Year 2007.[Footnote 65],[Footnote 66] This plan would allow 
systems contractors at the four incineration sites the opportunity to 
earn fees in addition to the 12 percent available to be earned 
annually. As currently planned, systems contractors will be able to 
earn incentive payments for completion of destruction operations tied 
to dates around the final CWC deadline and for the acceleration of 
facility closure activities, but the effectiveness of this approach is 
uncertain. Under CMA's plan, it would offer the additional incentive 
fee of up to $150 million to each site--up to $100 million for 
destruction operations and up to $50 million for facility closure 
activities completed within specified target ranges. Bonuses totaling 
$15 million would also be available to each site if all sites complete 
destruction operations before the treaty deadline and accelerate 
facility closure activities. 

CMA's plan, which would have to be implemented under each of the 
contracts, has a number of positive features. For example, the plan 
recognizes, as does the statute, that the enterprise of striving to 
capture these incentives is to be conducted consistent with the 
existing obligation to provide for maximum protection for the 
environment, the general public, and the personnel who are involved in 
the destruction of the chemical agents and munitions. The potential 
effectiveness of this particular planned incentive at motivating the 
systems contractors to complete destruction operations around the 
treaty deadline is also uncertain. This incentive arrangement has not 
yet been placed on contract and thus it is too early to assess its 
potential success. Additional details about the two recently 
established incentive fees are in appendix II. 

Conclusions: 

Destroying the nation's remaining stockpile of chemical weapons in a 
safe, efficient, and timely manner is essential to improve the 
potential for the United States to meet CWC obligations and to reduce 
the risk to the public and the environment of a potential catastrophic 
event associated with lethal chemical stockpiles. We recognize that 
maintaining high levels of safety and environmental compliance are 
inextricably linked to success in meeting schedule and cost goals. We 
also recognize that it is highly important that program management, 
schedule and cost estimates, and monetary incentives be soundly 
structured, well aligned with program goals and desired acquisition 
outcomes, reasonable, and reliable. While DOD has taken specific 
actions to strengthen program management, improving the focus of annual 
performance plans beyond a single year and refining and finalizing 
planned actions in the risk management area could greatly enhance the 
usefulness of these management tools for DOD, CMA, the sites, and 
Congress. In addition, without a fully developed and implemented risk 
management approach, CMA cannot ensure that it is fully anticipating 
and mitigating risks going forward. Without such improvements, it would 
be difficult for program officials to accurately track progress, 
effectively initiate timely actions to mitigate risks, and increase the 
potential for program activities to further contribute to the overall 
program goal of safely destroying the chemical weapons stockpile in a 
safe and environmentally compliant manner. 

Similarly, it is important that DOD, CMA, and Congress have reasonable 
schedule and cost estimates with reliable supporting data and other 
assumptions, including when they change significantly, either 
increasing or decreasing funding requirements. With most of the 
chemical agent destruction sites significantly ahead of the 2005 
program office schedule, the program milestones appear to be overly 
conservative, regardless of unexpected risks, such as the rocket fires 
at Umatilla. As a result, the program schedule milestones may no longer 
be a realistic baseline for measuring program performance and warrant 
further review and independent assessment to give DOD and Congress 
greater confidence that the program is being held accountable to a 
reasonable baseline to measure the program's progress. In addition, the 
Chemical Demilitarization Program's revised APBs and its most recent 
SARs--key management and oversight documents--lack up-to-date and 
reviewed schedule milestone information. Furthermore, DOD needs 
accurate and realistic closure cost estimates and other cost inputs for 
each of DOD's chemical demilitarization sites to accurately and 
adequately budget for the program, make the EVM systems useful for 
program oversight, and arrive at reliable cost estimates. 

Finally, from a management perspective, CMA could improve its use of 
award fees to incentivize the systems contractors to try to meet the 
CWC treaty deadline in a safe and environmentally compliant manner, 
particularly at the incineration sites where award fees are 
predominant. We identified opportunities for CMA to continue and 
further advance its recent efforts to emphasize schedule and cost and 
to incorporate more objective measures in its award fee evaluation 
criteria as called for in DOD guidance on award fees. Given the 
maturity of the operations phase, the experience gained by the systems 
contractors, and the composition of the remaining stockpile, CMA may be 
able to adjust the award fee plan to provide better interim incentives 
to accelerate the systems contractors' performance, and do so in a safe 
and environmentally sound manner. Doing so would also have the benefit 
of reducing the security vulnerabilities and other risks of storing 
lethal chemical agents any longer than necessary, which creates 
additional safety and environmental hazards. We recognize that to the 
extent that objective criteria are not feasible, using subjective 
criteria to determine award fees is reasonable and expected, and DOD 
guidance directs that award fee criteria be tied to performance 
measures, such as identifiable interim outcomes, discrete events, or 
milestones, as much as possible. We identified opportunities for CMA to 
improve the effectiveness of its monetary incentives by better aligning 
its award fee policy and evaluation criteria to DOD's March 2006 
guidance. 

Recommendations for Executive Action: 

We are making 13 recommendations for executive action. They are as 
follows. 

To strengthen program management and increase the likelihood that more 
progress can be made in destroying chemical agents, we recommend that 
the Secretary of Defense direct the Secretary of the Army to take the 
following three actions: 

* incorporate baseline and trend data for past performance and 
multiyear performance goals for the future in its annual performance 
plan; 

* develop interim destruction goals, approaches, and milestones that 
are directly linked to overall program goals for meeting CWC deadlines; 
and: 

* establish a time frame for completing and implementing its risk 
management approach, including integration across sites and with DOD. 

To provide more realistic cost estimates for chemical demilitarization 
sites in light of recent experience and information gained, we 
recommend that the Secretary of Defense direct the Secretary of the 
Army to update the 2005 program schedule milestones for each of DOD's 
chemical demilitarization sites to reflect recent site processing rates 
and then adjust the cost estimate accordingly. 

To help ensure that the program baseline schedule is current and 
achievable and to improve the accuracy and reliability of program 
manager estimates for measuring the Chemical Demilitarization Program's 
progress, we recommend that the Secretary of Defense direct the CAIG to 
establish a periodic schedule to review the program's processing 
history to determine whether the schedule milestones are still 
reasonable. 

To increase the ability of decision makers to measure the Army's and 
CMA's performance against schedule growth, we recommend that Secretary 
of Defense direct the Under Secretary of Defense for Acquisition, 
Technology, and Logistics to include in the Chemical Demilitarization 
Program's APB and SAR the updated and reviewed schedule milestones for 
completing destruction operations and facility closure activities for 
each of the chemical agent destruction sites. 

To provide more realistic cost estimates for closure of chemical 
demilitarization sites and to establish, stabilize, and maintain an 
accurate, valid, and current performance management baseline, we 
recommend that the Secretary of Defense direct the Secretary of the 
Army to take the following three actions: 

* coordinate with the systems contractors in defining closure 
requirements and developing accurate and realistic closure cost 
estimates for each of DOD's chemical demilitarization sites' contract 
performance measurement baseline; 

* define all contract requirements, including authorized unpriced work 
associated with facility closure activities, and develop comprehensive 
cost estimates through closure at each chemical destruction site; and: 

* establish a time frame for completing the negotiation for placing all 
authorized work that has not been priced on contract and ensuring that 
the negotiations take place. 

After the cost estimates are complete and schedule milestones have been 
updated and reviewed, we recommend that the Secretary of Defense direct 
the CAIG to conduct an independent cost review of the cost estimates. 

To continue to further the Army's efforts to improve the potential to 
meet the CWC deadline for destroying chemical weapons safely and within 
environmental compliance and reduce the risk of storing lethal chemical 
agents any longer than necessary, we recommend that the Secretary of 
the Defense direct the Secretary of the Army to determine for each of 
the four incineration site contracts whether greater weight can be 
applied to performance measures, such as schedule and cost, in award 
fee plans to further incentivize the systems contractors to increase 
the destruction rate of chemical agents in a safe and environmentally 
sound manner. 

To further link monetary incentives and key acquisition outcomes, and 
to provide a means for objectively measuring the systems contractors' 
performance during the assessment period as much as possible, we 
recommend that the Secretary of Defense direct the Secretary of the 
Army to take the following two actions: 

* better align its award fee policy with DOD's March 2006 guidance on 
award fees and: 

* link more of its award fee criteria to performance measures that 
focus on identifiable interim outcomes, discrete events, or milestones. 

Agency Comments and Our Evaluation: 

In written comments on a draft of this report, DOD concurred or 
partially concurred with 12 of our 13 recommendations. DOD did not 
concur with our recommendation regarding defining closure requirements 
and developing accurate and realistic closure cost estimates for each 
of DOD's chemical demilitarization sites. DOD recommended that we 
delete it, since DOD officials believe that a subsequent recommendation 
defining all contract requirements, including those associated with 
facility closure activities, was more encompassing. Based on DOD's 
comments, rather than deleting the recommendation, we clarified it and 
the related recommendation to more clearly distinguish the difference 
between the two recommendations, and consolidated it with our two other 
recommendations related to defining contract requirements. DOD also 
stated that the department will continue to use acquisition management 
tools and discipline to ensure the destruction of the United States' 
chemical weapons stockpile in a safe and secure manner, while being 
economical and meeting the U.S. commitments under the Chemical Weapons 
Convention. DOD identified a number of actions that it already has 
initiated in response to our recommendations. DOD also provided us with 
technical comments, which we incorporated in the report, as 
appropriate. DOD's comments are reprinted in appendix III. 

Regarding our four recommendations related to strategic planning tools, 
DOD concurred with our recommendation that it incorporate baseline and 
trend data for past performance and multiyear performance goals for the 
future in its annual performance plan. DOD stated that CMA currently is 
incorporating these items into its fiscal year 2008 Annual Performance 
Plan and expects this effort to be completed by December 31, 2007. DOD 
partially concurred with our recommendation that it provide direction 
to develop interim destruction goals, approaches, and milestones that 
are directly linked to overall program goals for meeting CWC deadlines. 
In the spirit of our recommendation, DOD stated that it would ensure 
that all future CMA performance plans incorporate annual goals and 
approaches leading toward the April 29, 2012, destruction deadline 
through direction issued annually in support of its existing strategic 
governance process and that additional direction would not be needed. 
We support this planned action, which, if implemented properly, would 
accomplish the same effect as our recommendation. DOD concurred with 
our recommendation that the Army establish a time frame for completing 
and implementing its risk management approach, including integration 
across sites and with DOD. DOD stated that it expects an approved risk 
management approach to be completed by June 30, 2008. DOD concurred 
with our recommendation to include in the Chemical Demilitarization 
Program's APB and SAR the updated and reviewed schedule milestones for 
completing destruction operations and facility closure activities for 
each of the chemical agent destruction sites. DOD stated that it would 
request administrative changes to remove placeholder terminology from 
these documents and replace them with the CAIG's schedule estimates. 

Regarding our two recommendations related to schedule and cost 
estimates, DOD partially concurred with our recommendation that it 
update the 2005 program schedule milestones for each of DOD's chemical 
demilitarization sites to reflect recent site processing rates and then 
adjust the cost estimate accordingly. DOD stated that it has gained 
only limited information during the past year regarding processing 
rates, unknown risks, and the potential for a chemical event, and thus 
does not believe it would be prudent to adjust cost and schedule 
estimates or rebaseline at this time. However, DOD will evaluate the 
cost and schedule estimates for the entire program during the upcoming 
Fiscal Year 2010-2015 Program Objectives Memorandum build process and 
program review, and will determine whether an update is warranted. DOD 
concurred with our recommendation to establish a periodic schedule for 
the CAIG to review the program's processing history to determine 
whether the schedule milestones are still reasonable. DOD stated that 
it conducts a biannual program review of the Chemical Demilitarization 
Program and that during the next review in the spring of 2008, it would 
evaluate the program's cost and schedule estimates. If significant 
differences exist between the program's baseline and the revised 
estimates, then DOD would request a CAIG review that could result in 
new cost and schedule estimates. However, we believe that our report 
provides clear and sufficient evidence demonstrating that there are 
differences between the program's current schedule and cost estimates 
and DOD's baseline to warrant a CAIG review and update as we 
recommended. 

Regarding our four recommendations related to contract requirements, 
based on DOD's comments, we consolidated and clarified two of our 
recommendations. DOD suggested that we delete the recommendation that 
CMA coordinate with the systems contractors in defining closure 
requirements and in developing accurate and realistic closure cost 
estimates for each of DOD's chemical demilitarization sites, since the 
subsequent recommendation is more encompassing. Instead, we clarified 
this recommendation by adding "for each of DOD's chemical 
demilitarization sites' contract performance measurement baseline" 
[emphasis added], and consolidated it with our two other 
recommendations related to defining contract requirements discussed 
below. As discussed in our report, we continue to believe that CMA and 
the systems contractors must coordinate their efforts to develop 
accurate and realistic closure cost estimates. In addition, we 
consolidated our recommendation that after the cost estimates are 
complete and schedule milestones have been updated and reviewed, the 
CAIG conduct an independent cost review of the cost estimates with our 
two recommendations that are discussed below. DOD partially concurred 
with our recommendation to define all contract requirements, including 
those associated with facility closure activities, and develop 
comprehensive cost estimates through closure at each chemical 
destruction site. Based on DOD's comments, we clarified this 
recommendation by adding "define all contract requirements, including 
authorized unpriced work associated with facility closure activities" 
[emphasis added]. DOD stated that since June 2007, three of the five 
CMA destruction sites have completed contract performance baselines 
that include contract requirements for operations, secondary waste 
processing, closure, and known risk. Negotiations are under way to 
complete the contract performance baseline for the Tooele, Utah, site 
and the contract requirements and cost estimates for the Newport, 
Indiana, site by June 30, 2008. DOD concurred with our recommendation 
to establish a time frame for completing the negotiation for placing 
all authorized work that has not been priced on contract and ensuring 
that the negotiations take place. DOD stated that it expects 
negotiations to be completed by June 30, 2008. Finally, DOD partially 
concurred with our recommendation that the CAIG conduct an independent 
review of the program's cost estimates stating that the program's 
biannual review in support of the budget submission will determine if a 
CAIG independent assessment is warranted. We continue to believe that 
providing an independent review of the program's cost estimates could 
provide greater assurance that cost estimates are reasonable. 

With regard to our three recommendations related to incentive fees, DOD 
concurred with our recommendation to determine for each of the four 
incineration site contracts whether greater weight can be applied to 
performance measures, such as schedule and cost, in award fee plans to 
further incentivize the systems contractors to increase the destruction 
rate of chemical agents in a safe and environmentally sound manner. DOD 
also concurred with our recommendation to better align CMA's award fee 
policy with DOD's March 2006 guidance on award fees, stating that this 
action had been completed on September 7, 2007. However, when a copy of 
the revised policy was requested, DOD stated that its response 
incorrectly referenced that this action has been completed. As such, 
DOD will direct a revision of the CMA policy that aligns it with the 
DOD guidance on award fees, and the action is to be completed by March 
31, 2008. DOD partially concurred with our recommendation to link more 
of its award fee criteria to performance measures that focus on 
identifiable interim outcomes, discrete events, or milestones. DOD 
stated that it would conduct a review of CMA award fee plans, to be 
completed by March 31, 2008, and would then determine whether further 
direction and action is required. While this review represents a 
positive step, our report shows that there are several areas in which 
DOD could better link more of its award fee criteria with performance 
measures that focus on identifiable outcomes and we believe that DOD 
should fully implement this recommendation. 

As agreed with your offices, unless you publicly announce the contents 
of this report earlier, we plan no further distribution of it until 30 
days from the date of this letter. We will then send copies of this 
report to the Chairmen and Ranking Members of the Senate Committee on 
Armed Services; Senate Committee on Homeland Defense and Governmental 
Affairs; Subcommittee on Defense, Senate Committee on Appropriations; 
House Committee on Oversight and Government Reform; and Subcommittee on 
Defense, House Committee on Appropriations, and other interested 
congressional parties. We are also sending copies of this report to the 
Secretary of Defense; the Secretary of the Army; the Director, Office 
of Management and Budget; and the Director, U.S. Army Chemical 
Materials Agency. We also will make copies available to others on 
request. In addition, this report will be available at no charge on the 
GAO Web site at [hyperlink, http://www.gao.gov]. 

If you or your staff have any questions concerning this report, please 
contact me at (202) 512-5431 or [email protected]. Contact points for 
our Offices of Congressional Relations and Public Affairs may be found 
on the last page of this report. GAO staff who made key contributions 
to this report are listed in appendix IV. 

Signed by: 

Davi M. D'Agostino: 

Director, Defense Capabilities and Management: 

[End of section] 

Appendix I: Scope and Methodology: 

To assess the progress the Department of Defense (DOD) and the Army 
have made in implementing our prior recommendations to strengthen 
program management, whether it has been sufficient, and whether actions 
taken to identify and mitigate the risk of future program schedule 
extensions and cost growth have been effective, we obtained and 
reviewed strategic and implementation plans and interviewed cognizant 
officials to identify whether key elements were present, such as a 
program mission statement, long-term goals and objectives, delineation 
of roles and responsibilities of DOD and Army offices, and near-term 
performance measures. 

To assess the reasonableness of the methodology used to determine 
schedule milestones as well as the reliability of cost estimates, we 
reviewed current program estimates, destruction schedules, earned value 
management (EVM) data, and other documents. We also obtained and 
reviewed the program's risk management plans and related documents. In 
addition, we also identified the issues that had caused delays and 
ascertained approaches being used to reduce the potential for delays in 
the future. 

The Army's Chemical Materials Agency (CMA) provided us with EVM reports 
submitted to the Army by the systems contractors for each of the 
chemical demilitarization facilities spanning the period from December 
2000 through February 2007. EVM reports were provided electronically in 
the form of Deltek wInsightTM transfer files for Cost Performance 
Report Formats 1 through 4 and separate Adobeï¿½, MS Word, or MS Excel 
files for Format 5. 

Our approach in assessing the adequacy of the systems contractors' EVM 
reporting was to analyze the data provided according to EVM analysis 
tasks and DOD earned value implementation guidance. The GAO EVM 
analysis tasks are a set of 10 analysis tasks developed by GAO that are 
used to assess the adequacy of EVM. These 10 analysis tasks were 
performed on each of the EVM reports provided to us. Guidance and 
procedures outlined within the DOD Earned Value Management 
Implementation Guide were also referenced to assess EVM system and 
reporting adequacy. 

Our approach in assessing the Army's ability to assess, synthesize, and 
incorporate systems contractor historical and estimated cost, 
technical, and schedule performance into the current program estimate 
was to align historical contract performance data as documented within 
the EVM reports with program estimate and baseline data documented 
within Selected Acquisition Reports, the President's budget, and the 
Fiscal Year 2005 Army Cost Position. 

In performing this assessment, we also conducted interviews with 
government and systems contractor officials at CMA headquarters in 
Aberdeen, Maryland; Tooele Chemical Agent Disposal Facility, Deseret 
Chemical Depot (Tooele), Utah; and Umatilla Chemical Agent Disposal 
Facility, Umatilla Chemical Depot (Umatilla), Oregon. 

To analyze the effectiveness of DOD's and the Army's use of monetary 
incentives to improve systems contractors' performance, we obtained and 
reviewed contract documentation related to the implementation of the 
contracts' incentives, including the basic contract and modifications, 
award fee plans, award fee determining official decisions, award fee 
evaluation board reports, and pre-and postnegotiation memorandums, and 
the performance-based incentive fees, including the Director's 
Programmatic Performance Based Incentive (DPPBI) fee. We also 
interviewed DOD and Army officials to determine the types of award and 
incentive fees offered at the incineration sites and at Newport 
Chemical Agent Disposal Facility, Newport Chemical Depot (Newport), 
Indiana. 

To evaluate the effectiveness of the award fees in controlling costs 
and schedule, we reviewed the award fee plans from fiscal year 2003 to 
fiscal year 2007 to identify the types of performance measures and 
weight factors used to incentivize the systems contractors to safely 
destroy the chemical weapons stockpile. We also looked at the extent 
the criteria used for the performance measures were linked to 
measurable outcomes. To identify the percentage of award fees paid to 
the systems contractors at the incineration sites and Newport, we 
collected and reviewed data on the award fee payments, including the 
amount available for the contactors to earn, the amounts earned from 
fiscal year 2003 through fiscal year 2006, and the guidance for 
improving the use of these fees. We used data on the award fee plans 
from fiscal year 2003 through fiscal year 2007 because they represented 
the period of time the sites were in the operations phase. 

To evaluate the extent documentation was prepared for adjustments to 
the award fee plans, we reviewed CMA's Integrated Process Team 
documents from fiscal year 2003 to fiscal year 2007 to identify 
adjustments made by the various stakeholders, such as the CMA Director, 
U.S. Army Sustainment Command, CMA Program and Policy Office, 
incinerator site project managers, and systems contractor site general 
managers. We evaluated whether Army officials provided complete 
documentation supporting their rationale for adjusting performance 
measures and related weights throughout the process. We also evaluated 
whether CMA prepared adequate supporting documentation to enable 
reviewers to understand and evaluate the process that was used, and 
whether it contained, among other things, an adequate discussion of 
alternative performance measures or the combination of award and 
incentive fees that it considered, tested, and rejected and the reasons 
for rejecting them. 

To evaluate the effectiveness of the authority contained in section 923 
of the National Defense Authorization Act for Fiscal Year 2007, we 
obtained and reviewed the incentive agreement plan and incentive fees 
for the Aberdeen Chemical Agent Disposal Facility (Aberdeen), Maryland, 
and Newport to determine the challenges those sites faced in 
negotiating the incentive fee agreements. Also, we interviewed 
officials with the U.S. Army Sustainment Command, CMA, and CMA and 
systems contractors at Tooele and Umatilla. 

We also met with DOD and Army program officials and interviewed 
officials at two of the chemical agent destruction sites. Finally, this 
report focuses primarily on the seven CMA-managed chemical destruction 
sites--Aberdeen; Anniston Chemical Agent Disposal Facility, Anniston 
Chemical Activity, Alabama; Johnston Atoll in the Pacific Ocean; 
Newport; Pine Bluff Chemical Agent Disposal Facility, Pine Bluff 
Chemical Activity, Arkansas; Tooele; and Umatilla--because most are in 
the operational phase and represent about 82 percent of the remaining 
stockpile of chemical agents to be destroyed, and have reached a level 
of maturity in processing a variety of types of munitions and agents 
that will help meet Chemical Weapons Convention (CWC) goals. This 
report does not focus on two sites--Aberdeen and Johnston Atoll--that 
have already completed operations and have either been closed or are 
being closed. It also does not focus on the two chemical agent 
destruction sites that are part of the Assembled Chemical Weapons 
Alternatives program--Blue Grass Chemical Agent-Destruction Pilot 
Plant, Blue Grass Chemical Activity (Blue Grass), Kentucky, and Pueblo 
Chemical Agent-Destruction Pilot Plant, Pueblo Chemical Depot, 
Colorado--that currently are in the design and construction phase to 
destroy chemical agents, report directly to DOD rather than to the 
Army, and are not expected to help meet CWC goals. 

We examined the reliability of the data used in this report by 
verifying EVM system certification, applying analytical tests to the 
EVM data, and examining EVM system effectiveness in providing 
meaningful performance measurement to program management. We found that 
EVM systems at each site had been certified. The results of our 
analytical tests found the data to be reasonably free of data errors 
and anomalies. However, when we examined the effectiveness of the EVM 
system to provide meaningful performance measurement, we found 
insufficient reliability to address our objectives. 

We conducted this performance audit from May 2006 through July 2007 in 
accordance with generally accepted government auditing standards. 

[End of section] 

Appendix II: Two Recently Introduced Performance-Based Incentive Fees: 

The Army's performance-based incentive fee to collaboratively improve 
schedule and cost performance was established in 2005 because the use 
of award fees had not been successful. Also, the Army plans to use a 
recently authorized $165 million incentive fee to accelerate 
destruction and closure activities to dates linked with the treaty 
schedule. 

Army's Performance-Based Incentive Fee Begun in 2005 Was Recently 
Targeted to Objective Performance: 

In 2005, CMA established the DPPBI fee to improve schedule and cost 
performance because CMA's use of award fees had not been successful in 
controlling cost and schedule at the incineration sites. This incentive 
fee is 2 percent of the total 12 percent available fees, which includes 
the 3 percent base fee that is not performance related. DPPBI was 
structured so that the collective performance of the systems 
contractors is evaluated and rewarded through improved collaboration 
among the systems contractors that operate the four incineration sites. 
That is, while each individual site is evaluated and scored separately, 
the individual scores are averaged to determine a composite score that 
results in all sites earning the same percentage of the available DPPBI 
fee pool. 

The evaluation factors for assessing the systems contractor performance 
for DPPBI have changed over time and are divided between objective 
criteria and a subjective assessment of systems contractor performance 
during the fiscal year. Three criteria were established for the initial 
DPPBI evaluation period of January 1 through September 30, 2005: risk 
mitigation, production assessment, and lessons learned. In the initial 
evaluation period, the contractors collectively were deemed to have 
earned 51 percent of the 2005 DPPBI pool or $3,313,954 out of 
$6,497,949. In fiscal year 2006, the DPPBI criteria were changed to put 
more emphasis on performance outputs and less on processes and 
procedures. The fiscal year 2006 DPPBI criteria were established to 
encourage the contractors to reduce project schedules while maintaining 
a high standard of surety, safety, and environmental compliance. The 
criteria also were established to increase CMA's confidence levels that 
the sites could achieve CWC destruction deadlines, although discrete 
goals were not specified. For the fiscal year 2006 period of 
performance, the contractors earned about 93 percent or $8,771,200 out 
of the $9,411,159 in the DPPBI pool for that period. 

Until fiscal year 2007, the DPPBI fee evaluation was not linked to 
objective schedule performance criteria linked to the April 29, 2012, 
CWC extended treaty deadline. In fiscal year 2007, schedule performance 
attainment was added to the DPPBI objective criteria amounting to 70 
percent of the total available DPPBI fee (or up to 1.4 percent for 
average collective performance) and amounting to 30 percent for 
subjective criteria related to the contractors' performance in the 
application of lessons learned, risk management, programmatic 
procurement, and other collaborative approaches to reduce cost and 
accelerate destruction of the U.S. stockpile. The amount of the fee 
that can be earned by the systems contractors depends not just on a 
systems contractor's individual performance, but also on the 
performance of the other systems contractors. For example, if the 
systems contractor at Anniston is performing well in meeting its 
schedule-related goals during a specific period, but the systems 
contractor at Tooele is not, then the Anniston systems contractor would 
be encouraged to offer assistance to the Tooele systems contractor. If 
the Tooele systems contractor's schedule performance could not be 
improved, then the Anniston systems contractor's potential to earn this 
fee would be reduced as would the potential of systems contractors at 
the other sites. 

Recently Authorized Incentives Are Targeted to Completing Destruction 
Operations around the Treaty Deadline and Accelerating Facility Closure 
Activities: 

CMA plans to offer incentive fees under the authority contained in 
section 923 of the National Defense Authorization Act for Fiscal Year 
2007,[Footnote 67] permitting systems contractors the opportunity to 
earn incentive payments for the completion of destruction operations by 
or near the CWC deadline and the acceleration of facility closure 
activities, but the effectiveness of this approach is uncertain. The 
purpose of section 923 is to provide the systems contractor for a 
chemical demilitarization facility an incentive to accelerate the safe 
elimination of the U.S. chemical weapons stockpile and to reduce the 
total cost of the Chemical Demilitarization Program by providing 
incentive payments for the early completion of destruction operations 
and the closure of such a facility.[Footnote 68] Section 923 authorizes 
the inclusion of incentives clauses in chemical demilitarization 
contracts for the completion of both destruction operations and 
facility closure activities within target incentive ranges. Target 
incentive ranges are to be jointly agreed upon by the contracting 
officer and the systems contractor concerned and are to be specified in 
the contract's incentives clause. Under the statute, the amount of 
incentive payment a systems contractor may earn through such incentive 
clauses will be based on how early within the specified target 
incentive range the systems contractor completes destruction operations 
and facility closure activities. Section 923 sets limitations on the 
amount of incentive payments a systems contractor may receive through 
the incentives clauses. It establishes $110 million as the maximum 
incentive payment for the completion of destruction operations within 
the target incentive range specified under the contract's incentives 
clause and $55 million for the completion of facility closure 
activities within the target incentive range specified under the 
contract's incentives clause.[Footnote 69] 

Although it had not been implemented at the time of our review, CMA had 
drafted a plan for implementation of the incentives authorized under 
section 923.[Footnote 70] The plan articulates that the systems 
contractors are to assume responsibility for identifying and mitigating 
all site-specific and programmatic risks in order to successfully 
complete program objectives within budgeted cost and schedule targets. 
The plan further relates that in return, the systems contractors will 
receive incentives measured against established schedule targets for 
operations completion and facility closure. 

Under CMA's plan, it would offer the additional incentive fee of up to 
$150 million to each site--$100 million for destruction operations 
completed within target ranges by or near the April 29, 2012, treaty 
deadline, and $50 million for facility closure activities brought 
successfully to completion within target ranges. CMA's plan establishes 
the period from November 1, 2011, through October 31, 2013, as the 
target incentive range for the completion of operations at all sites, 
with the maximum amount available to be earned, exclusive of the bonus, 
at $100 million. Ninety-five percent of the incentive can be earned for 
completing operations by the CWC extended deadline of April 29, 2012. 
After the CWC deadline, a percentage of the incentives can be earned 
through October 31, 2013, on a reduced sliding scale. The CMA plan also 
offers up to $50 million per site for the completion of facility 
closure activities within a specified range of dates unique to each 
site. One hundred percent of the $50 million would be earned by a 
systems contractor for completion of facility closure activities on the 
first date of the specified range. The systems contractor would earn 
some lesser percentage of the $50 million for completion on any date 
between the first and last dates of the specified range. 

Under CMA's plan, systems contractors can earn up to $165 million at 
their respective sites through bonus potential. In addition to the 
potential $100 million fee for completing operations ahead of the 
extended CWC deadline and $50 million for meeting specified closure 
targets, bonuses can also be earned. A bonus of $10 million is offered 
to each site if all the sites complete destruction operations prior to 
the CWC deadline of April 29, 2012. A bonus of $5 million is also 
offered to each site for accelerated completion of facility closure 
activities. 

CMA's plan, which would have to be implemented under each of the 
contracts, includes language that insulates the government from having 
to make incentive payments to the systems contractors if they did not 
strictly perform under the terms CMA has set under the plan. In firmly 
establishing the target incentive ranges, the plan states that the 
target dates are not subject to extension except for catastrophic 
causes beyond the control of and without the fault or negligence of the 
systems contractor, its subcontractors, and suppliers. The plan 
establishes that the systems contractors are not entitled to submit 
claims for loss of opportunity to earn the incentives or to increase 
incentive pools due to failure of the government to provide resources 
over and above those specifically identified in the contract. It also 
stipulates that payments of incentives will be subject to adjustment 
based upon final audits of operation costs and closure costs. At the 
same time, the plan recognizes, as does the statute, that the 
enterprise of striving to capture these incentives is to be conducted 
consistent with the existing obligation to provide for maximum 
protection for the environment, the general public, and the personnel 
who are involved in the destruction of the chemical agents and 
munitions. The effectiveness of this planned incentive is uncertain 
because it had not yet been negotiated or placed on contract at the 
time of our review. 

[End of section] 

Appendix III: Comments from the Department of Defense: 

Assistant To The Secretary Of Defense: 
3050 Defense Pentagon: 
Washington, DC 20301-3050: 

Nuclear And Chemical And Biological Defense Programs: 

November 16, 2007: 

Ms. Davi M. D'Agostino: 
Director, Defense Capabilities and Management: 
United States Government Accountability Office: 
441 G Street, N.W.: 
Washington, DC 20548: 

Dear Ms. D'Agostino: 

This is the Department of Defense (DoD) response to the GAO draft 
report, GAO-08-134, `Chemical Demilitarization: Additional Management 
Actions Needed to Meet Key Performance Goals of DoD's Chemical 
Demilitarization Program,' dated October 3, 2007 (GAO Code 350799). 

The DoD concurs with most of the draft report's recommendations, except 
those pertaining to when cost and schedule estimates should be revised. 
The Department will continue to use acquisition management tools and 
discipline to ensure the destruction of the United States' chemical 
weapons stockpile in a safe and secure manner, while being economical 
and meeting our commitments under the Chemical Weapons Convention. 

The Department appreciates the opportunity to provide comments on the 
draft report. These comments, including technical comments, are 
enclosed. For further questions concerning this report, please contact 
Barbara Burgess, Senior Program Analyst for the Chemical 
Demilitarization Program, (703) 588-1983, extension 113. 

Sincerely, 

Signed by: 

Jean D. Reed: 
Special Assistant: 
Chemical and Biological Defense and Chemical Demilitarization Programs: 

Enclosures: 
As stated: 

cc:
Deputy Assistant Secretary of the Army (Elimination of Chemical 
Weapons): 

GAO Draft Report ï¿½ Dated October 3, 2007 GAO Code 350799/GAO-08-134: 

"Chemical Demilitarization: Additional Management Actions Needed to 
Meet Key Performance Goals of DoD's Chemical Demilitarization Program" 

Department Of Defense Comments To The Recommendations: 

Recommendation 1: The GAO recommends the Secretary of Defense direct 
the Director of U.S. Army Chemical Materials Agency to incorporate 
baseline and trend data for past performance and multiyear performance 
goals for the future in its annual performance plan. 

DoD Response: Concur. The Department recognizes that U.S. Army Chemical 
Materials Agency (CMA) is currently incorporating baseline and trend 
data for past performance and multiyear performance measures in the CMA 
Annual Performance Plan for 2008. This effort is expected to be 
completed and staffed by end of the first quarter of Fiscal Year 2008. 

Recommendation 2: The GAO recommends the Secretary of Defense direct 
the Director of U.S. Army Chemical Materials Agency to develop interim 
destruction goals, approaches, and milestones that are directly linked 
to overall program goals for meeting Chemical Weapons Convention 
deadlines. 

DoD Response: Partially Concur. The Department recognizes that the U.S. 
Army Chemical Materials Agency (CMA) annual destruction goals are 
interim destruction goals for meeting the Chemical Weapons Convention 
(CWC) destruction milestones. The Chemical Demilitarization Program 
Fiscal Year 2007 Annual Performance Plan provided the annual 
destruction goals for meeting the CWC 45 percent destruction milestone 
of December 31, 2007. The Department will continue to ensure all future 
CMA performance plans incorporate annual goals and approaches leading 
toward the April 2012 destruction deadline through direction issued 
annually in support of strategic governance. Therefore, no further 
direction is required from the Secretary of Defense to develop interim 
goals, approaches, and milestones. 

Recommendation 3: The GAO recommends the Secretary of Defense direct 
the Director of U.S. Army Chemical Materials Agency to establish a time 
frame for completing its risk management approach, including 
integration across sites and with DoD. 

DoD Response: Concur. The U.S. Army Chemical Materials Agency (CMA) is 
currently evaluating and updating the CMA risk management approach 
including integration of risk management across sites and within CMA 
Headquarters. The Department expects an approved risk management 
approach to be completed by the third quarter of Fiscal Year 2008. 

Recommendation 4: The GAO recommends the Secretary of Defense direct 
the Director of U.S. Army Chemical Materials Agency to update the 2005 
program schedule milestones for each of DoD's chemical demilitarization 
sites to reflect recent site processing rates and then adjust the cost 
estimate accordingly. 

DoD Response: Partially Concur. The Department recognizes successes 
within the Chemical Demilitarization Program (CDP) over the past two 
years. However, there are still some areas of the program, (i.e., 
destruction of secondary hazardous waste and closure of the chemical 
weapons destruction facilities), with limited information to adjust 
cost and schedule estimates. As the GAO stated in their final report: 
NSIAD-00-80, Chemical Weapons Disposal: Improvements Needed in Program 
Accountability and Financial Management, May 2000: "The Army also needs 
additional time to develop a reliable baseline to estimate the costs 
for the closure and remediation of the chemical stockpile disposal 
facilities.Furthermore, because no stockpile disposal facility has yet 
to be remediated, the Army lacks real time experience on which to 
estimate these costs." The CDP now has two sites to compare closure 
activities, Johnston Atoll Chemical Agent Destruction System (JACADS) 
and Aberdeen Chemical Agent Disposal Facility (ABCDF). JACADS completed 
destruction operations in November 2000 and most closure activities in 
June 2004, with the exception of final closure of the Resource 
Conservation and Recovery Act permit, which is not expected until 
November 2008. The ABCDF completed operations in February 2006 and 
closure activities were completed in March 2007. With this limited 
information for closure, the U.S. Army Chemical Materials Agency (CMA) 
has recently completed contract performance baselines that include 
operations, secondary waste processing, closure and known risk, for 
sites at Anniston, Alabama; Pine Bluff, Arkansas; and Umatilla, Oregon. 

In addition, the Department approved the revised Acquisition Program 
Baselines for the Chemical Demilitarization Program ï¿½ U.S. Army 
Chemical Materials Agency (Chem Demil- CMA) and Chem Demil-CMA Newport 
Major Defense Acquisition Programs in April 2006, with the cost and 
schedule estimates based on the Cost Analysis Improvement Group's 
(CAIG) estimates. The CAIG estimates were certified to Congress in May 
2005 as part of the Nunn- McCurdy unit cost breach. The increase in the 
unit cost was due primarily to more realistic schedule estimates. 

Therefore, the Department does not believe it is prudent to adjust cost 
and schedule estimates or rebaseline at this time due to limited 
information gained during the past year associated with processing 
rates, unknown risks, and the potential for a chemical incident. The 
Department will evaluate the cost and schedule estimates for the entire 
CDP during the upcoming Fiscal Year 2010-2015 Program Objectives 
Memorandum build process and program review, and will determine whether 
an update is warranted. 

Recommendation 5: The GAO recommends the Secretary of Defense direct 
the Chemical Analysis Improvement Group to establish a periodic 
schedule to review the program's processing history to determine 
whether the schedule milestones are still reasonable. 

DoD RESPONSE: Concur. The Department conducts a biannual program review 
of the Chemical Demilitarization Program in support of the budget 
estimate submission. During this review, the program office schedule 
and cost estimates are reviewed in comparison with the approved 
Acquisition Program Baselines to determine whether there are any 
significant differences. If there are significant differences, the 
Department will request the Cost Analysis Improvement Group conduct an 
independent assessment and possibly develop cost and schedule 
estimates. The next review will occur during the Fiscal Year 2010-2015 
Program Objectives Memorandum build process and program review in the 
spring of calendar year 2008. 

Recommendation 6: The GAO recommends the Secretary of Defense direct 
the Under Secretary of Defense for Acquisition, Technology and 
Logistics to include in the Chemical Demilitarization Program's 
Acquisition Program Baseline and Selected Acquisition Report the 
updated and reviewed scheduled milestones for completing destruction 
operations and facility closure activities for each of the chemical 
agent destruction sites. 

DoD Response: Concur. The Department will request administrative 
changes to the Chemical Demilitarization Program-U.S. Army Chemical 
Materials Agency (Chem Demil ï¿½ CMA) and Chem Demil-CMA Newport 
Acquisition Program Baselines and upcoming Selected Acquisition Reports 
to remove the "To Be Determined (TBDs)," which were used as a 
placeholder for schedule milestones beyond April 2012, and replace with 
the Cost Analysis Improvement Group's schedule estimates. 

Recommendation 7: The GAO recommends the Secretary of Defense direct 
the Director of the U.S. Army Chemical Materials Agency to coordinate 
with the systems contractors in defining closure requirements and 
developing accurate and realistic closure cost estimates for each of 
DoD's chemical demilitarization sites. 

DoD Response: Non-Concur. The Department recommends GAO delete this 
recommendation since Recommendation #9 is more encompassing. 

Recommendation 8: The GAO recommends the Secretary of Defense direct 
the Cost Analysis Improvement Group to conduct an independent cost 
review of the cost estimates. 

DoD Response: Partially Concur. The Department conducts a biannual 
program review of the Chemical Demilitarization Program in support of 
the budget estimate submission. During this review, the program office 
schedule and cost estimates are reviewed in comparison with the 
approved Acquisition Program Baselines to determine whether there are 
any significant differences. If there are significant differences, the 
Department will request the Cost Analysis Improvement Group conduct an 
independent assessment and possibly develop cost and schedule 
estimates. Therefore, no direction to conduct a cost review will be 
given at this time. 

Recommendation 9: The GAO recommends the Secretary of Defense direct 
the Director of the U.S. Army Chemical Materials Agency to define all 
contract requirements, including those associated with facility closure 
activities, and develop comprehensive cost estimates through closure at 
each chemical destruction site. 

DoD Response: Partially Concur. Since June 2007, three of the five U.S. 
Army Chemical Materials Agency chemical weapons stockpile destruction 
sites (Anniston, Alabama; Pine Bluff, Arkansas; and Umatilla, Oregon) 
have completed contract performance baselines that include contract 
requirements for operations, secondary waste processing, closure and 
known risk. Negotiations are under way to complete the contract 
performance baseline by the end of the third quarter of Fiscal Year 
2008 for the Tooele, Utah, site. Negotiations are expected to be 
completed for the contract requirements and cost estimates associated 
with the closure phase for the Newport, Indiana, site by the end of the 
third quarter of Fiscal Year 2008. The Department believes no further 
direction is required. 

Recommendation 10: The GAO recommends the Secretary of Defense direct 
the Director of the U.S. Army Chemical Materials Agency to establish a 
time frame for completing the negotiation for placing all authorized 
work that has not been priced on contract and ensuring that the 
negotiations take place. 

DoD Response: Concur. The U.S. Army Chemical Materials Agency has 
negotiated almost all work presently on contract (current and future) 
resulting in low levels of authorized, unpriced work. The Department 
expects negotiations to be completed by the end of the third quarter of 
Fiscal Year 2008. 

Recommendation 11: The GAO recommends the Secretary of the Defense 
direct the Director of the U.S. Army Chemical Materials Agency to 
determine for each of the four incineration site contracts whether 
greater weight can be applied to performance measures, such as schedule 
and cost, in award fee plans to further incentivize the systems 
contractors to increase the destruction rate of chemical agents in a 
safe and environmentally sound manner. 

DoD Response: Concur. The Department will conduct a review of the award 
fee plans of the U.S. Army Chemical Materials Agency incineration site 
contracts to determine whether the weighting factors are appropriate to 
incentivize the systems contractors to reduce the schedule while 
operating in a safe, secure, and environmentally-sound manner. The 
results will be available by second quarter of FY08. 

Recommendation 12: The GAO recommends the Secretary of the Defense 
direct the Director of the U.S. Army Chemical Materials Agency better 
align its award fee policy with DoD's March 2006 guidance on award 
fees. 

DoD Response: Concur. This action was completed on September 7, 2007, 
in accordance with the March 29, 2006, Memorandum: Subject: Award Fee 
Contracts, from the Under Secretary of Defense for Acquisition, 
Technology and Logistics. CMA revised Policy Statement 29, "Procedure 
for Implementing Contractor Incentives to Include Award Fee (AF) and 
Performance Based Incentive (PBI) Fee." 

Recommendation 13: The GAO recommends the Secretary of the Defense 
direct the Director of the U.S. Army Chemical Materials Agency to link 
more of its award fee criteria to performance measures that focus on 
identifiable interim outcomes, discrete events, or milestones. 

DoD Response: Partially Concur. The Department will conduct a review of 
the U.S. Army Chemical Materials Agency (CMA) award fee plans to 
determine whether there are performance measures that focus on outcomes 
or outputs. This review will be completed along with the review of the 
award fee weighting factors by the end of the second quarter of Fiscal 
Year 2008. Based on the findings of the review, the Department will 
determine whether further direction and action is required. Currently, 
the CMA Fiscal Year 2006 and Fiscal Year 2007 Award Fee plans included 
objective schedule criteria that focused on U.S. tons of chemical agent 
destroyed to complete destruction by April 2012. 

[End of section] 

Appendix IV: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Davi M. D'Agostino, (202) 512-5431 or [email protected]: 

Acknowledgments: 

Mark A. Pross, Assistant Director; Bonita P. Anderson; Rodell B. 
Anderson; Susan C. Ditto; Jennifer K. Echard; Neil D. Feldman; James C. 
Lawson; Lonnie J. McAllister; Brian L. Octeau; Charles W. Perdue; Karen 
A. Richey; John W. Van Schaik; Cheryl A. Weissman; and Alex M. Winograd 
made key contributions to this report. 

[End of section] 

Related GAO Products: 

Cost Assessment Guide: Best Practices for Estimating and Managing 
Program Costs. GAO-07-1134SP. Washington, D.C.: July 2007 (Exposure 
Draft). 

Chemical Demilitarization: Actions Needed to Improve the Reliability of 
the Army's Cost Comparison Analysis for Treatment and Disposal Options 
for Newport's VX Hydrolysate. GAO-07-240R. Washington, D.C.: January 
26, 2007. 

Chemical Weapons: Sustained Leadership, Along with Key Strategic 
Management Tools, Is Needed to Guide DOD's Destruction Program. GAO-03- 
1031. Washington, D.C.: September 5, 2003. 

Chemical Weapons: Lessons Learned Program Generally Effective but Could 
Be Improved and Expanded. GAO-02-890. Washington, D.C.: September 10, 
2002. 

Chemical Weapons: FEMA and Army Must Be Proactive in Preparing States 
for Emergencies. GAO-01-850. Washington, D.C.: August 13, 2001. 

Chemical Weapons Disposal: Improvements Needed in Program 
Accountability and Financial Management. GAO/NSIAD-00-80. Washington, 
D.C.: May 8, 2000. 

Chemical Demilitarization: Funding Status of the Chemical 
Demilitarization Program. GAO/NSIAD-99-232R. Washington, D.C.: July 29, 
1999. 

Chemical Weapons: DOD Does Not Have a Strategy to Address Low-Level 
Exposures. GAO/NSIAD-98-228. Washington, D.C.: September 23, 1998. 

Chemical Weapons Stockpile: Changes Needed in the Management of the 
Emergency Preparedness Program. GAO/NSIAD-97-91. Washington, D.C.: June 
11, 1997. 

Chemical Weapons and Materiel: Key Factors Affecting Disposal Costs and 
Schedule. GAO/T-NSIAD-97-118. Washington, D.C.: March 11, 1997. 

Chemical Weapons Stockpile: Emergency Preparedness in Alabama Is 
Hampered by Management Weaknesses. GAO/NSIAD-96-150. Washington, D.C.: 
July 23, 1996. 

Chemical Weapons Disposal: Issues Related to DOD's Management. GAO/T- 
NSIAD-95-185. Washington, D.C.: July 13, 1995. 

Chemical Weapons: Army's Emergency Preparedness Program Has Financial 
Management Weaknesses. GAO/NSIAD-95-94. Washington, D.C.: March 15, 
1995. 

Chemical Stockpile Disposal Program Review. GAO/NSIAD-95-66R. 
Washington, D.C.: January 12, 1995. 

Chemical Weapons: Stability of the U.S. Stockpile. GAO/NSIAD-95-67. 
Washington, D.C.: December 22, 1994. 

Chemical Weapons Disposal: Plans for Nonstockpile Chemical Warfare 
Materiel Can Be Improved. GAO/NSIAD-95-55. Washington, D.C.: December 
20, 1994. 

Chemical Weapons: Issues Involving Destruction Technologies. GAO/T- 
NSIAD-94-159. Washington, D.C.: April 26, 1994. 

Chemical Weapons Destruction: Advantages and Disadvantages of 
Alternatives to Incineration. GAO/NSIAD-94-123. Washington, D.C.: March 
18, 1994. 

Arms Control: Status of U.S.-Russian Agreements and the Chemical 
Weapons Convention. GAO/NSIAD-94-136. Washington, D.C.: March 15, 1994. 

Chemical Weapon Stockpile: Army's Emergency Preparedness Program Has 
Been Slow to Achieve Results. GAO/NSIAD-94-91. Washington, D.C.: 
February 22, 1994. 

Chemical Weapons Storage: Communities Are Not Prepared to Respond to 
Emergencies. GAO/T-NSIAD-93-18. Washington, D.C.: July 16, 1993. 

Chemical Weapons Destruction: Issues Affecting Program Cost, Schedule, 
and Performance. GAO/NSIAD-93-50. Washington, D.C.: January 21, 1993. 

Chemical Weapons Destruction: Issues Related to Environmental 
Permitting and Testing Experience. GAO/T-NSIAD-92-43. Washington, D.C.: 
June 16, 1992. 

Chemical Weapons Disposal. GAO/NSIAD-92-219R. Washington, D.C.: May 14, 
1992. 

Chemical Weapons: Stockpile Destruction Cost Growth and Schedule 
Slippages Are Likely to Continue. GAO/NSIAD-92-18. Washington, D.C.: 
November 20, 1991. 

Chemical Weapons: Physical Security for the U.S. Chemical Stockpile. 
GAO/NSIAD-91-200. Washington, D.C.: May 15, 1991. 

Chemical Warfare: DOD's Effort to Remove U.S. Chemical Weapons from 
Germany. GAO/NSIAD-91-105. Washington, D.C.: February 13, 1991. 

Chemical Weapons: Status of the Army's M687 Binary Program. GAO/NSIAD- 
90-295. Washington, D.C.: September 28, 1990. 

Chemical Weapons: Stockpile Destruction Delayed at the Army's Prototype 
Disposal Facility. GAO/NSIAD-90-222. Washington, D.C.: July 30, 1990. 

Chemical Weapons: Obstacles to the Army's Plan to Destroy Obsolete U.S. 
Stockpile. GAO/NSIAD-90-155. Washington, D.C.: May 24, 1990. 

[End of section] 

Footnotes: 

[1] The Convention on the Prohibition of the Development, Production, 
Stockpiling and Use of Chemical Weapons and on Their Destruction is 
commonly known as the Chemical Weapons Convention. 

[2] In 1996, Congress directed DOD to evaluate the use of alternative 
technologies and suspended incineration planning activities at two of 
the nine storage sites with assembled weapons--Blue Grass Chemical 
Agent-Destruction Pilot Plant, Blue Grass Chemical Activity, Kentucky, 
and Pueblo Chemical Agent-Destruction Pilot Plant, Pueblo Chemical 
Depot, Colorado. See Pub. L. No. 104-201, ï¿½ 142 (1996) and Pub. L. No. 
104-208, ï¿½ 8065 (1996). In addition, Congress directed that those two 
sites be managed in a program independent from the Army's Chemical 
Demilitarization Program and report directly to DOD instead of to the 
Army. See Pub. L. No. 105-261, ï¿½ 142 (1998). The two sites are managed 
under a separate program known as the Assembled Chemical Weapons 
Alternatives program, and are not the focus of this report. 

[3] The U.S. deadline for 100 percent destruction of the unitary 
stockpile was extended from April 29, 2007, to April 29, 2012. The CWC 
allows an extension of up to 5 years, and an extension was granted to 
the United States at its request in December 2006. For purposes of this 
report, we refer to the extended CWC deadline of April 29, 2012, as the 
U.S. treaty deadline, unless otherwise indicated. 

[4] A Nunn-McCurdy unit cost breach occurs when a major defense 
acquisition program experiences an increase of at least 15 percent in 
program acquisition unit cost or procurement unit cost above the unit 
costs in the current APB. Unit cost reporting is required by 10 U.S.C. 
ï¿½ 2433. For programs with unit cost increases of at least 25 percent, a 
certification by the Secretary of Defense is required. Certification 
responsibility has been delegated to the Under Secretary of Defense for 
Acquisition, Technology, and Logistics, who, by law, must certify that 
(1) such an acquisition program is essential to national security; (2) 
there are no alternatives to such an acquisition program, which will 
provide equal or greater military capability at less cost; (3) the new 
estimates of the program acquisition unit cost or procurement unit cost 
are reasonable; and (4) the management structure for the acquisition 
program is adequate to manage and control program acquisition unit cost 
or procurement unit cost. See 10 U.S.C. ï¿½ 2433 (b)(2)(A) (2007). 

[5] Total program cost estimates exceeded APBs in 2001 and again in 
2005. On May 2, 2002, the Under Secretary of Defense for Acquisition, 
Technology, and Logistics--the Defense Acquisition Executive-- 
submitted a Nunn-McCurdy breach certification to Congress for the 
Chemical Demilitarization Program as required by 10 U.S.C. ï¿½ 2433. The 
unit cost breach was due primarily to (1) revised processing rates; (2) 
schedule extensions; (3) new environmental regulations; (4) worse-than- 
expected condition of the stockpile; (5) increase in equipment, labor 
rates, and construction costs; and (6) higher emergency preparedness 
costs. On May 6, 2005, the Under Secretary of Defense for Acquisition, 
Technology, and Logistics again certified to Congress that the program 
acquisition unit costs for the Chemical Demilitarization Program had 
increased 41 percent (in then-year dollars) due primarily to more 
realistic schedule estimates associated with operations and closure 
phases, environmental regulatory compliance, and concept/design 
maturation. Program acquisition unit costs for the Newport Chemical 
Agent Disposal Facility, Newport Chemical Depot, Indiana, program 
increased by 52 percent (in then-year dollars) due primarily to more 
realistic schedule estimates associated with the nerve agent 
neutralization process and secondary hazardous waste generation. 

[6] The Chemical Stockpile Elimination Project includes the program's 
four operating incineration sites and two bulk-agent-only 
neutralization sites. 

[7] These four incineration sites; the neutralization site at Newport 
Chemical Agent Disposal Facility, Newport Chemical Depot, Indiana; and 
the two closed sites that were at Johnston Atoll in the Pacific Ocean 
and Aberdeen Chemical Agent Disposal Facility, Maryland, are managed 
for the Army by the Chemical Materials Agency, which was formerly known 
as the Program Manager for Chemical Demilitarization. 

[8] GAO, Chemical Weapons: Sustained Leadership, Along with Key 
Strategic Management Tools, Is Needed to Guide DOD's Destruction 
Program, GAO-03-1031 (Washington, D.C.: Sept. 5, 2003). 

[9] GAO, Defense Acquisitions: DOD Wastes Billions of Dollars through 
Poorly Structured Incentives, GAO-06-409T (Washington, D.C.: Apr. 5, 
2006). 

[10] GAO, Defense Acquisitions: DOD Has Paid Billions in Award and 
Incentive Fees Regardless of Acquisition Outcomes, GAO-06-66 
(Washington, D.C.: Dec. 19, 2005). 

[11] The Office of Management and Budget (OMB) defines EVM as a project 
management tool that effectively integrates the project scope of work 
with cost, schedule, and technical performance elements for optimal 
project planning and control. Beginning in August 2005, OMB 
specifically required federal agencies and contractors to use EVM 
systems on all new capital assets. See Office of Management and Budget, 
Capital Programming Guide (Washington, D.C.: 2006). Also, DOD requires 
the use of EVM on all programs greater than $20 million. 

[12] GAO's prior recommendations are found in GAO-03-1031, p. 26. 

[13] GAO, Cost Assessment Guide: Best Practices for Estimating and 
Managing Program Costs,GAO-07-1134SP (Washington, D.C.: July 2007, 
Exposure Draft). 

[14] The Department of Defense Authorization Act for Fiscal Year 1986, 
Pub. L. No. 99-145, ï¿½ 1412(a) (1985). 

[15] Nerve agents are the most toxic and rapidly acting of known 
chemical warfare agents. GB, also known as sarin, and VX are human-made 
lethal chemical warfare nerve agents that affect the nervous system by 
interfering with the signals sent from the brain to vital organs. 

[16] Blister agents include mustard, which is a type of chemical 
warfare agent and powerful irritant that causes blistering of the skin 
and mucous membranes on contact. 

[17] Pub. L. No. 104-201, ï¿½ 142 (1996). 

[18] Since the 1930s, the Army and the chemical industry have used ton 
containers to store and ship bulk chemicals, including chemical agent. 
Equivalent in length and diameter to two stacked 55-gallon drums, a ton 
container weighs approximately 1,600 pounds and measures nearly 7 feet 
in length. 

[19] The stockpile at the Newport Chemical Depot, Indiana, is the first 
U.S. stockpile containing VX that will be destroyed by using 
neutralization--a process that mixes hot water and sodium hydroxide (a 
caustic chemical) with VX to change the chemical composition to a less 
toxic form. The resulting by-product is a liquid wastewater commonly 
referred to as hydrolysate that consists mostly of water but also has a 
caustic component and organic salts that need further treatment to meet 
CWC requirements and to meet federal and state environmental 
requirements for disposal. The hydrolysate is being stored in 
intermodal storage containers on-site until a post-treatment plan can 
be implemented. For additional information on the disposal of 
hydrolysate, see GAO, Chemical Demilitarization: Actions Needed to 
Improve the Reliability of the Army's Cost Comparison Analysis for 
Treatment and Disposal Options for Newport VX Hydrolysate, GAO-07-240R 
(Washington, D.C.: Jan. 26, 2007). 

[20] Pub. L. No. 104-201, ï¿½ 142 (1996). 

[21] Pub. L. No. 105-261, ï¿½ 142 (1998). 

[22] The CWC implementing legislation provides the statutory authority 
for domestic compliance with the convention's provisions. See Pub. L. 
No. 105-277, Division I (1998). 

[23] This report solely focuses on the weapons the convention defines 
as category 1, which are the most dangerous chemicals in the stockpile. 

[24] In 2004, DOD split the Chemical Demilitarization Program into 
three major defense acquisition programs: (1) all CMA chemical agent 
destruction sites, except for the Newport site; (2) the Newport site; 
and (3) the ACWA program. 

[25] GAO-03-1031. 

[26] Pub. L. No. 108-375 ï¿½ 931 (2004) directed the Under Secretary of 
Defense for Acquisition, Technology, and Logistics and the Secretary of 
the Army to jointly prepare a strategic plan for future activities for 
the destruction of the U.S. stockpile of lethal chemical agents and 
munitions. Section 931 further directed that the plan include, at a 
minimum, realistic budgeting for stockpile destruction and related 
support programs, contingency planning for foreseeable or anticipated 
problems, and a management approach and associated actions that address 
compliance with the obligations of the United States under the CWC 
treaty and that take full advantage of opportunities to accelerate 
destruction of the stockpile. 

[27] The performance plan applies to both CMA and the ACWA portions of 
the demilitarization program. It was the program's first plan and was 
issued jointly by both organizations. 

[28] GAO, Agency Performance Plans: Examples of Practices That Can 
Improve Usefulness to Decisionmakers, GAO/GGD/AIMD-99-69 (Washington, 
D.C.: Feb. 26, 1999). 

[29] The Probability of Program Success Report is designed to improve 
the Army's ability to accurately assess the program's probability of 
success, and clearly and concisely represent that success probability 
to leadership. 

[30] See, for instance, GAO, Executive Guide: Effectively Implementing 
the Government Performance Results Act, GAO/GGD-96-118 (Washington, 
D.C.: June 1996). 

[31] OMB's PART was developed to assess and improve program performance 
so that the federal government can achieve better results. A review 
using this tool helps identify a program's strengths and weaknesses to 
inform funding and management decisions aimed at making the program 
more effective. The tool, therefore, looks at all factors that affect 
and reflect program performance, including program purpose and design; 
performance measurement, evaluations, and strategic planning; program 
management; and program results. Because the tool includes a consistent 
series of analytical questions, it allows programs to show improvements 
over time, and allows comparisons between similar programs. See Office 
of Management and Budget, Program Assessment Rating Tool (PART), 
[hyperlink, http://www.whitehouse.gov/omb/part/]. 

[32] GAO/GGD/AIMD-99-69. 

[33] GAO-07-1134SP. 

[34] A histogram is a graphical display of tabulated frequencies. 

[35] The OSD CAIG has specific responsibility for providing independent 
cost estimates when requested by the Under Secretary of Defense for 
Acquisition, Technology, and Logistics in support of major milestone 
decision points (Milestone B, Milestone C, or the full-rate production 
decision review). Overall, CAIG serves as the principal advisory body 
to the Milestone Decision Authority on all matters concerning an 
acquisition program's life cycle cost. 

[36] Defense Contract Management Agency, Department of Defense Earned 
Value Management Implementation Guide (Washington, D.C.: Apr. 7, 2005). 
Also see DOD Memorandum, Revision to DOD Earned Value Management Policy 
(Washington, D.C.: Mar. 7, 2005), and Office of Management and Budget, 
Capital Programming Guide (Washington, D.C.: 2006). 

[37] EVM is a process required by OMB that can help program managers 
track program progress by using the value of work done as a basis for 
estimating the cost to compete the project. 

[38] Defense Contract Management Agency, Department of Defense Earned 
Value Management Implementation Guide. 

[39] GAO-07-1134SP. 

[40] Defense Contract Management Agency, Department of Defense Earned 
Value Management Implementation Guide. 

[41] In its technical comments on a draft of this report, DOD expressed 
concern about our statement that the EVM system used at each site may 
not be reliable. As a result, the department stated that it will 
request the Defense Contract Management Agency to conduct an audit of 
the CMA EVM system beginning no later than June 30, 2008. 

[42] The systems contractor and government have identified work that 
(authorized but unpriced) is negotiated and added to the contract price 
annually. 

[43] Defense Contract Management Agency, Department of Defense Earned 
Value Management Implementation Guide. Also, see GAO-07-1134SP. 

[44] In its technical comments on a draft of this report, DOD expressed 
concern about our statement that shortcomings in the underlying (EVM) 
data include invalid and unstable performance baselines. DOD stated 
that the issues with the contract baselines are issues of program 
management or risk, not issues of EVM data or process reliability. DOD 
stated that this problem has been ameliorated by the recent completion 
of life cycle contract negotiations at three of the five sites covered 
by our report (Anniston, Pine Bluff, and Umatilla). The contract 
negotiations for Newport and Toole are ongoing and should be completed 
by June 30, 2008. The department stated that it will request the 
Defense Contract Management Agency to conduct an audit of the CMA EVM 
system beginning no later than June 30, 2008. 

[45] "Comprehensive" means that the cost estimate is complete and 
accounts for all possible costs. Comprehensive cost estimates 
completely define the program, reflect the current schedule, and are 
technically reasonable. See GAO-07-1134SP. 

[46] Johnston Atoll was closed in December 2003 and Aberdeen was closed 
in May 2007. 

[47] The systems contractor and government have identified work that 
(authorized but unpriced) is negotiated and added to the contract price 
annually. 

[48] The contracts use Operating and Maintenance funds, which are 
available for obligation for 1 fiscal year. This creates the need to 
negotiate the contracts on a yearly basis, since each fiscal year's 
funds are appropriated. 

[49] GAO-07-1134SP. 

[50] Defense Contract Management Agency, Department of Defense Earned 
Value Management Implementation Guide. 

[51] GAO-07-1134SP. 

[52] In its technical comments on a draft of this report, DOD stated 
that the Army agrees that there is usually more work that is being 
considered and not yet in the contract baseline. The Army did not agree 
that this situation affects the EVM data. DOD stated that the Army uses 
the data to determine current variance and trend information. The data 
are also used by the systems contractors and CMA program office along 
with estimates of additional future work to evaluate the life cycle 
schedule and costs. The department stated that it will request the 
Defense Contract Management Agency to conduct an audit of the CMA EVM 
system beginning no later than June 30, 2008. 

[53] CMA had previously incentivized schedule as a distinct performance 
measure in fiscal year 2002 at all four incineration sites and 
additionally in fiscal years 2000 and 2001 at Tooele. 

[54] GAO-06-66. 

[55] The initial actual percentages of the base and award fees for each 
of these contracts were originally established through the full and 
open competitions conducted for each of these contracts. Each 
contractor's proposal included a percentage for both the base fee and 
the award fee, which the government evaluated as part of the contract 
selection process. With the exception of one systems contractor, all 
the contractors proposed a total fee of 10 percent with up to 3 percent 
proposed for the base fee and the remaining 7 percent allocated to 
award fee. The one exception is Newport where the contractor proposed a 
base fee of 3 percent and an award fee of 3 percent. 

[56] However, these incentives did not keep the Newport contract on 
schedule. The systems contractor missed the target completion date for 
the third of its four performance incentive milestones and the program 
was delayed by over a year. According to DOD, the failure to meet this 
milestone was due to unforeseen technical difficulties and could not 
have been ultimately influenced by any type of contractual language. 

[57] Schedule was a distinct performance measure in the Tooele award 
fee plan for fiscal years 2000 and 2001. In 2002, it was a distinct 
performance measure in the award fee plans at each of the incineration 
sites. 

[58] CMA officials noted they were acting in accordance with Congress's 
mandate that the program shall provide for the maximum protection of 
the workforce, the public, and the environment. See 50 U.S.C. 1521 
(c)(1)(A) (2007). 

[59] See the Award Fee Review Board Report for the Anniston Chemical 
Agent Disposal Facility, Evaluation Number 17, for the evaluation 
period from April 1, 2005, to September 30, 2005. 

[60] See the Award Fee Evaluation Board Report for the Umatilla 
Chemical Agent Disposal Facility, First Period, Fiscal Year 2005, for 
the period of performance from October 1, 2004, to March 31, 2005. 

[61] Memorandum from the Under Secretary of Defense for Acquisition, 
Technology, and Logistics on Chemical Demilitarization Program 
Strategic Governance (Washington, D.C.: Jan. 6, 2006). 

[62] GAO-06-66 and GAO, NASA Procurement: Use of Award Fees for 
Achieving Program Outcomes Should Be Improved, GAO-07-58 (Washington, 
D.C.: Jan. 17, 2007). 

[63] Memorandum from the Under Secretary of Defense for Acquisition, 
Technology, and Logistics on Award Fee Contracts (FAR 16, DFARS 215, 
DFARS 216) (Washington, D.C.: Mar. 29, 2006). In addition, the National 
Defense Authorization Act for Fiscal Year 2007 directed DOD to issue 
guidance, with detailed implementation instructions, to ensure that all 
new contracts using award fees link such fees to acquisition outcomes, 
which are defined in terms of program cost, schedule, and performance. 
See Pub. L. No. 109-364, ï¿½ 814 (2006). Furthermore, in an April 2007 
memorandum, DOD issued further award fee guidance stating that it is 
the policy of the department that for new solicitations issued on or 
after August 1, 2007, objective criteria be utilized, whenever 
possible, to measure contract performance. If it is determined that 
objective criteria do not exist, then the head of the contracting 
activity must sign a determination and finding that "the work to be 
performed is such that it is neither feasible nor effective to devise 
predetermined objective incentive targets applicable to cost, technical 
performance, or schedule." 

[64] See GAO-06-66 and GAO-07-58. 

[65] Pub. L. No. 109-364 (2006). 

[66] Chemical Materials Agency, Incentive Agreement for Completion of 
Destruction Operations and Facility Closure Activities at Chemical 
Stockpile Disposal Facilities (Washington, D.C.: May 19, 2007). 

[67] Pub. L. No. 109-364 (2006). 

[68] Pub. L. No. 109-364, ï¿½ 923 (a)(2) (2006). 

[69] Pub. L. No. 109-364, ï¿½ 923 (b)(2) (2006). 

[70] Chemical Materials Agency, Incentive Agreement for Completion of 
Destruction Operations and Facility Closure Activities at Chemical 
Stockpile Disposal Facilities (Washington, D.C.: May 19, 2007). 

GAO's Mission:  

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability.  

Obtaining Copies of GAO Reports and Testimony:  

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "Subscribe to Updates."  

Order by Mail or Phone:  

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to:  

U.S. Government Accountability Office: 
441 G Street NW, Room LM: 
Washington, D.C. 20548:  

To order by Phone: 
Voice: (202) 512-6000: 
TDD: (202) 512-2537: 
Fax: (202) 512-6061:  

To Report Fraud, Waste, and Abuse in Federal Programs:  

Contact:  

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: [email protected]: 
Automated answering system: (800) 424-5454 or (202) 512-7470:  

Congressional Relations:  

Gloria Jarmon, Managing Director, [email protected]: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548:  

Public Affairs: 

Chuck Young, Managing Director, [email protected]: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: 

*** End of document. ***