Best Practices: Setting Requirements Differently Could Reduce Weapon Systems' Total Ownership Costs (02/11/2003, GAO-03-57}

-------------------------Indexing Terms-------------------------
REPORTNUM:   GAO-03-57
    TITLE:   Best Practices: Setting Requirements Differently Could Reduce Weapon Systems' Total Ownership Costs
     DATE:   02/11/2003



This is the accessible text file for GAO report number GAO-03-57 
entitled 'Best Practices: Setting Requirements Differently Could Reduce 
Weapon Systems' Total Ownership Costs' which was released on February 
11, 2003.



This text file was formatted by the U.S. General Accounting Office 

(GAO) to be accessible to users with visual impairments, as part of a 

longer term project to improve GAO productsï¿½ accessibility. Every 

attempt has been made to maintain the structural and data integrity of 

the original printed product. Accessibility features, such as text 

descriptions of tables, consecutively numbered footnotes placed at the 

end of the file, and the text of agency comment letters, are provided 

but may not exactly duplicate the presentation or format of the printed 

version. The portable document format (PDF) file is an exact electronic 

replica of the printed version. We welcome your feedback. Please E-mail 

your comments regarding the contents or accessibility features of this 

document to [email protected].



Report to the Subcommittee on Readiness and Management Support, 

Committee on Armed Services, U.S. Senate:



United States General Accounting Office:



GAO:



February 2003:



BEST PRACTICES:



Setting Requirements Differently Could Reduce Weapon Systemsï¿½ Total 

Ownership Costs:



GAO-03-57:



GAO Highlights: 



Highlights of GAO-03-57, a report to the Subcommittee on Readiness and 

Management Support, Committee on Armed Services, U.S. Senate:



February 2003:



BEST PRACTICES:



Setting Requirements Differently Could Reduce Weapon Systemsï¿½ Total 

Ownership Costs:



Why GAO Did This Study: 



For fiscal year 2003, the Department of Defense (DOD) asked for about 

$185 billion to develop, procure, operate, and maintain its weapon 
systems. 

This request represents an increase of 18 percent since 2001 for the 
total 

ownership costs of DOD weapon systems. Often, DOD systems need 
expensive 

spare parts and support systems after they are fielded to meet required 

readiness levels. DOD has been increasingly concerned that the high 
cost 

of maintaining systems has limited its ability to modernize and invest 
in 

new weapons. This report examines the best practices of leading 
commercial 

firms to manage a productï¿½s total ownership costs and determines if 
those 

practices can be applied to DOD. 



What GAO Found: 



Even though DOD has implemented several initiatives to reduce total 

ownership costs, some systems, such as the Apache helicopter or the 
Abrams 

tank, have experienced costly maintenance problems and low readiness 
rates, 

which persisted even after the systems were fielded. We found several 

reasons for these problems. First, DOD based requirements for weapon 
systems 

in product development almost exclusively on technical performance, 
with 

little attention to operating and support costs and readiness at the 
beginning 

of development when there is the greatest chance of affecting those 
costs 

positively. Second, using immature technologies to meet performance 
goals 

weakened DODï¿½s ability to design weapon systems with high reliability. 
Finally, 

DODï¿½s organizational structure is linear and limits collaboration and 
feedback 

among organizations charged with requirements setting, product 
development, 

and maintenance.



In contrast, commercial companies that we visited considered operating 
and 

support costs to be integral to their new product development 
decisions. 

Studies have shown that by the time a product is ready for development, 
over 

90 percent of the operating and support costs have been determined. As 
a 

result, these companies required their equipment be easy to maintain, 
ready 

when needed, and reliable at a low cost. These requirements were of 
equal 

importance to other performance characteristics. After setting 
requirements, 

product developers then designed products to meet established 
reliability 

rates, using technologies that were proven through past use or testing. 
At 

all of the companies we visited, customers and product developers 
alike, had 

very collaborative processes and practices that draw extensively on 
data from 

past operations to influence the design of new products.



Figure: Percent of Life Cycle Costs Determined at Various Points in the 

Acquisition Process:



[See PDF for image]



[End of figure]



What GAO Recommends:



GAO recommends DOD (1) revise its guidance for setting requirements to 
include 

total ownership cost goals and readiness rates for any major weapons 
system as 

performance parameters equal to any others; (2) revise acquisition 
regulations 

to require a firm estimate of component and subsystem reliability by 
the systems 

integration phase and an estimate of system reliability at the 
production 

decision; and (3) structure contracts to ensure proper trade-offs 
between 

reliability and performance.



www.gao.gov/cgi-bin/getrpt?GAO-03-57.



To view the full report, including the scope and methodology, click on 
the 

link above. For more information, contact Katherine Schinasi at (202) 
512-4841.



Contents:



Letter:



Executive Summary:



Purpose:



Background:



Results in Brief:



Principal Findings:



Recommendations for Executive Action:



Agency Comments:



Chapter 1: Introduction:



Total Ownership Cost Is the Cost to Ensure Readiness:



Commercial Best Practices:



Objectives, Scope, and Methodology:



Chapter 2: DODï¿½S Requirements-Setting and Development Practices Yield 

Higher Total Ownership Costs:



DODï¿½s Weapon System Programs Encounter Cost Growth in Achieving 

Readiness Rates:



DODï¿½s Linear Acquisition Approach Makes It Difficult to Control 

Operations and Support Costs:



Chapter 3: Commercial Companies Deliberately Manage Ownership Costs 

through Product Requirements and Design:



A Best Practices Model:



Leading Companies Treat Readiness and Operating and Support Cost as 

Critical Product Requirements:



Knowledge-Based Product Development Is Critical to Achieving Desired 

Reliability and Managing Operating and Support Costs:



Leading Commercial Firms Use Feedback from Operations to Better 

Understand Customer Needs, Product Deficiencies, and Operating and 

Support Costs:



Chapter 4: Stressing Operating and Support Cost at the Outset of an 

Acquisition Could Help DOD Reduce Total Ownership Costs:



Differences in Practices Explain Different Outcomes for Commercial 

Companies and DOD in Controlling Total Ownership Costs:



Several DOD Efforts Underway to Reduce Total Ownership Costs:



DODï¿½s Current Environment Does Not Provide Incentives to Reduce Total 

Ownership Cost Early:



Chapter 5: Conclusions and Recommendations:



Recommendations for Executive Action:



Agency Comments and Our Response:



Related GAO Products:



Appendix I: Comments from the Department of Defense:



Appendix II: GAO Staff Acknowledgments:



Tables:



Table 1: Readiness and Operating and Support Costs for Selected 

Weapons:



Table 2: DOD and Commercial Practices for Controlling Operating and 

Support Costs:



Figures:



Figure 1: Nominal Life-Cycle Cost of Typical DOD Acquisition Program 

with a 30-Year Service Life:



Figure 2: Percent of Operating and Support Costs Determined at Various 

Points in the Acquisition Process:



Figure 3: Readiness, Reliability, and Operating and Support Costs:



Figure 4: DODï¿½s Linear Acquisition Process:



Figure 5: System Readiness Comes at High Operating and Support Costs 

When Reliability Is Not Ensured:



Figure 6: Apache Helicopter:



Figure 7: Abrams Tank:



Figure 8: Commercial Model for Reducing Operating and Support Costs:



Figure 9: Benefits of Ensuring High Reliability Rates During Product 

Development:



Figure 10: Polar Tankerï¿½s Polar Endeavor:



Figure 11: United Airlines/Boeing 777:



Figure 12: FedEx Express Delivery Van:



Figure 13: Joint Strike Fighter:



Figure 14: Advanced Amphibious Assault Vehicle:



This is a work of the U.S. Government and is not subject to copyright 

protection in the United States. It may be reproduced and distributed 

in its entirety without further permission from GAO. It may contain 

copyrighted graphics, images or other materials. Permission from the 

copyright holder may be necessary should you wish to reproduce 

copyrighted materials separately from GAOï¿½s product.



This is a work of the U.S. Government and is not subject to copyright 

protection in the United States. It may be reproduced and distributed

in its entirety without further permission from GAO. It may contain 

copyrighted graphics, images, or other materials. Permission from the 

copyright holder may be necessary should you wish to reproduce 

copyrighted materials separately from GAOï¿½s product.



Letter:



February 11, 2003:



The Honorable John Ensign

Chairman

The Honorable Daniel Akaka 

Ranking Minority Member

Subcommittee on Readiness and Management Support

Committee on Armed Services

United States Senate:



As you requested, this report examines how best practices offer 

improvements to the way the Department of Defense develops new weapon 

systems to reduce their total ownership cost, especially the operating 

and support costs, during design. It examines how the department 

currently designs for operating and support costs and how best 

practices could improve outcomes and reduce costs. We make 

recommendations to the Secretary of Defense for improvements to weapon 

system requirements policy, including establishing operating and 

support cost and readiness goals as performance parameters equal to any 

other performance parameters; revising acquisition policies to require 

a firm estimate of reliability during product development; and 

providing contract incentives for product developers to make 

appropriate trades between reliability and performance before 

production.



We are sending copies of this report to the Secretary of Defense; the 

Secretary of the Army; the Secretary of the Navy; the Secretary of the 

Air Force; the Director of the Office of Management and Budget; the 

Director, Missile Defense Agency; and interested congressional 

committees. We will also make copies available to others upon request. 

In addition, the report will be available at no charge on the GAO Web 

site at http://www.gao.gov.



If you have any questions regarding this report, please call me at 

(202) 512-4841. Other contacts are listed in appendix II.



Katherine V. Schinasi

Director

Acquisition and Sourcing Management:



Signed by Katherine V. Schinasi:



[End of section]



Executive Summary:



Purpose:



For fiscal year 2003, the Department of Defense (DOD) asked for about 

$184.9 billion to develop, procure, operate, and maintain weapon 

systems. With this funding, DOD will have received increases totaling 

about 18 percent since 2001 for what it defines as total ownership 

costs of its equipment. DODï¿½s budget for operations and maintenance 

increased by about 5.6 percent during the same period--from about $59 

billion to about $62.3 billion. DOD has been increasingly concerned 

that the high cost of maintaining weapon systems to meet required 

readiness levels is depleting modernization accounts and denying the 

department the flexibility to invest in new weapons. In recognition of 

this concern, DOD has established goals to reduce operating and support 

costs of weapon systems already in the field as well as those currently 

in development. In order to provide another perspective on this 

problem, GAO has continued a body of work to identify best practices 

used by leading commercial companies to manage new productsï¿½ total 

ownership costs to see if these practices can be applied to DODï¿½s 

weapon system acquisitions.



This report addresses how DOD can use best practices from commercial 

companies during its acquisition process to reduce total ownership 

costs of its major weapon systems. It presents a model of the process 

commercial companies use to incorporate reasonable and manageable 

operating and support cost into their product development process. In 

response to a request from the Chairman and Ranking Minority Member, 

Subcommittee on Readiness and Management Support, Senate Committee on 

Armed Services, GAO (1) determined the practices, processes and metrics 

DOD has historically used to manage and control operating and support 

costs of its major weapon systems; (2) determined the practices, 

processes and metrics commercial companies use to manage and control 

operating and support costs; and (3) analyzed the extent to which 

opportunities exist to apply best practices to reduce operating and 

support costs during product development.



Background:



Readiness is a critical parameter of all DODï¿½s weapon systems. If a 

system is not ready, its performance characteristics are of no use. 

Each weapon system has an expected readiness rate, usually expressed in 

some percentage of available units, that it is expected to maintain for 

our national security. Readiness can be achieved by building highly 

reliable weapon systems or, if the systems are not highly reliable, 

supporting them with an extensive logistics system that can ensure 

spare parts and other support items are available when needed. In 

essence, the cost of a productï¿½s readiness is the cost to develop, 

produce, and operate and maintain that system.



DOD recognizes that the total cost of a weapon system includes more 

than development and procurement costs. Traditionally, development and 

procurement have accounted for about 28 percent of a weaponï¿½s total 

ownership cost, while costs to operate, maintain, and dispose of the 

weapon system account for about 72 percent of the total. For a number 

of years, the departmentï¿½s goal has been to spend less on supporting 

systems and to devote more funds to development and procurement in 

order to modernize weapon systems. But, in fact, growth in operating 

and support costs has limited the departmentï¿½s buying power. DOD 

officials have cited shortages of spare parts and unreliable equipment 

as reasons for low mission-capable rates for some weapons. As a result, 

some modernization has been postponed in order to pay high and 

unexpected operating and maintenance costs.



GAO has issued a series of reports on best practices that commercial 

firms use to manage and control the acquisition cost of their products. 

Commercial firms attain knowledge early in the development process 

about technology that they plan to incorporate; they make sure the 

design is mature before production; and they have production processes 

under control before production begins. The building of knowledge in 

these areas can also contribute to the reduction of a productï¿½s 

operating costs over its life cycle, thereby reducing a major portion 

of its total ownership costs. While those reports focused on best 

practices for reducing the cost and cycle time for acquiring weapon 

systems, this report will focus on what DOD can do during the 

acquisition of a weapon system, prior to and during product 

development, to ensure that products are available when needed, more 

cost effective to operate and support and more reliable once fielded, 

thereby reducing their total ownership cost.



Results in Brief:



DOD weapons acquisition processes do not consider operations and 

support costs and readiness as key performance requirements for new 

weapon systems, and DOD places less emphasis on establishing operations 

and support cost and readiness as key nontradable goals early in 

product development. Generally, the department settles for lower 

reliability in its new weapon systemsï¿½ designs. In our review of data 

for five fielded weapon systems, we found none had established 

operating and support cost or readiness as key requirements. Although 

recent readiness levels were acceptable to the services for the most 

part, the five systems had experienced growth in their operating and 

support cost estimates of between 16 and 48 percent within the last 12 

years and problems with reliability once in the field.



We found several reasons for cost growth. DODï¿½s acquisition process is 

linear, progressing from one organization to the next with little 

interaction among the groups. Requirements-setting by the war-fighting 

community focuses on system performance. DOD policy does not require 

inclusion of readiness or operating and support cost goals as key 

performance requirements equal in importance to other performance 

requirements. None of the systems we reviewed had a readiness or an 

operating and support cost goal as a key requirement. Further, during 

product development, the use of immature technologies and components to 

meet performance goals worked against designing weapon systems with 

high reliability. Using immature technologies also acts as a barrier to 

manufacturing techniques such as open systems[Footnote 1] or designing 

for fewer parts, practices that typically help reduce maintenance costs 

of the system and increase its reliability. DODï¿½s systems for 

accumulating data to analyze operating and maintenance actions on 

weapon systems already in the field do not provide adequate or reliable 

information, thus making it difficult for DOD to understand the total 

cost of operations and support. The outcome of these practices at DOD 

has been an inability to stem the continuous growth in total ownership 

cost, with actual operating costs continuing to exceed initial 

estimates. As a result, DOD continues to request more operating and 

support funding to sustain its systems or to reprogram funds from other 

accounts to pay the bills.



In contrast, commercial companies consider reasonable operating and 

support costs and the readiness or availability of their equipment as 

requirements equal in importance to other performance characteristics, 

thereby ensuring that the developer places proper emphasis on achieving 

reliability and operating and support cost goals during product 

development. Commercial companies have a collaborative process for 

setting requirements, developing the product, and collecting and 

sharing data on maintaining and supporting the product once it is 

delivered. Both the customer and the developer have a voice in the 

process. During product development, especially during the design 

process, the maintainer has an active voice and is armed with 

information about operating and support cost drivers in the previous 

product. Commercial product developers maintain high standards for 

reliability, using proven technologies to achieve critical performance 

requirements. They find an evolutionary development process is critical 

to reducing operating and support costs and achieving high readiness. 

Emphasis is placed on reducing the number of parts in a design so there 

is less to maintain, using standardized parts that are readily 

available in the industrial base and using open systems to maintain 

competition. Once the product is delivered to the customer, maintainers 

keep detailed records on its reliability and the cost of its 

maintenance and support. Importantly, information on the productï¿½s 

performance is communicated back to the developer to be used to improve 

the product.



DOD has implemented initiatives to reduce the total ownership cost of 

its weapon systems. It has modified acquisition policies, established 

programs to reduce operating and maintenance costs in existing systems, 

and selected several acquisition programs to test different approaches 

to reduce life-cycle costs during development. However, these steps do 

not incorporate many of the practices used by commercial companies 

during requirements determination, product development, and fielding. 

In comparing DODï¿½s practices to those found at leading commercial 

firms, we have identified several differences. Because companies 

operate in an environment where operating costs and readiness are 

critical to their survival, commercial customers establish low 

operating and support cost and high readiness requirements when 

purchasing a product. This forces product developers to design reliable 

systems that are easy and relatively cheap to operate and maintain. The 

collaborative relationship between the customer and the product 

developer is essential to driving down operating and support costs. 

Further, companies understand that accurate operating and support cost 

data from current products are also necessary to make good decisions 

related to the purchase of a product, facilitate cost/performance 

trade-offs, and provide feedback to the manufacturer for continuous 

improvement.



GAO is making recommendations to the Secretary of Defense on ways to 

improve DODï¿½s management of operating and support costs. We are 

recommending that DOD revise its requirements generation process to 

include total ownership cost, especially operating and support cost, 

and weapon system readiness rates as performance parameters equal to 

any others. We also recommend that any revision of the current policy 

governing acquisition processes require a firm estimate of the systemsï¿½ 

reliability based on demonstrated reliability rates at component and 

subsystem levels no later than the end of the system integration phase 

and at the system level no later than the production decision. Finally, 

we recommend that DOD structure contracts by Milestone B, the start of 

the system development and demonstration phase, to ensure that proper 

trades are made between reliability and performance before the 

production decision.



Principal Findings:



DODï¿½S Current Practices for Setting Requirements and Developing New 

Weapon Systems Continue to Yield Higher Total Ownership Costs:



DOD is spending more on operating and support costs for its weapon 

systems than it planned. We found three primary reasons for the high 

cost of operating and supporting DODï¿½s fielded weapon systems. These 

were (1) little or no attention to the trade-offs between readiness 

goals and the cost of achieving them when setting the key parameters 

for weapon systems;[Footnote 2] (2) the use of immature technologies 

during product development and delays in acquiring knowledge about the 

design and its reliability until late in development, or in some cases, 

production; and (3) insufficient data on the operations and maintenance 

costs and actions for fielded systems that would allow improvements in 

products currently in development. The outcome of these practices in 

DOD has been an inability to stem continuous growth in total ownership 

cost, with actual operating costs continuing to exceed initial 

estimates. As a result, DOD continues to request more operating and 

support funding to sustain its systems or reprogram funds from other 

accounts to pay the bills.



Even though operating and support costs are the largest factor in a 

weapon systemï¿½s total ownership cost, they do not receive the same 

attention when requirements are set for a weapon system as other 

performance characteristics. In our review of data for five fielded 

weapon systems, we found that none had established an operating and 

support cost or a readiness goal as a key requirement prior to product 

development. In fact, operating and support cost estimates were not 

available in the Selected Acquisition Reports until at least 5 years 

into product development on these programs. Most of the fielded systems 

we reviewed were near or achieving readiness goals, but had experienced 

significant cost growth in operations and support cost estimates within 

the last 12 years to do so. Two of these systems, the Apache and the 

Abrams, were designated as the Armyï¿½s most expensive weapons to 

support. The C-17 reported a cost increase of almost 25 percent, and 

program officials stated that they would not have a firm estimate of 

operating and support cost until 2010--more than 25 years after the 

start of development.



We found practices in three areas--requirements-setting, product 

development, and operations and maintenance--that contributed to this 

condition. DODï¿½ s acquisition process is linear and serially involves 

several organizations whose responsibilities in the process have 

differing objectives. Communication among the different organizations 

is fragmented. Requirements focused on the weapon systemï¿½s performance 

characteristics. Once the weapon systemï¿½s requirements were set and the 

development of the system began, product development focused on 

achieving the programï¿½s acquisition cost, schedule and performance 

goals, rather than on increasing reliability in order to reduce its 

total life-cycle costs. We found that the maintainers had limited 

involvement in making design trades for lower operating and support 

costs during development and that best practices such as designing for 

open systems or ease of maintenance were not used by the developer. We 

also found that once a system is fielded, the servicesï¿½ systems for 

tracking operating and support costs were suspect, providing inadequate 

feedback to suppliers and requirements setters.



Commercial Companies Deliberately Manage Ownership Costs through 

Product Requirements and Design Process:



We found that commercial companies that use capital equipment 

considered operating and support costs integral to their new product 

development decisions. Companies such as United Airlines, FedEx 

Express, and Polar Tanker employ practices to maintain the readiness of 

their fleets at as low an operating cost as possible. Reducing these 

costs translates into revenues, profits, and market growth. Increases 

to these costs can mean market failure. As customers, they have 

established operating and support costs and product readiness as key 

system requirements before development begins for a new product that 

are equal in importance to requirements for its performance and the 

cost to develop and produce the product. For example, United Airlines 

requires that new aircraft maintain a readiness rate of 98.5 percent or 

the manufacturer must reimburse it for lost revenue. Polar Tanker 

established a requirement that its Endeavor Class tanker operate at 

least 330 days a year at a reduced operating cost per tanker. These 

dual requirements drove trades during design, sometimes increasing 

development costs to achieve lower operating costs. Before FedEx 

Express agreed to a new design for its delivery trucks, it required 

that the new design last for at least 300,000 miles over a specified 

number of years and at a specific cost per mile. In gaining agreement 

with product developers on these requirements prior to product 

development, these companies sometimes had to trade performance or 

spend more in development, but they received more reliable products, 

reduced total ownership costs, and made those costs more predictable.



To meet their customersï¿½ supportability requirements, we found that 

commercial product developers focused on designing a product that was 

easy to maintain, would be ready when needed, and reliable at low cost. 

They used an evolutionary development process. Consequently, they did 

not allow components or subsystems into a productï¿½s design unless the 

technology had been proven reliable through past use or testing. For 

example, Boeing told us that it defers use of immature technology to 

later evolutions of design and makes the reliability ratings of its 

components available to the airlines before it begins product 

development. Maytag completed reliability testing on every new product 

prior to going to production. These companies also emphasized product 

designs with fewer parts and open systems. Maytag has established a 

parts reduction program as part of its development process, and Boeing 

built its 777 so that any of three engines--GE, Pratt & Whitney, or 

Rolls Royce--would fit. Developers also gained insight into design 

features their customers valued through regular communication with 

them. For example, the design for Boeingï¿½s latest generation 737--

geared toward reducing operating costs--was inspired by the airlines. 

Boeing emphasized open systems, standardized parts, and reduced parts 

from one generation to the next to satisfy the airlinesï¿½ need for 

reduced operating costs.



All of the companies we visited, customers and product developers 

alike, had very collaborative processes and practices for drawing 

extensively on data from past operations to influence the design of new 

products. This information was used as a baseline for new product 

designs and was used to estimate the operating costs of new products. 

United Airlines officials told us that the airlines and the 

manufacturers both keep meticulous reliability and cost records at all 

levels of an airplane--components, subsystems, and at the system level. 

Major operating costs drivers are tracked on a daily basis by the 

airlines, and the manufacturers usually have personnel residing with 

the airlinesï¿½ maintenance crew to help solve problems on the spot and, 

just as importantly, to feed information back to the manufacturer so 

that the next product can be improved. FedEx Express and Polar Tanker 

both emphasized extensive data collection from current operations. In 

fact, FedEx Express sets annual targets for operating and support cost 

reductions based on data gathered on the road. Polar Tanker gathered 

maintenance data from past operations and established a team made up of 

its own maintenance personnel and outside consultants to determine 

areas that could result in higher reliability and lower maintenance 

costs in designing the new Endeavor Class tanker.



Greater Emphasis on Operating and Support Cost at the Outset and during 

an Acquisition Program Could Help DOD Reduce Total Ownership Costs:



DOD and the commercial companies we visited have policy goals of 

developing products that will meet customersï¿½ needs at the lowest 

possible cost to build and operate. The difference between them is in 

how each implements its policies. Leading commercial companies follow 

an integrated, collaborative process of setting requirements, 

developing the product, and ensuring that the product can be supported 

at an acceptable cost. DODï¿½s process is composed of disparate practices 

carried out by separate organizations with differing objectives and 

little communication among them about how to support fielded systems. 

While commercial firms focus on total ownership costs at the outset, 

DOD focuses mostly on technical performance. One cause of this is that 

in DOD the accountability and responsibility for total ownership costs 

are spread across many organizations with separate goals. Another cause 

lies in motivation for low costs. The commercial companies we visited 

are driven by the need to be as profitable as possible to survive, and 

low total ownership costs translate to higher profitability. DODï¿½s 

environment does not provide such incentives, and the organizations 

charged with acquiring and operating weapon systems are unconstrained 

by a need to lower costs since they can request additional operations 

and maintenance funding to keep systems working.



Some of the practices used by commercial companies to reduce a new 

productï¿½s operating costs during its development may be helpful to DOD. 

In setting requirements, commercial customers make readiness and 

operating cost requirements and collaborate directly with the product 

developer. Product developers establish sound cost estimates early; 

designs are simplified by using open systems and reducing parts; 

reliability testing is done early; and a reliability growth curve is 

established before production begins. Once a product is fielded, 

operating costs are managed to established targets; operating cost data 

is collected, analyzed, and used by the developer and the customer to 

develop more reliable products in the future; and continuous 

improvements are made to future products. The commercial practice of 

establishing readiness and operating cost as key requirements for a new 

product necessitates substantive input from operators and developers 

before and during product development. The commercial practices used 

during product development to design reliable systems that are easy and 

less costly to operate and maintain depend on the use of good product 

development practices including the use of mature technologies to meet 

requirements. Commercial firms use incremental product development 

processes and depend on a strong relationship between the manufacturer 

and the customerï¿½s operators and maintainers to continue throughout 

product development.



DOD does not focus on operating and support costs to the degree 

commercial companies do. In setting requirements, DOD does not make 

readiness and operating cost key parameters, performance is rarely 

reduced in favor of reliability or reduced operating cost, and there is 

no direct relationship between the requirement setters and the product 

developer. During product development, firm estimates of operating 

costs are not required, little attention is paid to reliability rates, 

and open systems or design for manufacturing techniques are rarely 

used. Once a weapon system is fielded, there is a lack of complete and 

reliable data available from the field, and there is little 

collaboration between maintainers and product developers to improve new 

systems. DODï¿½s acquisitions usually begin with critical technologies 

that are immature, with unproven reliability. This makes it difficult 

to implement best practices such as design for manufacturing during 

product development. Accurate operating and support cost data are not 

available for helping management make good decisions, facilitating 

cost/performance trade-off decisions, and providing feedback to the 

manufacturer for continuous improvement. On the weapon systems we 

reviewed, we found that the programs had poor initial estimates of 

operating and support costs for weapon systems, partly because they do 

not have reliable systems in place to track those costs per weapon 

system.



DOD has taken some steps to lower its weapon systemsï¿½ total ownership 

costs. Those actions include concurring with and implementing 

recommendations concerning the use of technology readiness levels, 

indicators of design maturity, and controlled production processes. 

Further, the department initiated pilot programs with 30 acquisition 

programs to develop methods for reducing total ownership costs. 

However, DODï¿½s current environment--both culturally and 

organizationally--is not presently conducive to applying them. 

Currently, its acquisition policies do not provide specific guidance 

for controlling total ownership cost and its requirements-generation 

policies provide no guidance for establishing readiness or cost goals 

for weapon systems once they are fielded.



Recommendations for Executive Action:



GAO recommends that the Secretary of Defense:



* revise the Chairman of the Joint Chiefs of Staff Instruction 3170.01B 

on the requirements generation process to include total ownership cost, 

especially operating and support cost, and weapon system readiness 

rates as performance parameters equal in priority to any other 

performance parameters for any major weapon system prior to beginning 

the acquisition program;



* revise the current policy governing the operation of the defense 

acquisition system (currently under revision) to require that the 

product developer establish a firm estimate of a weapon systemï¿½s 

reliability based on demonstrated reliability rates at the component 

and subsystem level no later than the end of the system integration 

phase, coinciding with the system-level critical design review, before 

proceeding into the system demonstration phase of product development; 

and at the system level no later than the full-rate production 

decision; and:



* structure DOD contracts for major systems acquisitions so that at 

Milestone B the product developer has incentives to ensure that proper 

trades are made between reliability and performance prior to the 

production decision. One option is to provide specific clauses in the 

development contract to address reliability growth.



Agency Comments:



DOD partially concurred with all of our recommendations but, for the 

most part, found no further action was needed to lower total ownership 

cost. We disagree. We believe that if DOD takes no further action in 

implementing these recommendations, it would ignore significant 

opportunities to improve readiness by lowering total ownership cost in 

a budget environment that demands more effort to reduce these costs.



A detailed discussion of DODï¿½s comments appears in Chapter 5 and the 

full text of DODï¿½s comments is in appendix I.



[End of section]



Chapter 1: Introduction:



For fiscal year 2003, DOD asked for $184.9 billion to fund research and 

development, procurement and direct operations and maintenance costs of 

its weapon systems. These elements along with disposal costs are 

defined as the total ownership cost of a weapon system. The budget has 

increased by about 18 percent for these activities since 2001--with 

direct cost for operations and maintenance of weapon systems increasing 

by about 5.6 percent from $59 billion to about $62.3 billion. Since the 

late 1990ï¿½s, DOD has been increasingly concerned that the cost of 

operating and supporting weapon systems to meet required readiness 

levels is depleting its modernization accounts and denying the 

department the flexibility to invest in new weapons.



Total Ownership Cost Is the Cost to Ensure Readiness:



Commercial companies and DOD both use readiness as a key indicator of a 

productï¿½s success. Readiness is usually expressed as the percentage of 

total units available and capable of performing a mission at any given 

time. If a weapon system is not ready when it is needed, its 

performance characteristics are of no use. In general, readiness can be 

achieved either by building highly reliable weapon systems or, if the 

systems are not highly reliable, supporting them with an extensive 

logistics system that can ensure spare parts and other support items 

are available when needed. In essence, the cost to ensure a productï¿½s 

readiness is the cost to develop, produce, operate, and maintain that 

product through its life cycle.



The development and production cost of a weapon system, also known 

as acquisition cost, usually represents about 28 percent of the weapon 

systemï¿½s total ownership costs. The acquisition cost is funded through 

DODï¿½s research, development, test and evaluation, and procurement 

accounts. These funds are used to mature new technology and design and 

manufacture new weapon systems. Operating and support costs[Footnote 3] 

are typically the highest portion of a weapon systemï¿½s total ownership 

cost because they represent the cost to operate the system and keep it 

ready for action over many years, sometimes more than 30 years. These 

costs are about 72 percent of the total ownership cost of a weapon 

system and are funded through DODï¿½s operations and maintenance account. 

Operating and support costs reflect the purchases of fuel, lubricants, 

and repair parts and their associated maintenance as well as 

modification kit procurement and installation. Figure 1 depicts the 

typical distribution of total ownership costs of DOD weapon systems 

over a 30-year life cycle.



Figure 1: Nominal Life-Cycle Cost of Typical DOD Acquisition Program 

with a 30-Year Service Life:



[See PDF for image]



[End of figure]



Figure 1 shows that the greater part of a weapon systemï¿½s total 

ownership cost is made up of its operating and support cost. While 72 

percent of the life-cycle cost of a weapon system is realized only 

after it is fielded, the decisions made during its acquisition--when 

its performance requirements are being established and its design is 

being matured--will dictate operating and support costs very early. In 

fact, studies show that about 85 percent of the operating and support 

costs of a weapon system will be determined as soon as requirements are 

set, while less than 10 percent of the life-cycle cost have been spent. 

By the time a product is ready for production, over 90 percent of the 

operating and support costs have been determined, and about 28 percent 

of the total life-cycle costs have been spent. Figure 2 illustrates 

this phenomenon.



Figure 2: Percent of Operating and Support Costs Determined at Various 

Points in the Acquisition Process:



[See PDF for image]



[End of figure]



Because so much of the eventual cost to support and maintain a weapon 

system is decided very early, it makes sense that more attention should 

be paid to supportability when the productï¿½s requirements are being set 

and its design is being finalized. World-class commercial companies 

that either use or develop high-performing products know this and set 

requirements and designs accordingly.



Commercial Best Practices:



GAO has undertaken an extensive body of work that examines DODï¿½s 

acquisition issues from a different, more cross-cutting perspective--

one which draws upon the lessons learned from the best commercial 

practices to see if they are applicable for DODï¿½s acquisition 

processes. Previous GAO best practices reports focused on what DOD 

could do to control product development costs that represent about 28 

percent of total ownership costs. This report will focus on best 

practices for reducing the largest segment of total ownership costs--

the operating and support costs. The concepts discussed build on our 

previous reports that looked at earlier phases of an acquisition, 

including matching customer needs with available resources and 

designing and manufacturing products within cost, schedule, and 

performance goals. A complete list of best practices reports is at the 

end of this report:



Leading commercial companies expect to obtain high-quality products 

that meet their expectations in terms of performance, price, and 

reliability. To ensure they make prudent buying decisions, they use a 

structured product development process that ensures a high level of 

knowledge exists about a product at key junctures during its 

development. Such a knowledge-based process enables decision makers to 

be reasonably certain that product quality, reliability, and timeliness 

are assured.



* Knowledge point 1 occurs when a match is made between the customerï¿½s 

needs and the available resources--technology, design, time, and 

funding. Commercial companies use this knowledge to meet essential 

product requirements, such as low operating and support costs. To 

ensure that the knowledge is attained, private companies require the 

product be demonstrated in its intended environment. In addition, the 

product developer must complete a preliminary product design, using 

systems engineering to balance customer desires with available 

resources.



* Knowledge point 2 occurs when the productï¿½s design demonstrates its 

ability to meet performance requirements. Program officials are 

confident that the design is stable and will perform acceptably when at 

least 90 percent of engineering drawings are complete. To obtain this 

knowledge, commercial companies use simulations and testing to fully 

understand how the product should be built.



* Knowledge point 3 occurs when the product can be manufactured within 

cost, schedule, and quality targets and is reliable. Important 

indicators of this are when critical manufacturing processes are in 

control and consistently producing items within quality standards and 

tolerances. Private companies demand these critical manufacturing 

processes be in control because they could affect the product 

reliability.



Objectives, Scope, and Methodology:



The Chairman and the Ranking Minority Member, Subcommittee on Readiness 

and Management Support, Senate Committee on Armed Services, requested 

that we examine best practices for reducing total ownership cost of 

DODï¿½s weapon systems. This report primarily covers the operating and 

support cost portion of total ownership costs. Our overall objective 

was to determine whether commercial best practices for reducing total 

ownership costs, particularly operating and support costs, prior to and 

during the acquisition of weapon systems offer opportunities to improve 

outcomes in DODï¿½s acquisitions and its efforts to manage and reduce 

total ownership costs. Specifically, we (1) determined the practices, 

processes, and metrics DOD uses to manage and control total ownership 

costs of its major weapon systems; (2) determined the practices, 

processes, and metrics commercial companies use to manage and control 

total ownership cost; and (3) analyzed the extent to which 

opportunities exist for DOD to apply best practices to reduce operating 

and maintenance costs during product development.



We used case studies of the following six DOD weapons, chosen to 

reflect all of the services across time, to examine DODï¿½s practices, 

processes and metrics:



The Advanced Amphibious Assault Vehicle: This vehicle is the U.S. 

Marine Corpsï¿½ replacement for its presently fielded amphibious assault 

vehicle. The new development vehicle is equipped with a 30mm automatic 

cannon and provides the capability to transport a Marine rifle squad at 

a speed of 20-25 knots in the water, and cross country with the agility 

and mobility equal to or greater than that of the M-1 tank. The 

contract for the Product Definition/Risk Reduction Phase was awarded in 

1996. The Marine Corps expects to buy 1,013 of these vehicles. 

Acquisition costs total $9.6 billion; operating and support costs, 

$16.0 billion.



The Joint Strike Fighter program: This aircraft is the next-generation 

aircraft for the Navy, Air Force, Marine Corps, and Allies. There are 

three variants: a carrier variant will provide the Navy a multi-role, 

stealthy aircraft to complement the F/A-18E/F. The Air Force variant 

will be a multi-role aircraft, but primarily used in an air-to-ground 

role to replace the F-16 and the A-10 and complement the F-22. The 

Marine Corps variant will be a short-takeoff and vertical-landing 

aircraft to replace the Sea Harrier and GR-7 for the United Kingdom 

Royal Navy and Royal Air Force. The program is currently in system 

development and demonstration. Acquisition costs total $226.5 billion; 

operating and support costs, $387.6 billion.



The Landing Platform Dock 17 ship program: These ships are used for 

transporting amphibious assault vehicles and other cargo. They 

incorporate both a flight deck for helicopters and a well deck to 

support landing craft. The contract for the detail design, integration 

and construction of the ship was awarded in 1996. The Navy expects to 

buy 12 ships to replace 27 older amphibious ships. Acquisition costs 

total $15.4 billion; operating and support costs, $56.5 billion.



The Apache helicopter program: This Army helicopterï¿½s mission is to 

find tanks and other armored targets and destroy them with its laser-

guided Hellfire missiles, its 2.75-inch rockets, or its 30-millimeter 

gun. Apache development began in 1973 and the helicopter entered 

production in 1982 and was fielded in 1984. The Longbow Apache is a 

remanufactured and upgraded version of the Apache, which includes 

improved radar, engine and Hellfire missiles. The Army currently fields 

741 Apache and Longbow Apache helicopters. Acquisition costs total $ 19 

billion; operating and support costs for the Longbow Apache are $11.1 

billion.[Footnote 4]



The C-17 cargo aircraft: The C-17 is a multi-engine, cargo aircraft 

expected to improve Air Force capability to rapidly deploy, reinforce, 

and sustain combat forces worldwide. The C-17 is capable of carrying 

outsized cargo over extended distances into unimproved airfields. The 

C-17 introduces a direct deployment capability that significantly 

improves airlift responsiveness. Development began in 1982 and the 

aircraft entered production in 1988 and was fielded in 1993. As of 

December 2002, the Air Force fielded 96 of 180 aircraft. Acquisition 

costs total $58.2 billion; operating and support costs, $144.9 billion.



The M-1 series Abrams tank program: This tank is the Armyï¿½s main battle 

tank. The M-1ï¿½s development began in November 1972, entered production 

in 1979, and was fielded in 1986. The M1A2 is an improved version of 

the M-1, with improved armor, digital electronics, and an improved 

commanderï¿½s weapon station. The Army has fielded over 8,800 M-1 and 

M1A1 tanks, with about 1,000 upgraded to the M1A2 versions. Acquisition 

costs total $29 billion; operating and support costs, $16 

billion.[Footnote 5]



For each of the six programs, we interviewed key managers and logistics 

representatives to discuss how operating and support costs were being 

managed and controlled during design. For the Joint Strike Fighter 

program we also visited the prime contractor, Lockheed Martin Aircraft 

Company, in Ft. Worth, Texas, and interviewed key designers and 

engineers. We analyzed operating and support cost data contained in 

Selected Acquisition Reports for the Apache Longbow helicopter, the 

C-17 cargo aircraft, the Abrams tank, the Advanced Amphibious Assault 

vehicle, Joint Strike Fighter and Landing Platform Dock 17. Information 

obtained from the Selected Acquisition Reports was not always 

consistent because three of the systems--the Advanced Amphibious 

Assault Vehicle, Joint Strike Fighter and Landing Platform Dock 17--are 

still in development. We supplemented information for the development 

systems with other program cost estimates. We also conducted limited 

analysis of the B-1, B-2, and F-22 operating and support cost and 

readiness data, based on information provided by their respective 

program offices and previous GAO reports.



In addition to the case studies, we reviewed DOD policy, describing the 

roles and responsibilities of various organizations in the requirements 

development process. We discussed the implementation of these policies, 

particularly the role of the logistics community in the requirements 

determination process, with officials from each of the six program 

offices listed above, as well as appropriate officials from the:



Under Secretary of Defense (Acquisition, Technology, and Logistics);

Air Mobility Command, Scott Air Force Base, Ill.;

437th Airlift Wing, Charleston Air Force Base, S.C.;

Office of the Assistant Secretary of the Army, Washington, D.C.; and

U.S. Army Aviation Center, Ft. Rucker, Ala.



To determine the best practices, processes, and metrics commercial 

companies use to manage and control operating and support costs, we 

used a case study methodology by judgmentally selecting companies based 

upon general literature searches and discussions with experts. On this 

basis, we identified a number of commercial companies that have a 

structured and defined process for managing and controlling their 

operating and support costs. The following are descriptions of the six 

commercial companies and one quasi-governmental agency we visited:



Boeing Commercial Aircraft designs and manufactures commercial 

airplanes. In 2001, it reported revenues totaling $58.2 billion. We 

visited its offices in Seattle, Washington, and discussed the 

development of the 737, the 767, and the 777 aircraft.



Chicago Transit Authority is a quasi-governmental agency that operates 

the nationï¿½s second largest public transportation system and covers the 

City of Chicago and 38 surrounding suburbs. In 2002, it reported an 

operating budget totaling $915 million. We visited its offices in 

Chicago, Illinois, and discussed the requirements determination 

process for acquiring buses.



Allison Transmission, a division of General Motors, designs and 

manufactures transmissions for medium and large vehicles as well as 

military vehicles. In 2001, General Motors reported sales and revenues 

totaling $177.3 billion. We visited its offices in Indianapolis, 

Indiana, and discussed the development process for new transmissions.



FedEx Express delivers packages, freight, and information to its 

customers worldwide. In 2001, it reported sales and revenues totaling 

$15.5 billion. We visited its offices in Memphis, Tennessee and 

discussed their process for setting requirements for a new package 

delivery vehicle.



Maytag designs and manufactures major home appliances. In 2001, it 

reported sales and revenues totaling $4.1 billion. We visited its 

offices in Newton, Iowa, and discussed the development process for new 

appliances, particularly the Neptune washer and the Wide-By-Side 

refrigerator.



Polar Tanker, a shipping division of ConocoPhillips Marine, manages the 

marine transportation of ConocoPhillipsï¿½ Alaska North Slope crude oil 

production. In 2001, it reported sales and revenues totaling $26.9 

billion. We visited its offices in Avondale, Louisiana, and discussed 

the development process for the new Endeavor Class tanker, the Polar 

Endeavor.



United Airlines, a commercial airline division for the UAL Corporation, 

is a major commercial air transportation company, engaged in the 

transportation of persons, property, and mail throughout the U.S. and 

abroad. In 2001, it reported sales and revenues totaling $16.1 billion. 

We visited its offices in Seattle, Washington; and San Francisco, 

California; and discussed the product development process and 

maintenance activities for the Boeing 777 aircraft.



At each of these organizations, we conducted structured interviews with 

representatives to gather uniform and consistent information about 

their processes, practices, and metrics for controlling operating and 

support costs. During these meetings, we obtained a detailed 

description of the practices and processes they believe are necessary 

and vital to control and reduce operating and support costs. We met 

with managers of reliability, maintainability, and new aircraft 

development; general directors of operations; controllers; directors of 

configuration and integration; and principal engineers. We did not use 

examples from Chicago Transit or from Allison Transmissions, but 

discussions with those firms helped to refine the commercial model and 

inform our analysis of commercial best practices.



During the past 5 years, we have also gathered information on operating 

and support costs from such companies as 3M, Chrysler Corporation, 

Caterpillar, Bombardier Aerospace, Ford Motor Company, Hughes Space and 

Communications, and Motorola Corporation. This information enabled us 

to develop an overall model to describe the general practices, 

processes, and metrics leading commercial companies take to control 

operating and support costs.



This report highlights several best practices in controlling operating 

and support costs based on our fieldwork. As such, they are not 

intended to describe all practices or suggest that commercial companies 

are without flaws. Representatives from the commercial companies 

visited told us that the practices and processes, which we considered 

best practices, evolved over many years and that the practices continue 

to be improved based on lessons learned and new ideas and information. 

They admit that the application and the use of these practices have not 

always been consistent or without error. However, they strongly 

suggested that the probability of success in controlling operating and 

support costs is greatly enhanced by the use of these practices and 

processes.



We conducted our review between August 2001 and February 2003 in 

accordance with generally accepted government auditing standards.



[End of section]



Chapter 2: DODï¿½S Requirements-Setting and Development Practices Yield 

Higher Total Ownership Costs:



DOD is spending more on operating and support costs for its weapon 

systems than it planned. We found three primary reasons for the high 

cost of operating and supporting DODï¿½s fielded weapon systems. These 

were (1) little or no attention to the trade-offs between readiness 

goals and the cost of achieving them when setting the key parameters 

for weapon systems; (2) the use of immature technologies during product 

development and delays in acquiring knowledge about the design and its 

reliability until late in development, or in some cases, production; 

and (3) insufficient data on the operations and maintenance costs and 

actions for fielded systems that would allow improvements in products 

currently in development. The outcome of these practices in DOD has 

been an inability to stem continuous growth in total ownership cost, 

with actual operating costs continuing to exceed initial estimates. As 

a result, DOD continues to request more operating and support funding 

to sustain its systems or reprogram funds from other accounts to pay 

the bills.



Three key groups are involved in DODï¿½s process to get a weapon system 

to the war fighter. First, the war fighterï¿½s service-based requirements 

community establishes requirements for a new system. Second, the 

service-based acquisition organizations design and produce a product. 

Finally, after the product is developed and produced, it is turned over 

to the war fighterï¿½s operating and maintenance communities, who have 

the responsibility to operate and maintain it. Decisions made in 

setting requirements very early in product development have the most 

impact on the subsequent costs of supporting a system. Trade-offs 

during the design process can also be significant. In DOD, the focus in 

the requirements and development process is to establish and meet 

technical war-fighting performance capabilities, and when trade-offs 

are made, they are usually to optimize those capabilities. The 

maintainers often come into this process very late and have little 

influence. At the end, DOD has no alternative but to pay the operating 

and support bills that accrue in order to maintain readiness.



DODï¿½s Weapon System Programs Encounter Cost Growth in Achieving 

Readiness Rates:



A weapon systemï¿½s operating and support cost will depend to a great 

extent on its performance characteristics, expected readiness rate, and 

the overall reliability of its design. If a weapon system has a very 

high expected-readiness rate but its design is not reliable, its 

operating and support costs will be high and unpredictable. Conversely, 

if the design has been thoroughly tested for reliability and is robust, 

the cost to operate and support it will be lower and more predictable. 

Ideally, there is a balance that customers and product developers can 

strike between readiness and operating cost. Figure 3 illustrates this 

balance.



Figure 3: Readiness, Reliability, and Operating and Support Costs:



[See PDF for image]



[End of figure]



A product developer can opt to drive higher reliability into the 

product during its development by reducing technical requirements, 

using highly reliable and proven components, or investing more in early 

testing. Those decisions would increase the productï¿½s reliability and 

consequently improve prospects for readiness and reduce operating costs 

across the life cycle of the product.



We reviewed five weapon system programs currently in the field and 

found that most had experienced significant growth in their estimated 

operating and support cost. We also found that none of the programs 

established goals for readiness or operating and support cost as key 

parameters for the weapon system prior to launching the acquisition 

program. In addition, we found that once fielded, some systems were not 

achieving the readiness rates that program officials thought were 

possible during development. Table 1 shows whether systems had 

specified readiness and operating and support cost goals as key 

requirements as well as the growth in operating and support costs that 

the systems have experienced within the last 12 years.



Table 1: Readiness and Operating and Support Costs for Selected 

Weapons:



Weapon: Apache; System readiness 

as a key parameter: No; Operating and support cost 

goals as a key parameter: No; Readiness: planned/actual: 62/73 %; 

Percentage growth for 

operating and support cost: 48 %.



Weapon: Abrams; System readiness 

as a key parameter: No; Operating and support cost 

goals as a key parameter: No; Readiness: planned/actual: 90/93    ; 

Percentage growth for 

operating and support cost: 24   .



Weapon: C-17; System readiness 

as a key parameter: No; Operating and support cost 

goals as a key parameter: No; Readiness: planned/actual: 92/84    ; 

Percentage growth for 

operating and support cost: 25   .



Weapon: B-2; System readiness 

as a key parameter: No; Operating and support cost 

goals as a key parameter: No; Readiness: planned/actual: 70/42    ; 

Percentage growth for 

operating and support cost: 35   .



Weapon: B-1; System readiness 

as a key parameter: No; Operating and support cost 

goals as a key parameter: No; Readiness: planned/actual: 67/64    ; 

Percentage growth for 

operating and support cost: 16   .



Source: DOD (data), GAO (analysis).



[End of table]



While some systems have maintained their expected readiness rates, 

they have experienced between 16 and 48 percent growth in estimated 

operating and support cost. It is reasonable to conclude that the 

systems have not achieved the reliability rates that were needed to 

meet their expected readiness goals and, as a result, had to expend 

more funds on parts and labor in the field than were planned for 

maintenance. Two of the systems, the Apache and the Abrams, were 

designated as the Armyï¿½s most expensive weapons to support. The C-17 

has already experienced a cost growth of 25 percent in its operating 

costs, but program officials stated that they would not have a firm 

estimate of operating and support costs until 2010--more than 25 years 

after the start of development.



DODï¿½s Linear Acquisition Approach Makes It Difficult to Control 

Operations and Support Costs:



Traditionally, DOD has used a linear approach to setting requirements 

and developing a product. It focuses attention during product 

development on achieving revolutionary performance goals while trying 

to keep acquisition costs for a program as low as possible. Often, it 

is not until the system is fielded and responsibility shifts to other 

agencies or the services that the operating and support costs become an 

overriding concern. By this time, there is no alternative but to pay 

the bills that accrue to maintain readiness, no matter the cost.



Three key groups are involved in DODï¿½s process to get a weapon system 

to the war fighter. First, requirements representatives from the war-

fighting community establish a need for a new system, and the combat 

developers formulate a set of operating performance requirements to 

address the need. Requirements concerning how available the system must 

be and the cost to operate and support it are not considered key 

performance parameters. Second, DOD establishes an acquisition program 

office to begin product development and coordinate design development 

and production with the defense contractor. Often, the technologies and 

components needed to achieve tough performance requirements are new and 

unreliable; however, the program manager is responsible for developing 

and producing the weapon system within certain acquisition costs and 

schedule guidelines. Third, after the product is developed and 

produced, the war fighter has the responsibility to operate and 

maintain it. Although decisions made in setting requirements very early 

in product development have the most impact on the cost to support the 

system, the personnel who maintain it have less influence on the 

product development process because the focus is on achieving difficult 

performance requirements.



Figure 4 briefly describes DODï¿½s process for managing a new weapon 

systemï¿½s requirements development, acquisition, and fielding. The most 

notable aspect of the model is that there is little communication or 

input from the maintenance community early in the process--during 

requirement setting and product development--when decisions will be 

made that will significantly influence the cost to maintain and support 

the weapon system. The model shows that, traditionally, DODï¿½s processes 

are separate and independent of each other. First, requirements are set 

independently of the maintainers and the product developers. Second, 

once the product development process begins, the focus is on maturing 

technologies and achieving a producible design that will meet the 

technical performance requirements for the weapon system. Finally, the 

operator is tasked to use and maintain the system that has been 

developed and produced.



Figure 4: DODï¿½s Linear Acquisition Process:



[See PDF for image]



[End of figure]



The goal of this process is to field a high-performing weapon system 

that will satisfy the needs of the war fighter better than any other 

weapon system available. We previously reported that competition for 

limited funding both within and among the services leads to performance 

requirements that will make the particular weapon system stand out from 

existing or alternative systems. [Footnote 6] Those that provide the 

greatest leap forward in promised performance stand the best chance of 

winning the funding. As a result, the design for the weapon system is 

usually based on undemonstrated components and subsystems that, when 

integrated into a weapon system, have low reliability and, ultimately, 

high operating and maintenance costs over their lifetime. Figure 5 

illustrates this phenomenon. The demonstrated reliability of the new 

system is lower, causing an imbalance in the relationship between 

readiness and operating cost toward the need for higher costs to 

maintain readiness.



Figure 5: System Readiness Comes at High Operating and Support Costs 

When Reliability Is Not Ensured:



[See PDF for image]



[End of figure]



Logistics officials at Air Mobility Command told us that even though 

they are represented in the requirements determination process for new 

weapon systems, they view their role as providing input on how the 

logistics community could support performance requirements of a new 

system, not on forcing trade-offs that would reduce operating and 

support costs. The officials also said that the logistics community 

leads many reliability improvement and cost reduction initiatives once 

aircraft are fielded. However, this approach is harder and more costly 

to implement than if reliability and operating and support costs had 

been considered when requirements were set and product development 

began. Further, the program must compete against other programs for 

operating and support funds to implement the upgrades and improvement 

initiatives.



Operating and Support Cost Estimates Are Provided Later in DODï¿½s 

Process:



Supportability aspects of system performance, such as operating the 

system at the lowest possible cost and the percentage of time the 

system has to be available for operations, are not given the same high 

priority as enhanced performance. Operating and support costs are not 

estimated until much later in development. For example, the B-2 bomber, 

Apache helicopter, and Abrams tank programs did not publish an initial 

estimate for the operating and support costs for those weapon systems 

in the Selected Acquisition Reports until more than 15 years after the 

start of development. The C-17 program did not publish an estimate 

until more than 7 years after development began. These estimates were 

not in the form of goals or key performance parameters. Throughout 

product development, then, design goals for reliability are not 

supported by the war fightersï¿½ need to meet operating cost and 

availability goals.



Maturing the Technology Is the Focus of Development:



In DOD, requirements for new weapon systems are usually based on the 

enhanced performance deemed necessary to achieve a certain war-fighting 

mission with little hesitancy in using new technology or what the cost 

to support it may be. The user representatives define system 

performance with limited input from the product developers and 

maintainers.



A case in point is the B-2 bomber. The low-observable requirement for 

the B-2 bomber could only be met with technology that was immature at 

program launch, and this technology continued to cause problems 

throughout development, production, and fielding. The B-2 program began 

full-scale development in 1981, and the Air Force began low-rate 

initial production concurrently with development in November 1987. By 

1991, problems with the B-2ï¿½s low-observable material were still being 

reported, causing delays in delivery and cost increases because the 

material was difficult to manufacture. Once fielded, the low-observable 

materials required very high maintenance. The processes to repair them 

were time consuming and required an environmentally controlled repair 

facility. Poor durability and extensive maintenance kept the aircraft 

from achieving its planned availability. All of these factors are due 

to decisions to proceed with product development without understanding 

this technology.



A more recent example is the F-22 fighter. The requirements for the 

F-22 were very demanding. Performance characteristics included low 

observability, super cruise speed, and fused avionics. These 

requirements caused the product design to include many new and unproven 

technologies. During product development, the program planned to 

achieve a system-level reliability rate for the F-22 of 3 flying hours 

mean time between maintenance actions when fully mature. The Air Force 

had estimated that in late 2001, when the F-22 entered limited 

production, it should have been able to demonstrate almost 2 flying 

hours between maintenance actions. However, when the F-22 actually 

began limited production, it could only fly an average of .44 hours 

between maintenance actions. In other words, the F-22 is requiring 

significantly more maintenance actions than planned. To date, the 

program has identified about 260 types of failures, such as main 

landing gears wearing out more quickly than planned, fasteners being 

damaged, and canopy delaminating, all leading to unanticipated 

operating and support costs.



Product Development in DOD Goes Forward in Spite of Poor Prospects for 

Reliability:



Our previous work identifying best practices during product development 

concluded that during this phase, the tasks are to ensure the stability 

of the design and to ensure that the product can be produced.[Footnote 

7] In DOD, the product developer frequently is trying to catch up to 

design and production tasks because product developments begin with 

immature technology. Schedule concerns override the need to capture 

knowledge about the design and production processes, and programs often 

proceed through development and into production before the reliability 

of the subsystems and systems has been demonstrated. Design features 

such as open architectures that allow systems to receive upgrades as 

technology advances or reductions to the number of parts in a design 

that reduce the need for spare parts and maintenance time are not given 

due consideration, even though they could lower operating and support 

costs of the system.



As the schedule tightens, the lack of knowledge becomes more 

acceptable, even preferred. Reliability testing is often pushed closer 

to fielding, resulting in supportability problems being identified 

during operational testing. The Office of the Director, Operational 

Test and Evaluation has commented that operational testers identified 

reliability as a problem in almost every program, because the product 

developer places more emphasis on performance requirements than 

suitability. According to the operational testers, many systems enter 

operational test and evaluation with known, but unresolved reliability 

problems.



Once a system is fielded, unless the reliability and ease of 

maintenance have been incorporated into the design already, there are 

limited opportunities to improve these metrics without costly redesign 

or retrofit. As the productï¿½s design becomes firm and the system is 

produced and fielded, the opportunities to influence these costs 

diminish. For example, the Army is currently attempting to reduce 

operating and support costs on its Apache helicopter and its Abrams 

tank. These efforts should have a favorable impact on the systemsï¿½ 

operating and support costs. However, the Army is retrofitting and 

replacing components that were identified as problems much earlier in 

the programs. Examples from the Apache and Abrams development 

illustrate the problems.



Figure 6: Apache Helicopter:



[See PDF for image]



[End of figure]



The Apache program focused on acquisition costs, schedule, and 

performance during product development, even when problems were 

identified that would impact reliability and maintainability. Today, 

the Apache helicopter is the Armyï¿½s most costly system in terms of 

operating and support costs with those costs rising over the years. For 

example, the target acquisition and designation system enables the 

Apache to find targets and guide its weapons. It is the helicopterï¿½s 

most sophisticated system, involving 26 major electrical, optical and 

mechanical components. However, the sight requires frequent 

maintenance, and its complexity reduces its reliability. The pilotï¿½s 

night vision system is also a highly complex system that experienced 

problems in development. Because its target acquisition and designation 

system and the pilotï¿½s night vision system proved unreliable, the 

Apache incurred higher than expected costs to maintain the helicopterï¿½s 

availability.



During development, problems identified with the Apache foreshadowed 

future support problems. These problems included excessive vibration 

and excessive aircraft weight in addition to difficulty in achieving 

reliability rates for the aircraftï¿½s target acquisition and designation 

system. During development, problems with the Apacheï¿½s fault detection 

system led to questions concerning whether it could operate safely and 

reliably during operations. Further, Army test and evaluation agencies 

warned that these problems could cause serious supportability issues 

since they would result in frequent need for maintenance and repair. By 

1990, the majority of the Apache helicopters had been produced and 

fielded, but Apache could achieve only 50 percent availability rates, 

well short of their 70 percent goal. Tests showed that the Apache 

required maintenance actions to correct significant problems every 2.5 

flying hours while the Armyï¿½s goal was 4 hours between failures. 

Maintenance units were physically unable to handle the repairs required 

to keep the helicopter flying. Subsequently, the Army added 18 

maintainers to the Apache battalion, constructed more contractor repair 

facilities, and hired more contractors. During Operation Desert Storm, 

although the system was effective in destroying tanks and other 

targets, it continued to experience reliability and logistical support 

problems that grounded some aircraft.



Figure 7: Abrams Tank:



[See PDF for image]



[End of figure]



Likewise, the Army produced the Abrams tank without first resolving 

reliability problems. The tank provided a major improvement in speed, 

agility and lethality over the older M-60 tank and was also supposed to 

decrease the operating and support cost burden. However, during 

development, the tank experienced serious failures with the track, the 

engine, fuel filters, the fuel and water separator, and the fuel pump. 

In particular, the durability and reliability of the Armyï¿½s turbine 

engine was a major concern. Tests performed up to the time of the 

initial production decision showed that the tank generally met its 

performance requirements. However, frequent breakdowns and component 

failures raised serious questions about approving the tankï¿½s 

production. In a 1993 operational test report from the Office of the 

Director, Operational Test and Evaluation, the reviewer concluded that 

the tank was operationally effective, but not operationally suitable 

because of its many support problems such as those mentioned above. 

Today, the Abrams is the second most costly Army system in terms of 

operating and support costs and accounts for half the repair parts 

costs by the entire ground combat fleet. The tank engine is a major 

contributor to the tankï¿½s high support costs. In recent years, the 

operating and support costs for the Abrams have remained steady at 

about $2 billion per year, although the Army has reduced the number of 

tanks to be supported by more than 300.



Data on Operating and Support Costs Is Not Reliable:



When estimating operating and support costs for a new weapon system or 

trying to establish maintenance trends, maintenance data on the current 

system are an important source of information. However, the three 

servicesï¿½ operating and support data collection systems do not provide 

accurate and reliable information about the cost to operate and 

maintain the systems because they do not collect and maintain data on 

all elements of the weapon systemï¿½s operating and support cost. Without 

reliable information, the services cannot assess trends or identify top 

cost drivers and take corrective action.



The Armyï¿½s Operating and Support Management Information System provides 

historical cost data on Army weapon systems and is the primary source 

of operating and support data used by the program managers to project 

costs of new systems, forecast spare parts budgets, and generally 

manage their programs. The Army uses this data system to develop its 

operating and support cost budget for weapon systems for consumable 

items such as repair parts, petroleum, oil, lubricant, fuel, and 

ammunition as well as for some maintenance. However, the system does 

not provide a complete and reliable basis for developing and reporting 

the costs of weapon system support. The data are often incomplete and 

12 to 18 months out-of-date. Several important cost elements used to 

establish the Armyï¿½s operating and support budget estimates--such as 

contractor logistics support, supply maintenance and software support-

-are not included in the database.



The Navyï¿½s central tracking system for operating and support costs, 

like the Army data collection system just described, provides 

historical cost data on weapon systems. Navy program officials told us 

that this data is often inaccurate and cannot be relied upon to 

pinpoint causes of failure or maintenance actions.



Until recently, tracking Air Force operating and support costs was 

difficult because the cost data were unavailable in a usable format or 

were of poor quality. Poor cost data weakened operating and support 

cost estimates. In 1998, the Air Force set up a total ownership cost 

database, with the objective of providing accurate and reliable data to 

identify cost drivers and support decision makers in making 

improvements to fielded aircraft. However, the new system is not 

available at all aircraft maintenance locations, and therefore data may 

not be complete. In any case, the Air Force will need several years of 

cost data before it can evaluate the effectiveness of the system.



[End of section]



Chapter 3: Commercial Companies Deliberately Manage Ownership Costs 

through Product Requirements and Design:



Leading commercial companies consider the total ownership cost of a 

new product, including its operating and support costs, integral to 

their new product development decisions. Companies that use airplanes, 

trucks, and ships to deliver goods and people such as United Airlines, 

FedEx Express, and Polar Tanker understand the importance of 

maintaining the readiness of their fleets at as low an operating cost 

as possible. Reducing these costs results in increased revenues, 

profits, and market growth. Increases in these costs, on the other 

hand, can result in failure. The companies also understand that unless 

reliability is designed into a product, there are limited opportunities 

to improve readiness and reduce cost without costly retrofit or 

redesign. They have been successful in reducing these costs because 

they developed a collaborative process with companies that develop 

those products, such as Boeing, for setting the productï¿½s requirements, 

developing the product with operating and support costs in mind, and 

capturing accurate operations and support data once it is delivered.



The companies we visited that bought products all established the 

productï¿½s operating cost and its availability as requirements equal to 

its performance characteristics prior to product development, thereby 

ensuring that the developer placed priority on those goals during 

product development. They were amenable to reducing the productï¿½s 

performance features to reduce its operating cost, as long as 

performance was within acceptable limits for achieving market 

objectives. They also considered bearing additional cost for the 

productï¿½s design if it resulted in a net benefit from reduced operating 

costs. Once product development began, maintainers had a strong voice 

in the productï¿½s design and leading product developers set high 

reliability standards for components they chose, using proven 

technologies to achieve performance requirements. Companies, such as 

Boeing and Maytag, chose an evolutionary approach to product 

development to achieve goals for life-cycle costs, testing extensively 

and early for reliability. These companies also emphasized other 

practices to reduce operating costs such as reducing the number of 

parts in a productï¿½s design, using standardized parts, and using open 

systems to ease maintenance and maintain competition. Once the product 

is delivered, maintainers keep detailed records on its reliability and 

maintenance and provide that information to the developer to improve 

future products.



A Best Practices Model:



Figure 8 represents a model of the best practices that were most 

helpful to the companies we visited in achieving high reliability and 

reducing a productï¿½s operating and support costs. Notably, the most 

critical events--those that have the most impact on a productï¿½s 

operating and support costs--take place very early, either prior to 

product development when the productï¿½s key requirements are 

established, or very early in product development before the design is 

finalized. Another notable aspect of the model is that each of the 

activities--requirements-setting, product development, and operations-

-depend on clear and constant communication and collaboration among the 

customer, the product developer, and the maintainer from the time a 

product is conceived until the operator disposes of it.



Figure 8: Commercial Model for Reducing Operating and Support Costs:



[See PDF for image]



[End of figure]



The goal of this process is to develop and field a product that will 

perform in accordance with the customerï¿½s needs and will be ready when 

needed within cost targets. As illustrated above, decisions about the 

productï¿½s performance and cost are finalized prior to beginning the 

productï¿½s development, and the costs to operate and support the product 

are key considerations. Once the requirements are set, the product 

developer can focus as much on achieving a reliable and robust design 

for the product as on achieving its performance goals. Finally, once 

the product is fielded, those responsible for its operation and 

maintenance continue to feed information back to the developer to 

improve future designs. Figure 9 illustrates what happens when the 

commercial firms we visited set requirements that force the product 

developer to consider reliability rates during design. When both 

operating cost and readiness are key requirements for the product 

developer, the developer focuses on using as many components and 

subsystems as possible that have demonstrated reliability rates. The 

customer, by demanding readiness at a certain cost as a hard 

requirement upfront has raised the importance of achieving it from the 

very start.



Figure 9: Benefits of Ensuring High Reliability Rates During Product 

Development:



[See PDF for image]



[End of figure]



Leading Companies Treat Readiness and Operating and Support Cost as 

Critical Product Requirements:



We visited three companies--Polar Tanker, an operator of large oil 

tankers; United Airlines; and FedEx Express--to determine their 

practices for ensuring low operating and support costs from the 

equipment they purchased from product developers. All three companies 

believe that understanding and controlling the cost to operate and 

support a product while it is being designed is essential to driving 

down the total ownership cost of a product. To do this, they set 

requirements for a new productï¿½s availability and its operating and 

support costs equal in importance to requirements for its performance 

and acquisition cost. We found these companies set requirements before 

development begins that their products be ready almost 100 percent of 

the time at the lowest operating cost possible. They typically set 

maintenance goals that drive operating and support cost decisions, such 

as maintenance cost per mile over a productï¿½s lifetime. The following 

summaries illustrate the commercial processes.



Figure 10: Polar Tankerï¿½s Polar Endeavor:



[See PDF for image]



[End of figure]



Polar Tanker:



Polar Tanker is a commercial oil-transporting firm that recently 

decided that a new oil tanker was necessary to haul oil between Prince 

William Sound and Puget Sound. The companyï¿½s critical requirements for 

the new Endeavor tanker, which Polar Tanker believed would reduce the 

cost of delivering oil, were:



* less expensive operations and maintenance over a 30-year life cycle 

(versus the industry standard of a 20-year life cycle); and:



* availability for operations at least 330 days a year.



Polar Tanker teamed its maintenance engineers with industry consultants 

to ensure these requirements were met. The procurement team relied on 

its archived maintenance data from previous Alaskan operations to 

develop its double-hulled tankers. It documented locations, lengths and 

types of fractures, and stresses in the structures of its existing 

inventory of ships. This documentation from past operations was used to 

determine structural changes required to reduce maintenance on the new 

Endeavor class. As a result of their record keeping and the constant 

communication with the product developer, Northrop Grummanï¿½s Avondale 

Shipyard, Polar Tankerï¿½s owners were successful in redesigning their 

ships and meeting their requirements for improved performance, 

reliability, and lower operations and maintenance costs.



The dual requirement of reduced operating and support cost coupled with 

high readiness rates drove design trades that increased development 

costs but improved reliability. For example, once Polar Tankerï¿½s 

procurement team identified ballast tank maintenance as one of the most 

significant maintenance burdens, it directed Northrop Grumman to use 

the best and most expensive epoxy coatings and specialized paints to 

protect the tanks from corrosion. Another design trade--adding 

additional structure to the shipï¿½s hull to reduce the impacts of 

fatigue cracking--increased the acquisition cost of the tanker but 

improved reliability and reduced the need for maintenance. Polar Tanker 

accumulated data from its current fleet and conducted extensive 

modeling of the hull design to understand where the most critical 

cracking occurred and to identify operating and support cost drivers. 

It hired an engineering consulting company to conduct further analysis. 

As a result, Polar Tanker and Northrop Grumman utilized the most 

current design tools to optimize the shipï¿½s structure. Polar Tanker 

also developed a list of equipment and suppliers based on reliability 

analysis and incorporated that list into the design contract with 

Northrop Grumman. Polar Tanker estimated that these design trades cost 

about $25 million in additional design costs, but they believe the 

changes will ensure its tankers will be able to operate more reliably 

over 30 years.



Polar Tankerï¿½s procurement team also required that the new Endeavor 

Class design use open systems when possible. It worked closely with 

Northrop Grumman, its contractor, to ensure requirements for an 

integrated bridge system that consisted of commercial-off-the shelf 

components with open systems to provide the capability to modernize the 

system much less expensively as technology improved. For example, 

systems in the tankerï¿½s wheelhouse including the autopilot, marine 

radars, bridge control console, and satellite communication equipment 

used the open systems concept. The design also included easily 

accessible decks to minimize delays and disruption during maintenance. 

Polar Tanker has already upgraded its electronics since putting its 

first ship, the Polar Endeavor, into service and experienced minimal 

disruption in operations.



Figure 11: United Airlines/Boeing 777:



[See PDF for image]



[End of figure]



United Airlines:



As the launch customer for the new Boeing 777, United Airlines 

established stringent requirements for aircraft readiness and operating 

costs, thereby ensuring that reliability would be an important design 

element. When United and Boeing negotiated the requirements, United 

stated that it wanted a twin-engine airplane that could fly extended 

ranges from any airport in the United States. In addition, United 

required that the 777 be available at the gate for departure within 15 

minutes of scheduled departure 98.5 percent of the time. Boeing 

guaranteed United that the 777 would meet the departure requirement by 

the third year of operation or Boeing would pay a financial penalty. 

According to United officials, the 98.5 percent rate was achieved by 

the third year. United also specified that operating and support costs 

for the 777 be no higher than on past airplanes. The agreement reached 

with Boeing was that Boeing would reimburse United for revenues lost as 

a result of airplanes being unavailable. By setting requirements for 

operating cost and readiness, United ensured that Boeing would build 

reliability into the design of the 777.



Boeing brought together a working group of customers--the leading 

commercial airlines--to discuss requirements for the new design. Boeing 

officials told us that during those requirements meetings the 

participating airlines defined major design initiatives for the Boeing 

777 based on estimates of life-cycle costs. Initially there were 

differences among the airlines as to what exactly was needed on the new 

aircraft--from wider fuselages to additional electronics--but from the 

very first meeting, the airlines were all equally concerned with 

operating and maintenance costs. They were focused on designing an 

aircraft that would be easy for mechanics to repair. The airlines 

unanimously agreed that an airline maintenance representative was 

needed to adequately address operations and maintenance requirements. 

Boeing named a chief mechanic who had previously worked for United to 

the working group that was influential in defining the maintenance 

requirements. Although Boeing provided its customers engineering 

estimates for the operating cost of its new aircraft in comparison to 

the older model, United officials said they developed their own 

estimates based on its historical experience with Boeing aircraft. 

United officials said that having two perspectives from which to 

consider its purchase was helpful.



Figure 12: FedEx Express Delivery Van:



[See PDF for image]



[End of figure]



FedEx Express:



The FedEx Express mission is to provide global air and ground 

transportation of high-priority goods and documents that require rapid, 

time-certain delivery. This mission demands high availability and 

reliability from its delivery equipment. In purchasing its newly 

designed fleet of delivery trucks, FedEx Express considered 

reliability, maintainability, and low operating and support costs to be 

critical measures of a successful acquisition. For example, improving 

availability and reliability were the key drivers in its acquisition of 

the new 700 cubic foot truck. FedEx Express collaborated with a product 

developer, Freightliner, to set the requirements for a newly designed 

vehicle with high reliability and endurance to withstand frequent 

stops, short travel distances between stops, and demanding use of the 

brakes. During the requirements-setting process, they also established 

cost and reliability requirements for the new truck that estimated an 

assumed number of stops per day, a certain number of miles per year at 

an assumed cost per mile.



To make sure all costs, particularly operating and support costs were 

considered during product development, FedEx Express had a logistics 

manager lead their discussions with Freightliner for the development of 

the new truck. The FedEx Express logistics managers told us that they 

are required to reduce the companyï¿½s operating and support costs, and 

the company believes that it is essential to give these managers an 

integral role during these discussions. The new delivery truck has been 

successful in meeting its reliability and maintenance goals. The truck 

currently averages 70,000 miles between breakdowns and is operating 

within the cost per mile of service that was set as a requirement by 

FedEx Express at the beginning of development.



Knowledge-Based Product Development Is Critical to Achieving Desired 

Reliability and Managing Operating and Support Costs:



To meet the readiness and operating and support requirements that their 

customers demanded, we found that product developers focused on 

designing a product that was durable, easy to maintain, ready when 

needed, and reliable at low cost. They used an evolutionary development 

process to meet these requirements and did not allow components or 

subsystems into a productï¿½s design unless their reliability had been 

proven through past use or testing. Boeing told us that it makes the 

reliability ratings of its components available to the airlines before 

it begins product development, and Maytag did extensive reliability 

testing on every new product prior to going to production. The 

companies also emphasized product designs with fewer parts and used 

open systems architectures as much as possible. Developers also gained 

valuable insight into design features their customers valued that drove 

down operating and support costs. For example, the design for Boeingï¿½s 

latest generation of the 737--geared toward reducing operating costs--

was inspired by the airlinesï¿½ request for more affordable operations 

and maintenance.



Once defined, the functional and operating and support requirements are 

tightly managed and controlled to minimize scope increase during 

product development. Companies work within a common framework to 

provide management oversight and control. To move a product 

successfully from concept to operations, the companies we visited used 

a ï¿½gatedï¿½ product development process and firm criteria to dictate when 

a product is ready to exit a stage. The design reviews address all 

operating and support cost requirements of the products. Senior 

management review teams grant product approval at each gate only after 

determining that business cases adequately address major drivers of 

operating and support cost as well as reliability goals.



Boeing:



Boeing uses a structured, gated product development process to define, 

evaluate, and approve projects and to integrate new technology into its 

aircraft. This process separates technology development from product 

development programs. In fact, Boeing keeps immature or untested 

technologies in a research and development phase until they have been 

tested for reliability in a realistic environment. The process forces 

the company to obtain purchase agreements from customers and build a 

business case that shows the expected profitability of a product line 

before detailed designs are developed and a large dollar investment is 

made in manufacturing. In order to get these purchase agreements, 

Boeing product teams work collaboratively with potential customers to 

set requirements for readiness, reliability, performance, acquisition 

costs, and operating and support costs. Because aircraft are large 

capital investments, customers are working with other developers to 

make sure that the competitive forces of the market will help them get 

the best aircraft at the lowest cost.



Boeing officials told us that to remain competitive in this 

environment, they focus on meeting customer requirements at the highest 

reliability leading to the lowest operating and support cost possible. 

For example, when it was designing the new 737, Boeing used maintenance 

records to prove to the airline that they could redesign the 737 with 

high reliability and reduced operating and support costs, a key market 

requirement for that aircraft. According to Boeing officials, they 

prepared a comparative analysis of the acquisition and operating and 

support costs for older 737 aircraft with the estimated costs for the 

737 Next Generation. This analysis showed that operating and support 

costs were significantly lower for the proposed next generation 737 and 

resulted in big savings to the customer. Boeing was able to develop 

good operating and support cost estimates for potential customers 

because it used an evolutionary approach for developing new aircraft. 

This approach allowed Boeing to improve performance and to insert new, 

reliable, and mature technology.



Once a new product line is approved, Boeing continues to work with its 

customers to identify improvements that can be made to an aircraft in 

terms of parts reduction, parts standardization, and ease of 

maintenance. For example, Boeing formed four airline working groups 

with representatives from 21 airlines around the world to focus on 

maintainability, interiors, power plant, auxiliary power units, and 

common display system issues on the 737. Their objective was to collect 

feedback from operators on design changes that would reduce maintenance 

costs. Examples of airline-driven design changes include:



* simplifying the wing flaps by eliminating one third of the parts, 

designing a simpler flap mechanism, and making parts removable with 

common grip length fasteners;



* reducing engine removal and installation time by increasing the on-

wing life of the engine by 40 percent and reducing the predicted change 

time from 12 hours to 6 hours; and:



* improving the reliability and standardizing parts of the fuel system 

by using a fuel shutoff valve that is common with other aircraft.



To facilitate and improve communication with its customers, Boeing 

oftentimes collocates customer representatives at its production 

facilities. Boeing has found that involving the customer in early 

design decisions improves their ability to design a reliable aircraft 

that is easy and less costly to maintain. For example, when it began 

developing the 777 aircraft, airline maintenance workers offered over 

5,000 suggestions for changes to the design based upon their 

experiences with other Boeing aircraft. These suggestions helped reduce 

parts and improve reliability of the 777, resulting in increased time 

between maintenance actions. Boeing also utilized the concept of open 

systems to reduce the total ownership cost of the 777 by allowing 

customers to choose from three different types of engines--GE, Pratt & 

Whitney, or Rolls Royce--depending on their needs.



Maytag:



Maytag follows a similar product development process. For Maytag, the 

most critical phase is when design specifications for product 

reliability and manufacturing feasibility are fully defined and 

understood. Early in its product development, an integrated product 

team takes all new product features through rigorous reliability growth 

analysis to determine what can be expected from the design and whether 

it will be able to meet the requirements. Maytag uses failure analyses, 

mock-ups, and other simulation tools to focus testing on the most 

critical elements of the new design and reduce the number of design/

build/test iterations of the product. Even though these analyses 

require more upfront planning, Maytag officials stated that they still 

cut testing times in half and yet improved reliability and lower costs.



During this phase, Maytag employs two project leaders, both a 

marketing manager and technical systems engineer, to conduct cost 

performance trades and co-chair subsystem design reviews. As a result 

of this co-leadership, most design issues are resolved immediately, and 

80 percent of all reliability testing and cost reductions take place 

early in the process. Also, during this phase the project leaders 

present their strongest business case, fully disclosing feasibility 

data, product definitions, and estimates of life-cycle costs for team 

approval. The team uses this information to make the critical product 

development decision to commit further to product development or kill a 

product idea. Maytag officials stated that their company has a quality 

image to uphold and that consumers expect the highest reliability and 

quality from their products.



Maytag officials were also conservative in their use of new 

technologies during product development. Decisions to incorporate new 

technology were made in the earliest phases of product development and 

were based on assessments of the adaptability and maturity of the 

technology and associated risks to achieving established reliability 

targets. Even though its Neptune washer incorporated over 90 percent 

new technologies in its development, Maytag officials stated that they 

spent time with suppliers and developers, maturing technologies to an 

acceptable level of reliability before launching the new development.



Leading Commercial Firms Use Feedback from Operations to Better 

Understand Customer Needs, Product Deficiencies, and Operating and 

Support Costs:



The collection and analysis of the operating and support costs for 

delivered products was considered essential by leading commercial 

companies. Once a product is fielded, leading companies track actual 

operating cost, reliability of parts, and readiness of the product 

against what was estimated during product development to make sure the 

company is getting what it paid for. We found companies are always 

identifying the top drivers of operating and support cost and working 

with the manufacturers to reduce these costs. When there are part 

failures, the companies can quickly identify whether or not they are 

under warranty and get the part replaced.



All of the companies we visited, customers and product developers 

alike, had very collaborative processes and practices for drawing 

extensively on data from current and past operations to improve the 

reliability of existing products or influence the design of new 

products. United Airlines officials told us that the airlines and 

Boeing both keep meticulous reliability and cost records at all levels 

of the 777: components, subsystems, and the system level. Major 

operating costs drivers are tracked on a daily basis by the airlines. 

The manufacturers usually have personnel residing with the airlinesï¿½ 

maintenance crews to help solve problems on the spot and, just as 

importantly, feed information back to the manufacturer so that the next 

product can be improved. FedEx Express and Polar Tanker both emphasized 

extensive collection of data from current operations. In fact, FedEx 

Express sets annual targets for operating and support cost reductions 

based on reliability data gathered on the road. Polar Tanker gathered 

maintenance data from past operations to determine areas that could 

result in higher reliability and lower maintenance costs in designing 

the new tanker.



United conducts quarterly meetings for each of its fleets to discuss 

open issues and short-term and long-term solutions to current problems 

with operational aircraft. Attendees at the meeting include maintenance 

mangers, financial representatives, representatives of the 

manufacturers, and executives with authority to resolve issues. 

Unitedï¿½s practice is to resolve problems as expeditiously as possible, 

no matter how small. United also monitors flight movements on a real 

time basis through a computer system that tracks each aircraft by tail 

number. The monitoring system provides the maintenance history of the 

aircraft, reports problems on a current flight that require maintenance 

upon landing, and alerts for other required maintenance based upon the 

number of flying hours on the aircraft. If a specific part is broken, 

the system also indicates whether or not it is under warranty. United 

archives information on parts that break, when they break, and whether 

they are still under warranties.



Manufacturers like Boeing and other major suppliers like this type of 

feedback because it provides useful information to them on how to 

improve the product for future iterations. They also believe that quick 

responses to customer problems will help them get repeat business. They 

use this feedback to develop preventive maintenance schedules, better 

estimate operating and support costs, and refine reliability 

requirements to be used in preparing budgets and cost estimates for 

future products. Feedback mechanisms also accumulated operations and 

maintenance data and ï¿½lessons learnedï¿½ that highlighted reliability 

problems and other maintenance issues. United officials stated that 

taking lessons learned from data gathered from current products is an 

effective tool for improving product reliability and maintainability or 

developing requirements for new products.



FedEx Express takes any failure to meet its on-time delivery goal very 

seriously and holds daily failure analysis meetings every morning to 

analyze and review each delivery failure from the night before. This 

constant feedback allows them to take immediate corrective action on 

individual vehicles and to identify trends that may result in larger 

problems and costlier maintenance requirements. FedEx Express 

maintained a metric for vehicle miles between road calls and 

maintenance costs per asset. In addition to tracking these costs per 

asset, FedEx Express has established a performance goal with its 

managers to reduce the overall costs of maintenance each year. FedEx 

Express managers set their annual targets for reductions in operating 

and support cost based on reliability data gathered on the road.



[End of section]



Chapter 4: Stressing Operating and Support Cost at the Outset of an 

Acquisition Could Help DOD Reduce Total Ownership Costs:



DOD and the commercial companies we visited have policy goals of 

developing products that will meet customersï¿½ needs at the lowest 

possible cost to build and operate. The difference between them is in 

how each implements its policies. Leading commercial companies follow 

an integrated, collaborative process of setting requirements, 

developing the product, and ensuring that the product can be supported 

at an acceptable cost. DODï¿½s process is composed of disparate practices 

carried out by separate organizations with differing objectives and 

little communication between them about how to support fielded systems. 

While commercial firms focus on total ownership costs at the outset, 

DOD focuses mostly on technical performance. One cause of this is that 

in DOD the accountability and responsibility for total ownership costs 

are spread across many organizations with separate goals. Another cause 

lies in motivation for low costs. The commercial companies we visited 

are driven by the need to be as profitable as possible to survive, and 

low total ownership costs translate to higher profitability. DODï¿½s 

environment does not provide such incentives. The organizations charged 

with acquiring and operating weapon systems are unconstrained by a need 

to lower costs since they can request additional operations and 

maintenance funding to keep systems working.



From time to time, DOD stated the need to lower its total ownership 

costs in policy documents and in annual budget statements; however, it 

has not been successful because it does not have an environment that 

demands collaboration and accountability in setting requirements and 

developing products with operating costs in mind. DOD has some efforts 

underway to improve. First, it has rewritten its acquisition and 

requirements generation policies. Second, a few programs now in 

development established an early estimate of operating and support 

costs and are working to gain knowledge of the impacts of requirements 

and design on those costs. Third, information from pilots on fielded 

systems, if disseminated throughout the acquisition community, could be 

used to lower costs. Results are pending.



What DODï¿½s efforts do not do is provide incentives to make investments 

for more reliable, less-costly-to-maintain systems at the beginning of 

an acquisition. Instead, DOD provides incentives to field systems with 

unknown reliability by allowing whatever funding necessary to operate 

and maintain the systems once they are fielded.



Differences in Practices Explain Different Outcomes for Commercial 

Companies and DOD in Controlling Total Ownership Costs:



In Chapter 3, we discussed our findings that leading commercial 

companies set specific requirements for readiness and operating and 

support costs prior to initiating product development that forced 

developers to design products with a high degree of reliability. In 

Chapter 2, we noted that DOD, on the other hand, typically focused its 

requirements on revolutionary performance that often forced developers 

to mature technologies at the same time they were completing detailed 

design work. As a consequence, system reliability often suffered, 

forcing the department to spend a great deal of money to maintain and 

repair fielded systems in an effort to achieve desired readiness 

levels. The following table provides a comparison of the specific 

practices used by commercial companies we visited and DOD programs we 

reviewed to address operating and support costs early in a new 

productï¿½s life cycle.



Table 2: DOD and Commercial Practices for Controlling Operating and 

Support Costs:



Commercial prevailing practice: Practices used to set initial product 

requirements.



Commercial prevailing practice: Operating and support cost goals as a 

key requirement.; DOD prevailing practice: Practices used to set 

initial product requirements: Operating and support cost goals are not 

established as key parameters..



Commercial prevailing practice: Readiness a key requirement.; DOD 

prevailing practice: Practices used to set initial product 

requirements: Readiness is not a key parameter..



Commercial prevailing practice: Trade performance for reduced operating 

and support costs, if appropriate; sometimes results in increased 

costs.; DOD prevailing practice: Practices used to set initial product 

requirements: Technical performance is sometimes traded using cost as 

an independent variable, but cost is usually production cost or 

development cost, and the trades occur during the design phase..



Commercial prevailing practice: Direct relationship during 

requirements-setting between the user and the product developer.; DOD 

prevailing practice: Practices used to set initial product 

requirements: User and product developer separated by user 

representative and government program office..



Commercial prevailing practice: Practices used during product 

development.



Commercial prevailing practice: Provide detailed operating and support 

cost estimates early in product development.; DOD prevailing practice: 

Practices used to set initial product requirements: Operating and 

support cost estimates not required until product development launch..



Commercial prevailing practice: User and developer focus on ways to 

reduce product parts and standardize parts across product lines.; DOD 

prevailing practice: Practices used to set initial product 

requirements: Product developer has responsibility of focusing on ways 

to reduce parts counts or use standardized parts with little input from 

the user (operators or maintainers)..



Commercial prevailing practice: Use open systems architecture approach 

to improve the cost effectiveness and installation efficiency of future 

upgrades to the product.; DOD prevailing practice: Practices used to 

set initial product requirements: Open systems approach is mandated but 

implementation is limited..



Commercial prevailing practice: Set realistic reliability growth goals 

for the product.; DOD prevailing practice: Practices used to set 

initial product requirements: Reliability goals set, but they are 

tradable or not met..



Commercial prevailing practice: Conduct reliability testing early.; DOD 

prevailing practice: Practices used to set initial product 

requirements: Reliability testing sporadically performed..



Commercial prevailing practice: Practices used during operations.



Commercial prevailing practice: Collect and analyze operations and 

support data.; DOD prevailing practice: Practices used to set initial 

product requirements: Data is often incomplete or unreliable..



Commercial prevailing practice: Manage operations and support costs to 

targets.; DOD prevailing practice: Practices used to set initial 

product requirements: Do not manage to operations and support targets..



Commercial prevailing practice: Identify areas for continuous 

improvement.; DOD prevailing practice: Practices used to set initial 

product requirements: Lack of complete and reliable data makes 

identifying areas for improvement difficult; some areas that are 

identified are not funded for improvement..



Commercial prevailing practice: Feedback to developer on product 

performance.; DOD prevailing practice: Practices used to set initial 

product requirements: Limited feedback to the developer. The maintainer 

does not have a direct relationship with the product developer..



[End of table]



Source: GAO.



Clearly, the practices used by the commercial companies we visited 

before product development when requirements are set, early in product 

development when the design is finalized, and during the new productï¿½s 

operating life focus as much on providing a reliable product as on 

providing a high-performance product. The companies make operating cost 

and readiness key requirements, they perform extensive reliability 

testing, and they aim toward continuous improvement once the product is 

in the field. In DOD, because performance is the overriding concern of 

the requirement setters, none of these practices are in place.



Several DOD Efforts Underway to Reduce Total Ownership Costs:



The changes to policies and the investment in improving systemsï¿½ 

reliability are encouraging indicators that DOD has focused its 

attention on reducing costs to support weapons. DOD has revised 

acquisition policies, tested new approaches for reducing costs in a few 

systems, explored differing approaches, and created initiatives to 

reduce costs of legacy systems. Each of these efforts had some initial 

success, but most are aimed at reducing costs after fielding when over 

90 percent of the costs have been determined.



DODï¿½s Requirements Generation and Acquisition Policies Could Be More 

Specific in Addressing Supportability Issues:



DOD has revised its 5000 series acquisition policies several times over 

the past 10 years with the intent of defining an acquisition 

environment that makes DOD a smart and responsive buyer. During this 

time, the policy has not substantively changed with regard to how 

acquisition programs can best control total ownership costs. The 

department is striving for an integrated acquisition and logistics 

process that is characterized by, among other things, a stronger focus 

on using supportability as a key design and performance factor. 

However, rules for total ownership cost goals at the outset of an 

acquisition program are defined by the Chairman of the Joint Chiefs of 

Staffï¿½s Instruction 3170.01B[Footnote 8] on requirements generation. 

This guidance states that cost should be addressed in the operational 

requirements document for a new weapon system, if an estimate is 

available at that time. However, policy does not require the services 

to set requirements for operating and support costs or readiness. 

Instead, it allows them to identify system capabilities or 

characteristics they consider essential for successfully completing the 

mission. It states that the DOD sponsor may make cost a key requirement 

if it desires and identify the cost it wishes to evaluate.



We previously reported that DOD officials believe they must promise 

new, revolutionary weapon systems with significantly better performance 

capabilities than the ones they are replacing in order to obtain 

funding.[Footnote 9] Therefore, key parameters are usually focused on 

performance rather than supportability.



In order to effectively minimize total ownership costs of its systems, 

the Department of the Navy recently issued its own guidance that 

establishes specific supportability and affordability thresholds and 

objectives[Footnote 10] for all requirements documents. The Navy 

believes that by establishing readiness and operating and support cost 

as required parameters, there is assurance that major drivers of total 

ownership costs will be addressed and minimized throughout the 

acquisition process. Specifically, the new guidance states that 

requirements documents must include goals for operating and support 

costs. It also states that operational availability be included as a 

key performance parameter, except when logistics delays are not an 

issue or if the requirements are for a major aircraft or ship platform. 

In those cases, mission capable rates or full mission capable rates 

focused on the platformï¿½s primary mission areas will be used as key 

requirements. The Army is discussing a similar change in its guidance.



Three New Acquisition Programs Are Placing Greater Emphasis on 

Readiness and Operating Cost Goals:



We found three DOD programs still in development--the Joint Strike 

Fighter, the Advanced Amphibious Assault Vehicle, and the Landing 

Platform Dock 17--that appear to be using good practices to reduce 

operating and support costs during product development. Each, in its 

own way, has had a powerful internal incentive to establish more 

collaborative practices or to focus attention on operating and support 

costs and product reliability.



Figure 13: Joint Strike Fighter:



[See PDF for image]



[End of figure]



The Joint Strike Fighter program is intended to produce an affordable 

next-generation aircraft to replace DODï¿½s aging aircraft inventory. The 

program is structured to use a common production line to produce three 

aircraft variants that meet conventional flight requirements for the 

Air Force, short take-off and vertical landing characteristics for the 

Marine Corps, and carrier operation suitability needs for the Navy. The 

program will also provide aircraft to the British Royal Navy and Air 

Force. A key objective of the acquisition strategy is affordability--

reducing the development, production, and operating costs of the 

program relative to prior fighter aircraft it will replace. The 

programï¿½s latest stated estimate for operating and support cost savings 

compared to legacy systems is $135 billion, or a 56 percent reduction 

in cost.



To achieve this affordability objective, the program office has 

incorporated various DOD and commercial initiatives into the 

acquisition strategy. For example, two key provisions in its 

operational requirements document--mission reliability and logistics 

footprint--will have a direct impact on operating and support costs. 

Specifically, all variants of the fighter are expected to achieve a 

mission reliability rate of over 90 percent and meet numeric goals of 

cargo aircraft or ship space needed to support a 30-day self-sustained 

deployment. These two requirements, along with other reliability and 

maintainability goals, demonstrate DODï¿½s desire to reduce total 

ownership costs. The product developer currently estimates the Joint 

Strike Fighter will be able to reduce operating and support cost 

primarily through efforts to improve:



* reliability and durability of materials,



* accessibility of parts or systems that need to be inspected or 

replaced,



* supportability of low observable materials,



* ability of on-board systems to predict impending flight critical 

failures, and:



* training materials and systems.



For example, DOD expects to save about $39 billion over the life of the 

fighter through reduced maintenance on low observable materials. The 

developer estimates that 99 percent of the maintenance actions will 

require no low observable restoration because they are using high 

durability materials, parts, or systems that are easier to access and 

harder to damage. In order to reach this level of savings, the 

developer spent a great deal of time evaluating previous DOD 

maintenance experience with the B-2A bomber and the F-117 fighter 

aircraft and used an evolutionary approach for upgrading these 

materials. Operating and support costs for the B-2A bomber, for 

example, were significantly increased by the decision to use an 

immature technology for low observability.



However, the Joint Strike Fighter program must be careful not to 

overestimate the total ownership cost savings it can achieve over 

legacy systems it will be replacing because the program is also 

depending on new technology for on-board systems to predict failure--

prognostics and health management technology--that is not yet ready for 

product development. In October 2001, GAO reported that this technology 

was not at an acceptable readiness level for inclusion in product 

development, but DOD and the contractor decided to include it in order 

to meet total ownership cost objectives.[Footnote 11] Program officials 

stated that about $16 billion--12 percent of the estimated $135 billion 

in total ownership cost savings--is expected to come from that 

technology. Since then, the officials have allowed for the possibility 

that the technology may not be included on initial production lots for 

the Joint Strike Fighter if it is not ready.



Figure 14: Advanced Amphibious Assault Vehicle:



[See PDF for image]



[End of figure]



The Advanced Amphibious Assault Vehicle is a Marine Corps program to 

improve its amphibious landing vehicle. The new development promises 

faster sea and land speeds, better protection, and more lethality. The 

development program has focused on maturing technology and paying 

attention to operating and support costs early in development. The 

program has used some of the best practices of commercial companies 

during development. Some of those include collocating the program 

office at the contractorï¿½s facility and making extensive use of Marine 

Corps war fighters and maintainers to provide a ï¿½hands onï¿½ assessment 

of how effective the vehicle would be in operations as well as how 

supportable it would be during operations. The Marines developed an 

early estimate of total ownership costs and included a reliability 

metric as one of its key requirements. The Marines developed three 

vehicle prototypes to mature the design and have conducted extensive 

reliability testing. The vehicle will have parts that are common with 

other weapons such as a gun that will be common with the Landing 

Platform Dock 17. Advanced Amphibious Assault Vehicle officials 

estimated that they will save $29 million in operating and support 

costs.



The Navy program office for the Landing Platform Dock 17 has adopted a 

total ownership cost approach. The program office established a process 

for suggesting and evaluating design trades that could reduce operating 

and support costs. Some of the design changes that the Navy made 

include enclosing the mast to reduce exposure to weather and salt 

water, and investing in high performance covering for the deck and well 

deck to mitigate corrosion. Other practices include involving users to 

complete tasks using the virtual software to test special design 

elements in the ship, making greater use of sensors and automated 

processes to reduce maintenance and to reduce crew requirements.



Both the Landing Platform Dock -17ï¿½s and the Advanced Amphibious 

Assault Vehicleï¿½s estimate for operating and support costs have 

recently increased. The Navy raised its estimate of Landing Platform 

Dock 17ï¿½s requirements for spare parts, fuel, and software maintenance. 

The Advanced Amphibious Assault Vehicle program office attributed its 

increase in the cost estimate to funding additional prototypes to 

improve reliability. Information like this allows options for decision 

makers, while the system is still in development, to accept the costs 

or re-examine the performance characteristics to see if they can be 

relaxed in order to improve reliability and, thereby, reduce operating 

and support costs.



DOD Pilot Programs Attempt to Reduce Total Ownership Costs:



In 1999, the Defense Systems Affordability Council implemented a 

program to explore ways to reduce the total ownership cost of its 

weapon systems. The Council--chaired by the Undersecretary of Defense, 

Acquisition, Technology, and Logistics--set a goal of reducing 

logistics costs for selected fielded weapon systems by 20 percent by 

fiscal year 2005. The Council also set a goal for selected systems 

still in development to achieve total ownership costs targets that are 

20 percent to 50 percent below historical norms.



DOD selected 30 programs (10 from each military department) to test 

various approaches for reducing total ownership costs, such as using 

commercial items or technology to reduce costs of legacy systems and 

using industry standards when developing systems to make upgrades 

easier and less costly to complete.



The Air Forceï¿½s C-17 program is one of the pilot programs. It is a 

fielded system whose operating and support costs increased by about 25 

percent between 1995 and 1999. One C-17 initiative is an engine upgrade 

to extend the time between removals, reduce unexpected shop visits and 

spares purchases, and reduce the number of engine overhauls. The Air 

Force believes the C-17 could avoid $724.5 million in support costs if 

the upgrade is completed.



The Apache recapitalization program, another of DODï¿½s pilots, 

integrates a number of selected upgrades that taken together are 

expected to achieve a 30 percent reduction in operating and support 

costs by 2010. Most of the Apache helicopters will be refurbished and 

modified to the Apache Longbow configuration. The target acquisition 

and designation system, the top cost driver, is a focus of these 

improvements along with improvements in the drive train, the rotor, and 

the propulsion system. The current average cost per flying hour for the 

Apache fleet is $3,348. The Armyï¿½s projected cost per flight hour after 

the modifications is $2,230.



The Abrams tank is also undergoing a major upgrade estimated to cost 

about $5 billion. The top cost driver on the tank is the power pack, 

which includes the engine and transmission, followed by the auxiliary 

automotive system, hull and frame, fire control system, armament, and 

track. Army officials believe upgrading and replacing the engine is the 

most effective way to reduce operating and support costs for the tank. 

The current cost per mile for the Abrams fleet is $181 per mile, 

including repair parts and fuel but excluding most personnel cost. The 

recapitalization programï¿½s goal is to reduce the cost to $107 per mile 

and to improve reliability.



Other Initiatives Show Promise, but Implementation Is Slow:



We found three other initiatives--the change in acquisition policy 

toward evolutionary acquisition, an open systems approach for weapon 

systems and the Commercial Operations and Support Savings Initiative--

that could help DOD reduce total ownership cost. However, 

implementation has been limited in the latter two initiatives because 

consistent high-level support is lacking.



DOD defines evolutionary acquisition as an approach for delivering 

capability in increments, recognizing the need for future capability 

improvements. DOD allows two processes to achieve an evolutionary 

acquisition, both of which include requirements for collaboration 

between the user, the tester, and the developer. The first process is 

referred to as incremental development. In an incremental development 

process, a desired capability is identified, an end-state requirement 

is established, and the requirement is met over time by the development 

of several increments of the product, each dependent on available, 

mature technology. The second process is referred to as spiral 

development. In a spiral development process, the end-state requirement 

is not known, and each increment of the product is based on feedback 

from the user. Each increment yields the best possible capability for 

the user. The movement toward evolutionary acquisition and time-phased 

requirements bodes well for the potential to understand reliability, 

readiness, and implications for total ownership cost early, because an 

evolutionary process allows an acquisition program to design a weapon 

system to requirements based only on demonstrated technologies. This is 

very similar to commercial practices.



DOD chartered an open-systems joint task force to implement an open 

systems approach in weapon systems acquisitions. Open systems can 

reduce cost through use of widely accepted standard products from 

multiple suppliers, allowing DOD to benefit from the commercial market 

place and take advantage of the competitive pressures that motivate 

commercial companies to improve products and reduce prices. DOD 

expected open systems to reduce the cost of ownership of weapon 

systems, delay system obsolescence, and allow fielding superior war-

fighting capability more quickly. The DOD Inspector General recently 

reported that the DOD acquisition community has not fully applied the 

use of open systems objectives in the acquisition planning and review 

process.[Footnote 12] The report recommended DOD enforce the use of an 

open systems approach as part of the acquisition milestone review 

process.



Another initiative that showed promise but lacks high level support is 

the Commercial Operations and Support Savings Initiative introduced to 

improve weapon system readiness and reduce operating and support costs 

by inserting existing commercial items or technology into military 

legacy systems. It emphasizes the rapid development of prototypes and 

fielding of production items based on current commercial technology. 

According to a 2001 report by an independent assessment team, the 

initiativeï¿½s three objectives of reducing operations and support costs 

for legacy systems, simplifying prototype development, and attracting 

commercial firms to the defense marketplace are being met. But, the 

initiative lacks high-level support. The Under Secretary of Defense 

(Acquisition, Technology, and Logistics) recently directed that funding 

for the program be terminated.



DODï¿½s Current Environment Does Not Provide Incentives to Reduce Total 

Ownership Cost Early:



While initiatives for acquisition programs and potential reductions for 

the fielded systems are welcomed, the department has not 

institutionalized the practices used in the initiatives by demanding 

them on all acquisition programs. As we discussed in Chapter 1, 90 

percent of the operating and support costs are determined before 

fielding, and these initiatives do not attack the causes of higher 

operating and support costs. Those are: the division of responsibility 

among the requirements community, the acquisition community, and the 

maintenance community for controlling costs; the lack of focused 

attention on reliability early in development; and the lack of 

accountability for total ownership cost when setting requirements that 

is caused by the division of responsibilities across these communities.



Companies we visited have incentives to make operating cost and product 

readiness equal to technical performance when setting requirements for 

new products because these factors largely determine their 

profitability and, therefore, survival in the market place. Lower 

operating costs translate to higher profits and increased sales. 

Customers cannot afford to have large amounts of capital tied up in 

extra equipment, spare parts, or personnel to ensure their equipment is 

ready to perform when needed. They cannot afford to have equipment fail 

during operations, because failure precludes accomplishment of the 

companyï¿½s mission and loss of revenue. These companies are constrained 

by a finite amount of funding to acquire and operate their equipment, 

and, therefore, they hold the people setting the product requirements 

accountable for total ownership cost. Many of the companies we visited 

use one integrated product team to identify needs, set requirements, 

and monitor product development. Most importantly, the organization 

that will be responsible for supporting the equipment in the field sets 

requirements for new products. There is also a direct relationship 

between the requirements-setting team and the product developer while 

the productï¿½s requirements are being set, during development, and after 

products are put into service. Information flows throughout the 

integrated process, with each new phase in the process being informed 

by knowledge from the phase just ending.



DODï¿½s current acquisition environment does not provide the same 

incentives or practices. Traditionally, DOD does not constrain its 

requirement setters in the same way. Requirement setters in DOD have 

demanded weapon systems that, due to their performance features, 

consistently cost more to operate and support than anticipated to 

achieve necessary readiness levels. This has been accepted because a 

large logistics organization--separate from the requirement setting 

organization--is charged with supporting these weapon systems and uses 

monies from a different budget to do so. In essence, DODï¿½s environment 

frees the requirements community to insist on technical requirements 

that cannot be made into reliable products, are costly to support, and 

cannot be maintained cost effectively. Accountability for operating and 

support costs does not rest with the requirement setters, or, for that 

matter, with the acquisition community. Eventually, maintenance 

organizations have no choice but to request sufficient funding to keep 

weapon systems operating once they are fielded. DOD has identified this 

division of responsibility as a key cause of higher weapon system 

operating and support costs. In this current environment, there is no 

incentive for collaboration and accountability in setting requirements 

and developing products with operating costs in mind. Instead, it 

provides incentives to field systems with unknown reliability by 

allowing whatever funding necessary to operate and maintain the systems 

once they are fielded.



[End of section]



Chapter 5: Conclusions and Recommendations:



Acceptable readiness levels are a function of having platforms 

available when required. Such levels can be achieved by having highly 

reliable platforms, by spending whatever is necessary on ongoing 

maintenance, or by having excess capacity. The high cost of maintaining 

weapon systems to meet required readiness levels is depleting DODï¿½s 

modernization accounts and denying DOD the flexibility to invest in new 

weapons. DOD must find ways to reduce total ownership cost while 

maintaining needed readiness rates. Readiness is a critical component 

of all DOD weapons systems. If a system is not ready, its performance 

capabilities are of no use. The decision on whether readiness will be 

achieved by spending additional funds on operations or by designing 

high reliability into the weapon system must be made while requirements 

are being set and early in product development.



DODï¿½s prevailing practices run counter to achieving high reliability. 

Often, DOD does not make readiness or operating cost performance 

parameters equal in importance to others when it establishes 

requirements for weapons systems. Further, reliability growth during 

product development is hampered by immature technologies and delays in 

gaining knowledge about the productï¿½s design. Finally, DOD does not 

have sufficient knowledge about its fielded systems to inform its 

product development process for new systems. DOD is at a crossroad in 

this regard. It has made positive changes to acquisition policy in 

order to change its environment. Requiring higher readiness at lower 

cost will enable DOD to take the next step, ensuring lower total 

ownership cost.



In contrast, commercial companies that are in the market for new 

capital equipment understand that they must specify and control the 

readiness and total ownership cost of a product, especially the 

operating and support costs early in development. Therefore, they 

specify how available or ready the products must be in order to carry 

out the companyï¿½s mission. Further, they set goals for operating costs 

when acquiring new equipment; they make sure they understand their own 

operating costs from data they have collected and analyzed on equipment 

they are now using. Those two goals--how available the product must be 

and how much the customer wants to spend per operating unit to support 

the equipment--are key requirements equal in importance to other 

performance characteristics that the commercial customer demands from 

the companies that develop the products. Bounded by the twin 

requirements of specific operating costs and availability, the product 

developer sets reliability goals for the components, subsystems, and 

the full system once it is integrated into a product that will satisfy 

the customerï¿½s requirements. Product developers remain focused on good 

product development practices with mature technologies, stable designs, 

and production processes that are in control. During operations they 

collect data from their customers on reliability and performance and 

use that data to predict operating and support costs for subsequent 

developments or upgrades.



DOD has initiatives underway that partially address the issue of 

controlling operating and support costs. However, without significant 

emphasis on providing a better framework for decision-making, these 

initiatives will not yield sufficient improvements. The department has 

encouraged the services to include key performance parameters in its 

newer developments such as the Joint Strike Fighter that indicate how 

long a system must perform between maintenance actions. It has moved to 

follow best practices for reducing risk from technology and achieving 

more stable designs in the Advanced Amphibious Assault Vehicle and the 

Joint Strike Fighter. However, these programs are early in development 

and it will take some time to see how reliably they perform. We believe 

that practices found at the commercial companies we visited to make 

operating and support costs and product readiness requirements equal in 

priority to other performance characteristics forces developers to 

focus on achieving high reliability and that adopting these practices 

will help DOD achieve high readiness and control total ownership cost.



Recommendations for Executive Action:



DOD should take steps to make the cost to operate and support weapon 

systems at required readiness rates a priority when setting weapon 

system requirements for an affordable weapon system and finalizing the 

design of the selected system. To do this, its requirements and 

acquisition communities must collaborate to fully understand and 

control the costs to operate and support a weapon system prior to and 

early in product development, when it is possible to have significant 

impact on those costs. In establishing requirements for a weapon 

system, the requirements community should include the costs to operate 

and support the weapon system over its life cycle and the readiness 

rate for the weapon system. To establish an affordable design for the 

weapon system, the acquisition community and acquisition programs 

should be required to accurately estimate--based on demonstrated 

component and subsystem reliability testing--that portion of the costs 

that DOD plans to spend on operations and support of the weapon system 

throughout its life cycle before the design is finalized.



With this in mind, to ensure that the userï¿½s requirements for a weapon 

system can be met with a reliable design, we recommend that the 

Secretary of Defense:



* revise the Chairman of the Joint Chiefs of Staff Instruction 3170.01B 

on the requirements generation process to include total ownership cost, 

especially operating and support cost, and weapon system readiness 

rates as performance parameters equal in priority to any other 

performance parameters for any major weapon system before beginning the 

acquisition program;



* revise the current policy governing the operation of the defense 

acquisition system (currently under revision) to require that the 

product developer establish a firm estimate of a weapon systemï¿½s 

reliability based on demonstrated reliability rates at the component 

and subsystem level no later than the end of the system integration 

phase, coinciding with the system-level critical design review, before 

proceeding into the system demonstration phase of product development; 

and at the system level no later than the full-rate production 

decision; and:



* structure its contracts for major systems acquisitions so that at 

Milestone B the product developer has incentives to ensure that proper 

trades are made between reliability and performance prior to the 

production decision. One option is to provide specific clauses in the 

development contract to address reliability growth.



Agency Comments and Our Response:



DOD partially concurred with all of our recommendations; however, for 

the most part, it found no further action was needed to lower total 

ownership cost. We disagree. We believe that if DOD takes no further 

action in implementing these recommendations, it ignores significant 

opportunities to improve readiness and lower the total ownership cost 

of its major weapon systems. The current budget environment demands 

more effort in reducing these costs.



The performance of weapon systems as described in this report is 

evidence that they demand much more money than planned to remain ready. 

DOD should consider each of these recommendations as parts of a whole 

solution for its ï¿½death spiralï¿½--that is, the inability to modernize 

its forces because the cost to operate and maintain unreliable weapon 

systems at needed readiness rates constantly impinges on its 

modernization budget. Taken as a whole, our recommendations encourage 

DODï¿½s requirement setters to demand readiness at an affordable cost as 

a part of a systemï¿½s performance, provide a mechanism to hold the 

product developer accountable for determining the reliability needed to 

satisfy DODï¿½s requirements, and provide contractual incentives for the 

product developer to build reliability into a weapon system very early 

in its development. The details of DODï¿½s response to each 

recommendation are summarized below along with our rebuttal.



In a response prepared by the Office of the Joint Chiefs of Staff, DOD 

partially concurred with our recommendation to include readiness and 

total ownership cost as performance parameters equal in priority to any 

others before beginning an acquisition program, commenting that they 

are currently equal in priority to ï¿½non-key performance parameters.ï¿½ We 

are concerned that DOD does not recognize the importance of requiring 

targets for a systemï¿½s readiness and its total ownership cost before 

beginning product development. These targets are critical to providing 

a realistic goal for the product developer to deliver reliable, cost-

efficient weapon systems.



We examined five deployed weapon systems for this report. None had key 

requirements for readiness rates or operating costs. All had 

significant problems with reliability and, therefore, readiness and 

total ownership cost. We also reviewed several commercial products that 

were developed with readiness and total ownership cost as critical 

requirements. In each case, the products were ready to perform when 

needed at affordable and predictable cost. Unless these requirements 

are equal in importance to any others, they will not withstand the 

pressures of an acquisition program. The Joint Chiefs stated that its 

requirements generation policy is currently under revision and that no 

decision has been made about the priority readiness and cost should 

have as requirements. We believe DOD has an excellent opportunity to 

finally lower the total ownership cost of current and future weapon 

acquisitions, thereby freeing significant funds for modernization if it 

implements this recommendation.



In a response prepared by the Office of the Undersecretary of Defense 

for Acquisition, Technology, and Logistics, DOD partially concurred 

with our second recommendation to establish estimates of a weapon 

systemï¿½s reliability--first, based on demonstrated reliability rates at 

the component and subsystem level by the end of system integration, 

coincident with the critical design review and next, at the system 

level at the time the production decision is made--however, DOD found 

no need to revise the policy governing the defense acquisition system 

to achieve this. We disagree. Demonstrating reliability during product 

development has not received the priority it requires if DOD is to have 

a realistic opportunity to reduce the total ownership cost of its 

weapons while maintaining required readiness levels. The current policy 

does not provide a mechanism to ensure consistent application of 

reliability estimates based on demonstrated performance. We believe 

that the disparity between the actual costs to operate and maintain 

weapon systems and what DOD had estimated those costs to be during the 

weapon systemsï¿½ development, as described in the report, provides 

strong evidence that product developers do not understand reliability 

under DODï¿½s current process. Implementation of this recommendation will 

assist DOD in requiring reliability estimates on a consistent basis.



In a response prepared by Undersecretary of Defense, Acquisition, 

Technology, and Logistics to our recommendation that DOD structure its 

development contracts to include requirements to provide incentives for 

product developers to trade performance for reliability when it makes 

sense, DOD found no need for additional incentives to contractors 

beyond giving them total system performance responsibility. We have yet 

to see evidence that total system performance responsibility has 

provided an incentive for any product developer to trade performance 

for reliability in order to reduce total ownership cost. Further, we 

believe a contractual agreement similar to those we found in commercial 

cases--such as financial penalties for readiness below certain 

specified rates--would provide an excellent incentive for product 

developers to gain the knowledge required to meet reliability rates 

early in a weapon systemï¿½s design before committing to production.



DOD provided some technical comments in attachment 2. We have addressed 

those in the report as necessary. The full text of the departmentï¿½s 

response to the recommendations is provided in appendix I.



[End of section]



Related GAO Products:



Best Practices: Capturing Design and Manufacturing Knowledge 

Early Improves Acquisition Outcomes. GAO-02-701. Washington, D.C.: 

July 15, 2002.



Defense Acquisitions: DOD Faces Challenges in Implementing Best 

Practices. GAO-02-469T. Washington, D.C.: February 27, 2002.



Best Practices: Better Matching of Needs and Resources Will Lead 

to Better Weapon System Outcomes. GAO-01-288. Washington, D.C.: 

March 8, 2001.



Defense Acquisitions: Higher Priority Needed for Army Operating and 

Support Cost Reduction Efforts. GAO/NSIAD-00-197. Washington, D. C.: 

September 29, 2000.



Defense Acquisitions: Air Force Operating and Support Costs Reductions 

Need Higher Priority. GAO/NSIAD-00-165. Washington, D. C.: August 29, 

2000.



Best Practices: A More Constructive Test Approach Is Key to 

Better Weapon System Outcomes. GAO/NSIAD-00-199. Washington, D.C.: 

July 31, 2000.



Defense Logistics: Actions Needed to Enhance Success of Reengineering 

Initiatives. GAO/NSIAD-00-89. Washington, D. C.: June 23, 2000.



Defense Acquisition: Employing Best Practices Can Shape Better Weapon 

System Decisions. GAO/T-NSIAD-00-137. Washington, D.C.: April 26, 

2000.



Best Practices: DOD Training Can Do More to Help Weapon System Programs 

Implement Best Practices. GAO/NSIAD-99-206. Washington, D.C.: 

August16, 1999.



Best Practices: Better Management of Technology Development Can Improve 

Weapon System Outcomes. GAO/NSIAD-99-162. Washington, D.C.: July 30, 

1999.



Defense Acquisitions: Best Commercial Practices Can Improve Program 

Outcomes. GAO/T-NSIAD-99-116. Washington, D.C.: March 17, 1999.



Defense Acquisition: Improved Program Outcomes Are Possible. GAO/

T-NSIAD-98-123. Washington, D.C.: March 18, 1998.



Best Practices: DOD Can Help Suppliers Contribute More to Weapon System 

Programs. GAO/NSIAD-98-87. Washington, D.C.: March 17, 1998.



Best Practices: Successful Application to Weapon Acquisition Requires 

Changes in DODï¿½s Environment. GAO/NSIAD-98-56. Washington, D.C.: 

February 24, 1998.



Major Acquisitions: Significant Changes Underway in DODï¿½s Earned Value 

Management Process. GAO/NSIAD-97-108. Washington, D.C.: May 5, 1997.



Best Practices: Commercial Quality Assurance Practices Offer 

Improvements for DOD. GAO/NSIAD-96-162. Washington, D.C.: 

August 26, 1996.



[End of section]



Appendix I: Comments from the Department of Defense:



ACQUISITION, TECHNOLOGY AND LOGISTICS:



OFFICE OF THE UNDER SECRETARY OF DEFENSE:



3000 DEFENSE PENTAGON WASHINGTON, DC 20301-3000:



DEC 2002:



Ms. Katherine V. Schinasi:



Director, Acqusition and Sourcing Management U.S. General Accounting 

Office:



441 G Street, N. W. Washington, D.C. 20548:



Dear Ms. Schinasi:



This is the Department of Defense (DoD) response to the GAO draft 

report, ï¿½BEST PRACTICES: Setting Requirements for Cost and Readiness 

Could Reduce Weapon Systemsï¿½ Total Ownership Costs,ï¿½ dated November 26, 

2002 (GAO Code 120092/GAO-03-057). There are two enclosures. The first 

contains the DoD detailed comments to the recommendations. The second 

are additional comments on the report. The DoD response to 

Recommendation 1 was provided by the Joint Chiefs of Staff.



Sincerely,



Spiros G. Pallas, Principal Deputy, Strategic and Tactical Systems:



Signed by Spiros G. Pallas:



Enclosure(s): As stated:



ENCLOSURE (1):



GAO DRAFT REPORT - DATED NOVEMBER 26, 2002 GAO CODE 120092/GAO-03-057:



ï¿½BEST PRACTICES: Setting Requirements for Cost and Readiness Could 

Reduce Weapon Systemsï¿½ Total Ownership Costsï¿½:



DEPARTMENT OF DEFENSE COMMENTS TO THE RECOMMENDATIONS:



RECOMMENDATION 1: The GAO recommended that the Secretary of Defense 

revise the Chairman of the Joint Chiefs of Staff Instruction 3170.01 on 

the requirements generation process to include total ownership cost, 

especially operating and support cost, and weapon system readiness 

rates as performance parameters equal in priority to any other 

performance parameters for any major weapon system prior to beginning 

the acquisition program. (p. 66/GAO Draft Report):



DoD RESPONSE: Partially Concur. The Operational Requirements Document 

currently addresses program affordability, stated in terms of a 

performance parameter threshold and objective, and system performance 

parameters, including mission reliability; they are equal in priority 

to any other ï¿½non-KPPï¿½ performance parameter. The supporting guidance 

in CJCSI 3170.01 is currently under revision-the Department has not 

decided whether to clarify it with a more specific description towards 

weapon system readiness rates.



RECOMMENDATION 2: The GAO recommended that the Secretary of Defense 

revise the current policy governing the operation of the defense 

acquisition system to require that the product developer establish a 

firm estimate of a weapon systemï¿½s reliability based on demonstrated 

reliability rates at the component and subsystem level no later than 

the end of the system integration phase, coinciding with the system-

level critical design review, before proceeding into the system 

demonstration phase of product development; and at the system level no 

later than the full-rate production decision. This policy is currently 

under revision, to be finalized in early 2003. (p. 66/GAO Draft 

Report):



DoD RESPONSE: Partially Concur: While there is no formal direction to 

provide a firm estimate of reliability during critical design review, 

program system performance is reviewed for the final design 

configuration, including compliance to any reliability requirement or 

specification included in the contract. Because this is already 

accomplished, there is no need to further direct the recommendation.



RECOMMENDATION 3: The GAO recommended that the Secretary of Defense 

structure its contracts for major systems acquisitions at Milestone B 

to provide incentives for the product developer to ensure that proper 

trades are made between reliability and performance prior to the 

production decision. One option is to provide specific clauses in the 

development contract to address reliability growth. (p. 66/GAO Draft 

Report):



DoD RESPONSE: Partially Concur: DoD views reliability as part of system 

performance. Also, DoD already encourages system design trades 

throughout development. Incentivizing reliability growth is one of many 

possible approaches already used in some cases. There is no need for 

additional emphasis by DoD. Other approaches used by DoD include total 

system support responsibility (TSSR) which automatically incentivizes 

contractors to provide reliable systems while increasing their profits.



[End of section]



Appendix II GAO Staff Acknowledgments:



Acknowledgments:



Cheryl Andrew, Beverly Breen, Belinda LaValle, Carol Mebane, Gary 

Middleton, Michael Sullivan, Adam Vodraska, and Earl C. Woodard:



FOOTNOTES



[1] An open system is one that is designed with interfaces to accept 

upgrades easily without redesign of the total unit. Replacements in an 

open system only have to meet interface requirements to be accepted.



[2] A key performance parameter represents a capability that is so 

significant that failure to meet the minimum value could be a reason 

for DOD or the services to reevaluate the concept or system or 

terminate the program.



[3] Operations and support of weapons systems is a part of the 

Operations and Maintenance budget, which also includes amounts for 

health care, base and facilities support, and other activities for the 

well-being and operations of the military forces. Costs for operations 

and support of weapon systems were about 48 percent of the Operations 

and Maintenance budget in fiscal year 2002.



[4] The operating and support cost for the Apache is not available 

before 1993.



[5] The operating and support cost for the M-1 series Abram is not 

available before 1993.



[6] U.S. General Accounting Office, Best Practices: Better Matching of 

Needs and Resources Will Lead to Better Weapon System Outcomes, GAO-01-

288 (Washington, D.C.: Mar. 8, 2001).



[7] U.S. General Accounting Office, Best Practices: Capturing Design 

and Manufacturing Knowledge Early Improves Acquisition Outcomes, GAO-

02-701 (Washington, D.C.: July 15, 2002).



[8] Chairman of the Joint Chiefs of Staff Instruction, Requirements 

Generation System, (Washington, D.C.: Apr. 15, 2001).



[9] GAO-01-288.



[10] A threshold is the minimum acceptable operational combat 
capability 

required to meet war-fighter minimum requirements. An objective is the 

capability desired of the system beyond minimum requirements.



[11] U.S. General Accounting Office, Joint Strike Fighter Acquisition: 

Mature Critical Technologies Needed to Reduce Risks, GAO-02-39 

(Washington, D.C.: Oct. 19, 2001).



[12] Department of Defense, Office of the Inspector General, Use of An 

Open Systems Approach for Weapon Systems, Report No. D-2000-149 

(Washington, D.C.: June 14, 2000).



GAOï¿½s Mission:



The General Accounting Office, the investigative arm of Congress, 

exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability 

of the federal government for the American people. GAO examines the use 

of public funds; evaluates federal programs and policies; and provides 

analyses, recommendations, and other assistance to help Congress make 

informed oversight, policy, and funding decisions. GAOï¿½s commitment to 

good government is reflected in its core values of accountability, 

integrity, and reliability.



Obtaining Copies of GAO Reports and Testimony:



The fastest and easiest way to obtain copies of GAO documents at no 

cost is through the Internet. GAOï¿½s Web site ( www.gao.gov ) contains 

abstracts and full-text files of current reports and testimony and an 

expanding archive of older products. The Web site features a search 

engine to help you locate documents using key words and phrases. You 

can print these documents in their entirety, including charts and other 

graphics.



Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as ï¿½Todayï¿½s Reports,ï¿½ on its 

Web site daily. The list contains links to the full-text document 

files. To have GAO e-mail this list to you every afternoon, go to 

www.gao.gov and select ï¿½Subscribe to daily E-mail alert for newly 

released productsï¿½ under the GAO Reports heading.



Order by Mail or Phone:



The first copy of each printed report is free. Additional copies are $2 

each. A check or money order should be made out to the Superintendent 

of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. 

Orders should be sent to:



U.S. General Accounting Office



441 G Street NW,



Room LM Washington,



D.C. 20548:



To order by Phone: 	



	Voice: (202) 512-6000:



	TDD: (202) 512-2537:



	Fax: (202) 512-6061:



To Report Fraud, Waste, and Abuse in Federal Programs:



Contact:



Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: [email protected]



Automated answering system: (800) 424-5454 or (202) 512-7470:



Public Affairs:



Jeff Nelligan, managing director, [email protected] (202) 512-4800 U.S.



General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.



20548: