Defense Management: DOD Needs to Demonstrate That		 
Performance-Based Logistics Contracts Are Achieving Expected	 
Benefits (09-SEP-05, GAO-05-966).				 
                                                                 
The Department of Defense (DOD) contracts with private sector	 
companies to perform depot maintenance of weapon systems using	 
performance-based logistics--that is, purchasing a defined level 
of performance over a defined time period at a fixed cost to the 
government. After implementing such contracts, program offices	 
are to validate their efficacy using cost and performance data;  
DOD cannot otherwise ensure cost savings and improved performance
are being achieved through the use of performance-based 	 
logistics. GAO was asked to review the implementation of	 
performance-based logistics to determine whether DOD could	 
demonstrate cost savings and improved responsiveness from these  
arrangements. In conducting its review, GAO analyzed the	 
implementation of performance-based logistics arrangements for 15
weapon system programs. 					 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-05-966 					        
    ACCNO:   A36220						        
  TITLE:     Defense Management: DOD Needs to Demonstrate That	      
Performance-Based Logistics Contracts Are Achieving Expected	 
Benefits							 
     DATE:   09/09/2005 
  SUBJECT:   Contract administration				 
	     Contract performance				 
	     Cost effectiveness analysis			 
	     Department of Defense contractors			 
	     Equipment maintenance				 
	     Internal controls					 
	     Logistics						 
	     Weapons systems					 
	     Performance-based contracting			 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-05-966

     

     * Report to the Subcommittee on Readiness and Management Support,
       Committee on Armed Services, U.S. Senate
          * September 2005
     * defense management
          * DOD Needs to Demonstrate That Performance-Based Logistics
            Contracts Are Achieving Expected Benefits
     * Contents
          * Results in Brief
          * Background
          * DOD Program Offices Could Not Demonstrate Cost Savings and
            Performance Improvement Resulting from the Use of
            Performance-Based Logistics
               * Projected Cost Savings and Performance Improvements Were Not
                 Validated
               * DOD Program Offices Relied on Contractors' Data Without
                 Verifying the Reliability of the Data
               * DOD Program Offices Made Limited Use of Resources Available
                 to Assist in Monitoring Performance- Based Logistics
                 Contracts
          * Conclusions
          * Recommendations for Executive Action
          * Agency Comments and Our Evaluation
          * Scope and Methodology
     * Comments from the Department of Defense
     * GAO Contact and Staff Acknowledgments

                 United States Government Accountability Office

Report to the Subcommittee on

Readiness and Management Support, Committee on Armed Services, U.S. Senate

September 2005

DEFENSE MANAGEMENT

    DOD Needs to Demonstrate That Performance-Based Logistics Contracts Are
                          Achieving Expected Benefits

                                       a

GAO-05-966

DEFENSE MANAGEMENT

DOD Needs to Demonstrate That Performance-Based Logistics Contracts Are
Achieving Expected Benefits

  What GAO Found

DOD program offices could not demonstrate that they have achieved cost
savings or performance improvements through the use of performance-based
logistics arrangements. Although DOD guidance on implementing these
arrangements states program offices should update their business case
analysis based on actual cost and performance data, only 1 of the 15
program offices included in GAO's review had performed such an update
consistent with DOD guidance. In the single case where the program office
had updated its business case analysis, it determined that the
performance-based logistics contract did not result in expected cost
savings and the weapon system did not meet established performance
requirements. In general, program offices had not updated their business
case analysis after entering into a performance-based logistics contract
because they assumed that the costs for weapon system maintenance incurred
under a fixed-price performance-based logistics contract would always be
lower than costs under a more traditional contracting approach and because
they lacked reliable cost and performance data needed to validate
assumptions used. Furthermore, the Office of the Secretary of Defense has
not established procedures to monitor program offices to ensure they
follow guidance and update the business case analysis. Additionally,
program officials said because of limitations in their own information
systems, they typically relied on cost and performance data generated by
the contractors' information systems to monitor performance-based
logistics contracts. The program offices, however, had not determined
whether contractor-provided data were sufficiently reliable to update
their business case analysis. Although the Defense Contract Management
Agency and the Defense Contract Audit Agency are most commonly used to
monitor higher risk contracts, such as cost plus contracts, they are
potential resources available to assist program offices in monitoring
fixed-price performance-based contracts. In doing so, these DOD agencies
have the capability to verify the reliability of contractors' information
systems and collect cost and performance data needed to update their
business case analysis. Until program offices follow DOD's guidance and
update their business case analysis based on reliable cost and performance
data, DOD cannot evaluate the extent to which performance-based logistics
arrangements are achieving expected benefits and being effectively
implemented within DOD.

                 United States Government Accountability Office

Contents

Letter                   Results in Brief Background DOD Program     1 3 4 
                            Offices Could Not Demonstrate Cost Savings   7 11 
                            and Performance Improvement Resulting from  12 12 
                            the Use of Performance-Based Logistics         13 
                            Conclusions Recommendations for Executive  
                            Action Agency Comments and Our Evaluation  
                            Scope and Methodology                      
Appendixes Appendix I:   Comments from the Department of Defense           
              Appendix II:  GAO Contact and Staff Acknowledgments      16 18
Table                    Table 1:   Performance-Based Logistics            
                                       Programs Included in Our Review     13

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

A

United States Government Accountability Office Washington, D.C. 20548

September 9, 2005

The Honorable John Ensign Chairman The Honorable Daniel K. Akaka Ranking
Minority Member Subcommittee on Readiness and Management Support Committee
on Armed Services United States Senate

The Department of Defense (DOD) reported that it spent $23.6 billion in
fiscal year 2004 for depot maintenance of military weapon systems.1 Almost
half of those expenditures were paid to private sector companies to
perform depot maintenance activities for selected weapon systems. DOD's
preferred approach for providing long-term total system support for weapon
systems is a concept it calls performance-based logistics.
Performance-based logistics is a variation of other contractor logistics
support strategies and involves defining a level of performance that the
weapon system is to achieve over a period of time at a fixed cost to the
government.2 Specific performance measures are established in each
contract. DOD believes this contracting strategy for supporting weapon
systems and components will achieve higher levels of performance while
minimizing costs. For example, DOD has claimed millions of dollars in cost
savings and significant performance improvements in such areas as material
availability and logistics response time through the use of
performance-based logistics. As a result of aggressively pursuing this
contracting strategy, DOD expected an increasing number of weapon systems
and components to be maintained for the military services by private
sector companies under long-term, fixed-price, performance-based
contracts. To justify the use of a performance-based logistics strategy,
according to DOD guidance, program offices should conduct a business case
analysis prior to adopting this approach. Following implementation of a
performance-based logistics contract, program offices should update their
business case analysis to validate the assumptions used based on

1Depot maintenance encompasses materiel maintenance or repair requiring
the overhauling, upgrading, or rebuilding of parts, assemblies, or
subassemblies and the testing and reclamation of equipment.

2Performance-based logistics arrangements for weapon system support can
include either a contract with a private sector company or an arrangement
to use a government facility.

actual cost and performance data and to support future weapon systems
plans.

As requested, we reviewed DOD's implementation of performance-based
logistics arrangements. Our objective was to determine whether DOD could
demonstrate cost savings or improved responsiveness through the use of
performance-based logistics arrangements. This is the second of two
reports addressing DOD's implementation of performance-based logistics. In
an August 2004 report,3 we identified the types of contractor logistics
support arrangements the private sector used when outsourcing the
maintenance of complex and costly equipment with life-cycle management
issues similar to those of military systems. We also identified several
lessons learned from comparisons between private sector contractor support
practices and the performance-based logistics approach that DOD was urging
the services to use. That report recommended that DOD incorporate into
guidance the private sector's practice of using performance-based
contracts to achieve economies at the component level, rather than as a
preferred practice at the platform level. DOD issued policy memoranda and
revised its guidebook to implement our recommendation.

As part of our current review, we collected and analyzed data on 15 weapon
system programs that were among those identified by the Office of the
Secretary of Defense and the military services as programs that have
successfully used performance-based logistics arrangements. We selected
weapon system programs that were sufficiently mature to provide lessons
learned on the use of performance-based logistics. We reviewed contracts
and related files and interviewed program officials to determine how these
arrangements were structured and managed. We examined business case
analysis documents prepared by the program offices to justify the use of a
performance-based logistics strategy. We discussed with program officials
the systems they used to monitor contractor cost and performance. We also
reviewed DOD policies, procedures, and guidance related to the use of
performance-based logistics for weapon system support. We determined that
the data used were sufficiently reliable for our purposes. We performed
our work from August 2004 through June 2005 in accordance with generally
accepted government auditing standards. The Scope and

3GAO, Defense Management: Opportunities to Enhance the Implementation of
Performance-Based Logistics, GAO-04-715 (Washington, D.C.: Aug. 16, 2004).

Page 2 GAO-05-966 Performance-Based Logistics

                                Results in Brief

Methodology section contains more detailed information about the work we
performed.

DOD program offices could not demonstrate that they have achieved cost
savings or performance improvements through the use of performance-based
logistics arrangements. Although DOD guidance on implementing these
arrangements states program offices should update their business case
analysis based on actual cost and performance data, only 1 of the 15
program offices included in our review had performed such an update
consistent with DOD guidance. In the single case where the program office
had updated its business case analysis, it determined that the
performance-based logistics contract did not result in expected cost
savings and the weapon system did not meet established performance
requirements. In general, program offices had not updated their business
case analysis after entering into a performance-based logistics contract
because they assumed that the costs for weapon system maintenance incurred
under a fixed-price, performance-based logistics contract would always be
lower than costs under a more traditional contracting approach and because
they lacked reliable cost and performance data needed to validate
assumptions used. Furthermore, the Office of the Secretary of Defense has
not established procedures to monitor program offices to ensure they
follow guidance and update the business case analysis. Additionally,
program officials said because of limitations in their own information
systems, they typically relied on cost and performance data generated by
the contractors' information systems to monitor performance-based
logistics contracts. The program offices, however, had not determined
whether the contractor-provided data were sufficiently reliable to update
their business case analysis. Although the Defense Contract Management
Agency (DCMA) and the Defense Contract Audit Agency (DCAA) are most
commonly used to monitor higher risk contracts, such as cost plus
contracts, they are potential resources available to assist program
offices in monitoring fixed-price performance-based contracts. In doing
so, these DOD agencies have the capability to verify the reliability of
contractors' information systems and collect cost and performance data
needed to update the business case analysis. Until program offices follow
DOD's guidance and update their business case analysis based on reliable
cost and performance data, DOD cannot evaluate the extent to which
performance-based logistics arrangements are achieving expected benefits
and being effectively implemented within DOD.

                                   Background

This report contains recommendations aimed at ensuring that DOD program
offices follow DOD guidance for updating their business case analyses and
verify the reliability of contractor cost and performance data. In
commenting on a draft of this report, DOD concurred with our findings and
recommendations. DOD's response is included in appendix I.

DOD has increasingly relied on contractors to provide logistics support
for weapon system maintenance. These logistics support arrangements have
taken various forms. In fiscal year 1998, DOD directed the armed services
to pursue logistics support "reengineering" efforts with contractors to
achieve cost savings and improve efficiency.4 A 1999 DOD study identified
30 pilot programs to test logistics support concepts that placed greater
reliance on the private sector. Some of the pilot programs involved
performance-type arrangements that were subsequently converted to, or
designated as, performance-based logistics contracts.5 DOD's Quadrennial
Defense Review Report6 advocated the implementation of performance-based
logistics, with appropriate metrics, to compress the supply chain7 by
removing steps in the warehousing, distribution, and order fulfillment
processes; reducing inventories; and reducing overhead costs while
improving the readiness of major weapon systems and commodities.

Over the last few years, DOD has issued guidance on the implementation of
performance-based logistics. In November 2001, the Office of the Deputy
Under Secretary of Defense issued guidance recommending that program
managers conduct a sound business case analysis to decide whether they

4Reengineering was the term DOD used to describe its efforts to increase
the cost-effectiveness of its weapon systems throughout their life cycles
by ensuring that support infrastructures were competitive, timely, and
efficient.

5Examples of these logistics support arrangements include (1) contractor
logistics support, where the contractor provided most or all elements of
logistics support, including depot maintenance; (2) total system
performance responsibility, under which the contractor assumed
responsibility for the weapon system's life-cycle management; and (3)
total system sustainment responsibility or total system support
responsibility, which gave the contractor responsibility for all
contracted sustainment actions including parts management and depot-level
repair.

6DOD, Quadrennial Defense Review Report, (Washington, D.C.: Sept. 30,
2001).

7Supply chain management refers to all of the interrelated components and
processes required to ensure that the correct amount of product is in the
correct location at the right time and at the lowest cost.

should implement performance-based logistics for new systems and major
acquisitions for already fielded systems.8 In an August 2003 memorandum to
the military departments, the Under Secretary of Defense (Acquisition,
Technology and Logistics) stated that DOD should continue to increase its
use of performance-based logistics acquisitions. On February 4, 2004, the
Deputy Secretary of Defense (1) directed the Under Secretary of Defense
(Acquisition, Technology and Logistics), in conjunction with the Under
Secretary of Defense (Comptroller), to issue clear guidance on purchasing
logistics support using performance criteria and (2) directed each service
to provide a plan to aggressively implement performance-based logistics
for current and planned weapon system platforms. Then, based on
recommendations in our August 2004 report,9 the Under Secretary of Defense
(Acquisition, Technology and Logistics) issued a memorandum reemphasizing
that the use of this type of support strategy was intended to optimize
weapon system availability while minimizing costs and the logistics
footprint10 and may be applied to weapon systems, subsystems, and
components. The memorandum also provided specific definitions of
performance metrics to be used.

DOD describes performance-based logistics as the process of

(1)
           identifying a level of performance required by the warfighter and

(2)
           negotiating a performance-based arrangement between the government
           and a contractor or government facility to provide long-term total
           system support for a weapon system at a fixed level of annual
           funding. Instead of buying spare parts, repairs, tools, and data
           in individual transactions, DOD program offices that use a
           performance-based logistics arrangement buy a predetermined level
           of performance that meets the warfighter's objectives. Although
           established performance measures should be tailored to reflect the
           unique circumstances of each performance-based logistics
           arrangement, the measures are expected to support five general
           objectives:

(1)
           percentage of time that a weapon system is available for a mission
           (operational availability); (2) percentage of mission objectives
           met (operational reliability); (3) operating costs divided by a
           specified unit of

8DOD, Product Support for the 21st Century: A Program Manager's Guide to
Buying Performance (Washington, D.C.: Nov. 6, 2001).

9 GAO-04-715 .

10The logistics footprint refers to the size or presence of the government
or contractor personnel, equipment, inventory, facilities, transportation
assets, or real estate needed to deploy, sustain, and move the weapon
systems or components.

Page 5 GAO-05-966 Performance-Based Logistics

measure (cost per unit usage); (4) size or presence of support required to
deploy, sustain, or move a weapon system (logistics footprint); and

(5) period of time that is acceptable between the demand or request for
support and the satisfactory fulfillment of that request (logistics
response time). Currently, a DOD task force is refining these objectives
into DOD standard performance definitions to be used by program offices in
every service when preparing performance-based logistics arrangements.

DOD guidance recommends that program offices prepare a business case
analysis prior to adopting a performance-based logistics approach to
support a weapon system.11 The aim of the business case analysis is to
justify the decision to enter into a performance-based logistics contract.
The business case analysis is to include cost savings that are projected
as a result of using a performance-based logistics approach and the
assumptions used in developing the business case analysis. Furthermore,
DOD guidance states that program offices should update their business case
analyses at appropriate decision points when sufficient cost and
performance data have been collected to validate the assumptions used in
developing the business case analyses, including the costs of alternative
approaches, projected cost savings, and expected performance levels.
Further, GAO Internal Control Standards state that it is necessary to
periodically review and validate the propriety and integrity of program
performance measures and indicators. Also, actual performance data should
be continually compared against expected or planned goals, and any
difference should be analyzed. Additionally, management should have a
monitoring strategy that emphasizes to program managers their
responsibility for internal controls (i.e., to review and validate
performance measures and indicators) and that includes a plan for periodic
evaluation of control activities.

11DOD, Performance-Based Logistics: A Program Manager's Product Support
Guide (Washington, D.C.: Sept. 30, 2004); Product Support Strategy (PSS)
Business Case Analysis (BCA) Guiding Principles, (Washington, D.C.: Jan.
23, 2004); and Business Case Development Guide (Washington, D.C.).

Page 6 GAO-05-966 Performance-Based Logistics

  DOD Program Offices Could Not Demonstrate Cost Savings and Performance
  Improvement Resulting from the Use of Performance-Based Logistics

DOD program offices could not demonstrate that their use of
performance-based logistics arrangements had achieved cost savings and
performance improvements because they had not updated their business case
analysis as suggested by DOD guidance. Specifically, of the 15 DOD program
offices, only 1 updated its business case analysis to validate assumptions
concerning cost and performance. Other DOD program offices had not updated
their business case analysis in part because they lacked reliable
contractor cost and performance data. The program offices typically relied
on cost and performance data generated by contractors' information systems
without verifying that the data were sufficiently reliable to update the
business case analysis. Two DOD agencies, DCMA and DCAA, have the
capability to assist program offices in monitoring fixed-price
performance-based contracts, verifying the reliability of contractors'
information systems, and collecting cost and performance data.

    Projected Cost Savings and Performance Improvements Were Not Validated

None of the 15 program offices included in our review could demonstrate
that use of a performance-based logistics arrangement had achieved cost
savings and performance improvements. Although an updated business case
analysis based on actual cost and performance data might show that cost
savings and performance improvements were being achieved, only 1 of the 15
program offices had updated its business case analysis consistent with DOD
guidance. Of the 15 program offices, 11 had developed a business case
analysis prior to entering into a performance-based logistics
arrangement.12 In their analysis, these program offices projected that
they would achieve significant cost savings. For example, an

12Four of the 15 program offices we reviewed had not prepared a business
case analysis. Officials at two program offices stated that their
contracting arrangements did not meet the existing definition of a
performance-based logistics contract and, therefore, the requirement to
conduct a business case analysis was not applicable. Subsequently, the
Under Secretary of Defense (Acquisition, Technology and Logistics) issued
guidance on purchasing weapon systems logistics support using
performance-based criteria and clarified the types of contract
arrangements to be considered as performance-based logistics arrangements.
DOD determined that under these clarified criteria, these two contracts
meet the definition of a performance-based logistics contract. The other
two program offices determined that they lacked sufficient cost and
performance information necessary to prepare business case analyses. To
collect baseline cost and performance data, they entered into cost-plus
contracts rather than fixed-price contracts. On the basis of an assessment
of baseline data collected in the course of the cost-plus contracts, the
program offices expect to determine at a later time if it would be
advantageous to transition from the cost-plus contract to fixed-price
performance-based logistics arrangements. Before doing so, they would
prepare a business case analysis as suggested under DOD guidance.

Army program office projected total cost savings of $508.5 million, and a
Navy program office projected cost savings of $29.7 million. However, only
the Navy's T-45 program office had subsequently updated its business case
analysis consistent with DOD guidance to determine whether cost savings
were being achieved.13 Realizing that the contractor was not meeting the
aircraft availability performance measure, the program office reassessed
its business case assumptions and found that costs per flying hour were
higher than estimated because the aircraft was flying fewer hours than
forecasted. As a result, the program office negotiated separate contracts
for the airframes and engines, which resulted in estimated cost savings of
$144 million over 5 years.

Performance indicators tracked by the program offices showed that the
contractors met or exceeded performance requirements. Of the 15 programs,
10 reported that performance levels exceeded contract requirements, and 5
reported that performance levels were meeting contract requirements. For
example, an Army program office reported a weapon system availability rate
of 99 percent, which is 7 percent higher than what was projected in the
business case analysis. Similarly, a Navy program office reported a weapon
system availability rate of 97 percent, which is 7 percent higher than
projected.

Despite the reported performance improvements, the program offices had not
analyzed the performance data to validate the improvements and determine
whether these improvements could be attributed directly to their use of
performance-based logistics arrangements to support the weapon systems. In
addition, we noted that program offices in the past reported they had also
met or exceeded required levels of performance using other contractual
arrangements for weapon system maintenance. Moreover, the DOD program
offices reporting that performance levels were exceeding contract
requirements under performance-based logistics arrangements had not
determined the incremental costs associated with achieving these higher
levels of performance. As a result, they had no way of knowing whether
incremental costs outweighed the benefits derived from achieving
performance levels in excess of requirements.

Program officials did not follow DOD guidance to update and validate their
business case analyses because they assumed that costs incurred under

13The T-45 Goshawk is a tandem-seat jet trainer aircraft used by the Navy
and Marine Corps for training in carrier aviation and tactical strike
missions.

Page 8 GAO-05-966 Performance-Based Logistics

    DOD Program Offices Relied on Contractors' Data Without Verifying the
    Reliability of the Data

fixed-price performance-based logistics arrangements would always be lower
than costs incurred under more traditional contracting arrangements, and
several program officials cited a lack of reliable data needed to validate
expected costs savings and improved performance. However, the experience
of the T-45 program showed that it is possible for program offices to
validate the assumptions in the business case analysis and to determine
whether expected cost savings and performance improvements were achieved.
There are also other benefits derived from validating the assumptions used
in the business case analysis. Validation can provide a better
understanding of costs associated with the repair and maintenance of
weapon systems, ensure that proper performance metrics are in place to
satisfy logistical demand, isolate incremental costs associated with
achieving higher levels of performance, and make cost and performance data
available for contract renegotiations in order to obtain the best value
for the government. Furthermore, we did not find evidence that the Office
of the Secretary of Defense had established procedures to monitor whether
program offices were following its guidance to update their business case
analyses. The results of these updates could be used by DOD to assess the
implementation of performance-based logistics arrangements and evaluate
the extent to which performance-based logistics arrangements are achieving
expected benefits.

DOD program offices included in our review stated that because of
limitations in their own information systems, they typically relied on
cost and performance data generated by the contractors' information
systems to monitor performance-based logistics contracts. Program offices
acknowledged limitations in their own information systems in providing
reliable data to closely monitor contractor cost and performance. Existing
systems are capable of collecting some cost and performance information on
performance-based logistics contracts; however, according to program
officials, the systems are not capturing sufficiently detailed cost and
performance information for monitoring performance-based logistics
contracts.14 Program officials told us they had more confidence in the
accuracy and completeness of contractor systems than in their legacy
systems. The program offices, however, had not determined whether the
contractor-provided data were sufficiently reliable to update their
business case analyses. As a result, the program offices did not have the
reliable data they needed to validate the assumptions used in the business
case analysis and to determine whether their performance-based logistics
arrangements were achieving expected cost savings and improved
performance.

As we noted in a prior report on DOD's management of depot maintenance
contracting,15 to reduce personnel and save costs, DOD decided to rely
more on contractors to manage and oversee fixed-price contracts because
these contracts are considered low risk. The contractor assumes most of
the risks for fixed-priced contracts, with the government taking a more
limited role in monitoring these contracts. In our prior work on defense
contract management, we discussed the importance of monitoring
contractors' systems to ensure the accuracy and completeness of
information generated by these systems.16 In addition, during our review
of the private sector's use of performance-based logistics,17 we noted
that private-sector companies that use performance-based logistics
contracts, whether fixed price or cost-plus, closely monitor cost and
performance information to effectively manage their contracts. These
companies said they rely on their own systems and personnel to verify the
cost and quality of work performed by the contractor. The private sector
takes this approach (1) to ensure that expected costs under the contracts
are accurate and meet the company's reliability standards, (2) to validate
the business case decision used to justify a performance-based logistics
arrangement, and (3) to obtain the data necessary to renegotiate the
contract.

14In a previous report, we found that despite a significant investment in
business systems, DOD continues to have long-standing financial and
business management problems that preclude the department from producing
reliable and timely information for making decisions. See GAO, DOD
Business System Modernization: Billions Being Invested without Adequate
Oversight, GAO-05-381 (Washington, D.C.: April 29, 2005).

15GAO, Defense Depot Maintenance: Contracting Approaches Should Address
Workload Characteristics, GAO/NSIAD-98-130 (Washington, D.C.: June 15,
1998).

16GAO, High Risk Series: Defense Contract Management, GAO/HR-97-4
(Washington, D.C.: Feb. 1997).

17See GAO-04-715 .

    DOD Program Offices Made Limited Use of Resources Available to Assist in
    Monitoring Performance-Based Logistics Contracts

DCMA and DCAA have the capability to monitor contractor cost and
performance, verify the reliability of contractor-provided data, and
collect detailed cost and performance data. However, most of the DOD
program offices we reviewed made limited use of these agencies' resources
because they viewed fixed-price performance-based logistics contracts to
be low risk compared with other types of contracts. Before a contract is
awarded, DCMA can provide advice and service to help construct effective
solicitations, identify potential risk, select the most capable
contractors, and write contracts that meet the needs of DOD customers.
After the contract is awarded, DCMA can monitor contractors' information
systems to ensure that cost, performance, and delivery schedules are in
compliance with the terms and conditions of the contracts. DCAA performs
contract audits for DOD components and provides accounting and financial
advisory services during contract negotiation and administration of
contracts.

DCMA and DCAA officials said that they have a greater role in monitoring
cost information for cost-plus contracts because such contracts are
considered high risk. According to DCMA and DCAA officials, their level of
oversight is significantly less for fixed-priced contracts, including
performance-based logistics arrangements, because DOD considers these
contracts to be low risk, thereby diminishing the need for monitoring
contractor performance. Without a request from program offices or specific
contract clauses, DCMA and DCAA generally would not conduct periodic
reviews or audits of fixed-price contracts to verify cost and performance
information. DCMA and DCAA officials also said that in the past,
monitoring fixed-price contracts was included in their workload, but
because of a reduction in staff and streamlining of operations, they
focused their efforts on contract areas that have the highest risk for
cost growth. DCMA and DCAA officials said they would support increasing
their role in monitoring fixed-price performance-based contracts depending
on the availability of their resources.

                                  Conclusions

DOD is expanding its use of performance-based logistics as its preferred
support strategy in support of weapon systems but has not yet demonstrated
that this long-term support strategy is being effectively implemented
DOD-wide. DOD guidance states that program offices, after entering into
performance-based logistics arrangements, should update their original
business case analysis using actual cost and performance data to validate
their assumptions, but most of the program offices we reviewed had not
followed this guidance, and the Office of the Secretary of Defense was not
monitoring whether program offices were following the guidance. The
program offices therefore could not substantiate that cost savings and
performance improvements for weapon system support were being achieved
through the use of performance-based logistics arrangements. Program
offices also have lacked reliable cost and performance data needed to
validate the results of performance-based logistics arrangements. Reliable
data could be collected and analyzed by increasing oversight of these
contracts with the assistance of DCMA and DCAA.

To demonstrate that performance-based logistics arrangements are

  Recommendations for

resulting in reduced costs and increased performance, and to improve

oversight of performance-based logistics contracts, we recommend that the
Secretary of Defense direct the Under Secretary of Defense (Acquisition,
Technology and Logistics) to take the following two actions:

 report, DOD concurred with our
recommendations regarding the validation of business case decisions for
performance-based logistics arrangements and verification of reliability
of contractor data. While DOD was generally responsive to our
recommendations, specific details on how DOD planned to validate and
verify contractor data were not provided.

Regarding our recommendation to reaffirm guidance and develop procedures
to track whether program offices validate their business case

  Scope and Methodology

decisions, DOD stated that the department will reaffirm DOD guidance on
updating the business case analysis after implementing performance-based
logistics arrangements and will work with the military services to develop
procedures to track whether program offices validate their business case
decisions consistent with DOD guidance.

With regard to our second recommendation to direct program offices to
verify the reliability of contractor cost and performance data, DOD stated
that it will issue guidance on verifying the reliability of contractor
cost and performance data. DOD did not provide specific information on
what the guidance would include nor did it indicate whether it would
increase the use of DCMA or DCAA to verify the reliability of contractor
cost and performance data.

DOD also provided technical comments, which we have incorporated as
appropriate.

To determine whether DOD could demonstrate that cost savings and improved
performance were being achieved through the use of performance-based
logistics arrangements, we collected and analyzed data on 15 weapon system
programs identified by the Office of the Secretary of Defense and the
military services as programs that have successfully used
performance-based logistics arrangements. The 15 programs are listed in
table 1.

      Table 1: Performance-Based Logistics Programs Included in Our Review

Weapon system or 
Military service       component       Program office                      
Air Force        C-17                  Air Force Materiel Command          
Air Force        F-117                 Air Force Materiel Command          
Air Force        JSTARS                Warner Robins Air Logistics Center  
Air Force        C-130J                Air Force Materiel Command          
Navy                  ALR-67 (V3)      Naval Supply Systems Command        
Navy             Auxiliary Power Units    Naval Supply Systems Command and 
                                                    Naval Air Systems Command 
Navy                F-18 E/F FIRST        Naval Supply Systems Command and 
                                                    Naval Air Systems Command 
Navy             F-404                    Naval Supply Systems Command and 
                                                    Naval Air Systems Command 
Navy                 T-45 engines         Naval Supply Systems Command and 
                                                    Naval Air Systems Command 

(Continued From Previous Page)

Weapon system or Military service component Program office

                 Navy V-22 engines a Naval Air Systems Command

 Navy/Marine Corps KC-130J b Naval Supply Systems Command and Naval Air Systems
                                    Command

                    Army HIMARS Aviation and Missile Command

                 Army Javelin CLU Aviation and Missile Command

                   Army TOW-ITAS Aviation and Missile Command

                 Army TUAV Shadow Aviation and Missile Command

Source: GAO analysis of services' data.

a This is a joint contract with the Navy and Marine Corps. b The Marine
Corps' maintenance contracting was handled by a Navy team.

We reviewed DOD and service policies, procedures, and guidance related to
the use of performance-based logistics and met with program officials to
discuss how their performance-based logistics contracts were structured
and managed and how these contracts were validated to ensure that cost
savings and improved performance were being achieved as a result of using
performance-based logistics. We also obtained and analyzed available
documentation, including business case analyses, contracts, and related
files. We did not assess the methodology program offices used to prepare
their business case analyses or the quality of these analyses. We
discussed with program officials the systems they used to monitor
contractor cost and performance. We also interviewed officials from the
Office of the Secretary of Defense and military department headquarters to
discuss implementation of performance-based logistics, lessons learned,
and the benefits derived from using performance-based logistics approaches
and practices. To determine how private-sector companies ensure that cost
and performance levels under a performance-based contract are as expected,
we reviewed the information provided by seven companies identified in our
prior report that used complex and costly equipment that had life-cycle
management issues similar to military weapon systems, and outsourced some
portion of their maintenance work under performance-based contracts. These
seven companies consisted of six airline companies and one mining company.
We contacted officials at DCMA and DCAA to determine those agencies' roles
in monitoring the costs and performance of fixed-priced contracts,
including performance-based logistics contracts, how audits are requested
or initiated, and the procedures for reporting the results of the audits.

We are sending this report to the Chairman and Ranking Minority Member,
Senate Subcommittee on Readiness and Management Support, Committee on
Armed Services. We will also send copies to the Under Secretary of Defense
(Acquisition, Technology and Logistics). Copies of this report will be
made available to others upon request. In addition, the report will be
available at no charge on GAO's Web site at http://www.gao.gov. If you or
your staff have any questions on the matters discussed in this report,
please contact me at (202) 512-8412 or [email protected]. Contact points for
our Offices of Congressional Relations and Public Affairs may be found on
the last page of this report. GAO staff who made major contributions to
this report are listed in appendix II.

William M. Solis Director, Defense Capabilities and Management

                                   Appendix I

                    Comments from the Department of Defense

Appendix I Comments from the Department of Defense

Appendix II

                     GAO Contact and Staff Acknowledgments

William M. Solis (202) 512-8412 or [email protected].

  GAO Contact

In addition to the contact named above, Thomas Gosling, Assistant

  Acknowledgments

Director; Thom Barger; Judith Collins; Pamela Valentine; and Cheryl
Weissman were major contributors to this report.

  GAO's Mission

The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts GAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To

have GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."

                             Order by Mail or Phone

The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:

U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548

To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061

Contact:

To Report Fraud, Web site: www.gao.gov/fraudnet/fraudnet.htm

  E-mail: [email protected]

Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470

Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Relations
Washington, D.C. 20548

Paul Anderson, Managing Director, [email protected] (202) 512-4800

  Public Affairs

U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***