Military Aircraft: Institute for Defense Analyses Purchase Price
Estimate for the Air Force's Aerial Refueling Aircraft Leasing
Proposal (14-OCT-03, GAO-04-164R).
On September 4, 2003, GAO provided the Senate Committee on Armed
Services our observations on the Air Force's proposed lease of
100 Boeing 767 aircraft modified for aerial refueling, to be
known as the KC-767A. At the hearing, the Committee heard
testimony from the Institute for Defense Analyses (IDA)
concerning the results of a study it did on the estimated
acquisition cost of each aircraft. IDA concluded that $120.7
million was a reasonable price for a KC-767A aerial refueling
aircraft. At the time of the hearing, we had just obtained access
to the IDA study and were not in position to comment on it. On
September 5, 2003, the Committee's Chairman and Ranking Minority
Member asked us to analyze the IDA study and provide our
assessment. This letter responds to that request. Our objectives
were to assess the reasonableness of IDA's approach to the issue
of pricing the KC-767A aircraft and to provide any comments that
we had on its methodology used to estimate the base prices and
costs for each aircraft.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-04-164R
ACCNO: A08730
TITLE: Military Aircraft: Institute for Defense Analyses
Purchase Price Estimate for the Air Force's Aerial Refueling
Aircraft Leasing Proposal
DATE: 10/14/2003
SUBJECT: Air Force procurement
Contract costs
Cost effectiveness analysis
Equipment leases
Military aircraft
Military cost control
KC-767A Aircraft
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-04-164R
United States General Accounting Office Washington, DC 20548
October 14, 2003
The Honorable John Warner
Chairman
The Honorable Carl Levin
Ranking Member
Committee on Armed Services
United States Senate
Subject: Military Aircraft: Institute for Defense Analyses Purchase Price
Estimate for the Air Force's Aerial Refueling Aircraft Leasing Proposal
On September 4, 2003, we provided the Senate Committee on Armed Services
our observations on the Air Force's proposed lease of 100 Boeing 767
aircraft modified for aerial refueling, to be known as the KC-767A.1 At
the hearing, the Committee heard testimony from the Institute for Defense
Analyses (IDA) concerning the results of a study it did on the estimated
acquisition cost of each aircraft.2 IDA concluded that $120.7 million was
a reasonable price for a KC-767A aerial refueling aircraft. At the time of
the hearing, we had just obtained access to the IDA study and were not in
position to comment on it. On September 5, 2003, you asked us to analyze
the IDA study and provide you our assessment. This letter responds to that
request. Our objectives were to assess the reasonableness of IDA's
approach to the issue of pricing the KC-767A aircraft and to provide any
comments that we had on its methodology used to estimate the base prices
and costs for each aircraft.
To analyze the IDA report, we reviewed the report submitted to the Office
of the Under Secretary of Defense (Acquisition Technology and Logistics)
and the Office of the Director, Program Analysis and Evaluation,
Department of Defense (DOD).3 We met with the IDA (Cost Analysis and
Research Division) and Air Force acquisition officials from the Office of
the Assistant Secretary (Acquisition), Directorate of Global Reach. We
also
1 Military Aircraft: Observations on the Proposed Lease of Aerial
Refueling Aircraft by the Air Force. GAO-03-923T. Washington, D.C.:
September 4, 2003.
2 Purchase Price Estimate for the KC-767A Tanker Aircraft. IDA Paper
P-3800. Alexandria, Virginia.: July 2003.
3 These offices sponsored the IDA study.
Summary
Background
met with Office of Management and Budget officials to discuss the
methodology, conclusions, and data used in IDA's analysis. In addition, we
examined the Air Force's draft lease (which is still in negotiation and is
subject to change) and reviewed documents and briefings from the Office of
the Assistant Secretary of the Air Force for Acquisitions, Air Mobility
Programs, to identify issues and costs that are related to the IDA study.
Finally, we used data gathered for our review of the DOD response to your
suggestion that the Air Force lease 25 aircraft and purchase 754 and
additional data gathered for our on-going review of tanker requirements
being conducted for the House Armed Services Committee's Subcommittee on
Readiness.
We believe the IDA has provided a reasonable and comprehensive estimate
within the stated objective and assumptions given by the study authors.
The objective of the IDA study was to establish a reasonable direct
purchase price for 100 KC-767A aircraft. (Refer to the enclosure for a
description of the aircraft specifications as assessed by the IDA.) The
IDA was not asked to and did not address or make any assumptions about
provisions of the proposed lease, financing, suitability of leasing, or
any other acquisition alternatives to the proposal. Further, the IDA did
not attempt to reconcile its study results to coincide with the aircraft
configuration currently being negotiated between the Air Force and Boeing.
The Department of Defense Appropriations Act for fiscal year 20025
included a provision allowing the Air Force to establish a multiyear pilot
program for leasing Boeing 767 commercial aircraft to be used as aerial
refueling aircraft. Aerial refueling provides a key capability in
enhancing the mobility of U.S. forces and the Air Force is in the process
of planning for the replacement of its aging aircraft fleet. As you know,
the Air Force is in the final stages of negotiating a lease agreement with
Boeing for 100 new 767 aircraft that will be modified for use as refueling
aircraft.
While recognizing that aerial refueling is a key capability that is
essential to the mobility of U.S. forces, the Senate Committee on Armed
Services
4 Military Aircraft: Observations on DOD's Aerial Refueling Aircraft
Acquisition Options. GAO-04-169R. Washington, D.C.: October 14, 2003.
5
Pub. L. No. 107-117, S: 8159, 115 Stat. 2230, 2284-85.
The IDA Study and Our Assessment
has raised many issues concerning the proposed Air Force lease, including
questions about the validity of an independent DOD-commissioned study
performed by the Institute for Defense Analyses, a federally funded
research and development center. The July 2003 IDA study entitled Purchase
Price Estimate for the KC-767A Tanker Aircraft concluded that $120.7
million was a "conservative, robust" estimate of a reasonable purchase
price for the proposed KC-767A aircraft.
To determine the cost of acquiring 100 KC-767A aircraft and to formulate
the associated assessment methodology and data sources, IDA first
developed specific categories to describe the aircraft elements being
priced. These categories,6 reflecting the most significant financial
investments to the proposed aircraft purchase, were (a) basic B767-200ER
aircraft, (b) enhanced B767-200ER features, (c) combination (so-called
"combi") configuration modifications, (d) auxiliary fuel tanks, (e) tanker
and other modifications, and (f) development costs. The Table provides a
detailed summary of IDA's purchase price analysis.
Table 1: Summary of Institute for Defense Analyses' KC-767A Tanker/Combi
Purchase Price Analysis
Source: IDA and Air Force.
IDA separately estimated each of the above categories using a variety of
estimating techniques including cost estimating relationships, cost
models, and analogous data. In most cases, they used several techniques to
generate multiple cost estimates of a single item. The IDA took an average
6 A detailed description of these categories may be found in the IDA
report page S-2.
of these estimates as the final estimate value in each category. Likewise,
the IDA used commercial pricing techniques and marketing analysis where
possible and traditional cost analysis techniques, as dictated by the
content of the aircraft, where IDA determined that no significant
commercial market existed.7 In addition, IDA's analysis relied on data
from a variety of public sources, including other government sources, the
analyses of consultant organizations hired by the IDA, data supplied by
Boeing, other aerospace suppliers, and the Air Force. The specific
categories and our conclusions about the reasonableness of the IDA
analysis of each category follow:
o Basic B767-200ER: This element represents the cost to acquire the
baseline aircraft, the commercial aircraft upon which the KC-767A is
based. The necessary modifications and design changes would be applied to
this aircraft. To determine the cost, the IDA gathered pricing information
from five different commercial and government sources. These prices were
then averaged to obtain the price used in the study. We believe this was a
reasonable approach. In analytical terms, the coefficient of variance for
the five data points used is very small (4 percent), which shows the
various estimates are close together and provides confidence in the
estimate.8 We also note that data provided by the Air Force in October
2003 showed that they used six different methods to estimate the cost of
the basic B767-200ER aircraft. The prices developed under the approaches
ranged from $60.0 million to $80.5 million per aircraft for an average of
$71.1 million. Air Force officials ultimately used a weighted average and
concluded that a reasonable price for the basic aircraft would be about
$79.0 million per aircraft.
o Enhanced B767-200ER features: This category includes features added to
the basic aircraft to create the KC-767A. These items are not normally
available on the 200ER variant of the B767 but have been specified by the
Air Force for the KC-767A. However, Boeing has still not released its
updated version of the system specifications negotiated for the B767-200ER
aircraft variant. Each feature (e.g. enhanced cockpit, landing gear, and
engines; upgraded power supply, etc.) was estimated separately using
either commercially available information or IDA cost models. Again, given
the complexity and assumptions associated with these
7 The Boeing Company has asserted that a commercial market does exist for
its KC-767A aircraft.
8 Coefficient of variation is a measure of dispersion. It is the ratio of
the standard deviation to the arithmetic mean expressed as a percent. The
smaller the number, the less the variation in the distribution and
therefore the closer observations are to the mean.
engineering features, we believe the IDA's approach to be reasonable and
comprehensive for estimating this element's cost.
o `Combi' modifications: This element contains modifications to the
baseline B767-200ER to allow the aircraft to transport passengers or
freight, or a combination of the two simultaneously. To calculate this
estimate, IDA performed three separate estimates and then took the
average. Because this is a well-accepted and standard means to conducting
analysis of this type, we believe this methodology represents a sound
approach. Also, the coefficient of variation is very small (6 percent).
This shows that the various estimates are close together and provides
further confidence in the estimate.
o Auxiliary fuel tanks: This category provides for extra fuel capacity
(i.e., the lower fuselage fuel tanks, pumps, and installation materials
required to give the KC-767A additional fuel capacity). IDA solicited
quotes from two different vendors to generate the estimate. The two quotes
were averaged for the final result. Although, for assessment purposes, the
quotes were further apart with a coefficient of variation of 34 percent,
we believe they are only slightly above the rule of thumb for the range of
reasonableness of about a 30 percent coefficient of variation.
o Tanker and other Air Force-unique modifications: This sub-grouping
includes modifications associated with the refueling, fuel-receiving and
military-unique capabilities. These modifications are associated with the
ability to receive and offload fuel, including the installation of the
centerline boom, the hose/drogue unit, the fuel receiving receptacle, the
remote aerial refueling operator, and the plumbing and electrical changes
associated with these items.9 Avionics and miscellaneous Air Force
requirements are also included in this category. The IDA separately
estimated each modification that would occur. A cost estimating
relationship was used to develop the final estimates, which were based on
historical data collected by the IDA.10 We believe this to be a reasonable
methodology and analytical tool for pricing this element.
o Development costs: This element, broken down into three
sub-categories, captures any non-recurring investment costs needed to
design the KC-767A and represents the investment Boeing would make to
create the KC-767A configuration. The IDA accepted Air Force data for most
of the costs and used a cost estimating relationship to estimate the
9 "Hose and drogue" or "probe and drogue" and "centerline boom and
receptacle" refer to different types of refueling equipment systems used
to refuel different types of aircraft in flight.
10 A cost-estimating relationship is a cost function whose arguments are
variables related to the performance of the items or to specific features
of their designs.
remainder. The IDA assumed that a portion of these development costs
would be borne by the impending foreign sales to Italy and Japan and
adjusted their estimate accordingly. We believe this is a reasonable
approach for pricing this element.
In addition, as a further check on its assessment approach, the IDA
enlisted an expert panel of former and/or retired distinguished Air Force
officials to review its analytical approach and study methodology as well
as its positions on a range of reasonable prices for the KC-767A. IDA
officials told us that the panel concurred with IDA's final report
assessment and reporting results.
In its analysis, IDA used a December 19, 2002, system specification
document identifying the KC-767A aircraft configuration. The notional
aircraft is configured as a combination aerial refueling, cargo, and
passenger aircraft designed to permit both freight hauling and passenger
transport in the same mission. We provide a detailed description of the
notional aircraft in appendix I. To obtain a lower negotiated price on the
KC-767A aircraft, the Air Force has changed the specifications to
eliminate
the requirement that the aircraft be capable of passenger transport and
cargo hauling on the same flight while retaining the other planned combi
modifications.
Air Force and Office of
Management and Budget
Comments on the
IDA Study
Both the Air Force and Office of Management and Budget acknowledge that
the IDA study was a useful and valuable tool used in negotiations with
Boeing and believe it assisted the Air Force in negotiating a lower price
for the 100 KC-767A aircraft leasing package. However, the Air Force
believes that the IDA purchase price estimate did not have the "fidelity"
or accuracy of its negotiation position with Boeing. Air Force officials
told us that any line-by-line comparison of what individual items should
cost between the IDA study results and those negotiated by the Air Force
and Boeing is of limited value because the contract negotiation price was
for 100 aircraft and each party would undoubtedly divide the pieces
differently. However, the Air Force did not provide sufficient evidence to
explain how negotiating for 100 aircraft would necessarily lead to a
higher price than the average unit price for 100 aircraft given that IDA
did not assume that a volume discount would be given.
In addition, according to the Air Force, even if the IDA estimate proved
correct and the price per aircraft were lower than the negotiated price,
the government would be protected. Air Force officials pointed out that
the contract as currently proposed includes a best price guarantee and a
return-on-sales cap that enables the government to receive an equitable
adjustment if Boeing's profit exceeds the cap. While the draft contract
does include a "best price guarantee," this provision only guarantees that
The Boeing Company will not sell comparable KC-767A aircraft for less than
the Air Force would pay but it does not address the question of whether
the Air Force could have obtained a lower price. Moreover, as we testified
before the Senate Committees on Armed Services, and Commerce, Science and
Transportation,11 it is not clear to us why the sales cap is 15 percent as
negotiated, when a financial analysis has concluded that Boeing's profit
on commercial 767 aircraft is in the range of 6 percent.12
The Air Force also raised several other concerns with the IDA estimate
surrounding different acquisition strategies used, different bases for
estimates, etc. However, after review, we considered most of the
differences to be inherent in the nature of IDA's tasking and not
attributable to significant discrepancies or voids in information. For
example, the IDA was directed to consider only the cost of a direct
purchase price, not a leasing arrangement, so the IDA did assume a
different acquisition strategy than the Air Force. However, the Air Force
did not provide sufficient evidence to explain how a different acquisition
strategy would necessarily lead to a different purchase price for the
aircraft. Moreover, the cost estimates used by IDA to develop its purchase
prices are applicable and also used by DOD cost estimators for both firm
fixed price and cost contracts, and the planned lease of KC-767A aircraft
is to be done under a firm fixed price contract.
In oral comments on a draft of this correspondence, representatives from
the Air Force did not disagree with our analysis or our conclusions.
However, these officials believe that they negotiated a reasonable price
for the aircraft as planned to be configured by the Air Force and in the
quantity to be delivered, and that the IDA estimate of $120.7 million per
aircraft was not achievable. We do not know whether such a price is
achievable because such analysis was outside the scope of our work.
Agency Comments
and Our Evaluation
11 Military Aircraft: Observations on the Air Force's Plan to Lease Aerial
Refueling Aircraft. GAO-03-1143T. Washington, D.C.: September 3, 2003; and
Military Aircraft: Observations on the Proposed Lease of Aerial Refueling
Aircraft by the Air Force.
GAO-03-923T. Washington, D.C.: September 4, 2003.
12 Morgan Stanley, Does 767 Tanker Equate to 700+ Comml Orders?, (May 30,
2003).
In addition, officials from the Office of the Secretary of Defense and IDA
generally concurred with our analysis and our report.
We conducted this work from September to October 2003 in accordance
with generally accepted government auditing standards.
Unless you announce its contents earlier, we plan no further distribution
of this letter until 10 days from its issue date. At that time, we will
send
copies of this letter to the chairman and ranking member of the Committee
on Armed Services, House of Representatives, and the defense
subcommittees of the Senate and House Committees on Appropriations.
We will send a copy to the Chairman, Subcommittee on Readiness, House
Committee on Armed Services, for whom we are conducting a broader
body of work in this area. We will also send copies to the Secretary of
Defense, the Director of the Office of Management and Budget, and the
President of the Institute for Defense Analyses. We will also make copies
available to other interested parties upon request. In addition, the
letter
will be available at no charge on the GAO Web site at http://www.gao.gov.
We appreciate this opportunity to be of assistance. If you or your
staffs have any questions regarding this letter, please contact me at
(202) 512-4914 or Brian J. Lepore, Assistant Director, at (202) 512-4523.
Other key contributors to this review were Ann M. Dubois, Joseph J. Faley,
Jennifer K. Echard, Kenneth W. Newell, Madhav S. Panwar, Charles W.
Perdue, Kenneth E. Patton, and Tim F. Stone.
Neal P. Curtin, Director
Defense Capabilities and Management
Enclosure
Enclosure I: Description of Proposed Aircraft as Assessed by IDA
The KC-767A tanker/combi aircraft (the aircraft can serve as an
air-refueling tanker, can carry freight or passengers, or can combine
freight and passengers in the same mission) is to be based on the
commercial B767-200ER.1 Modifications would include the addition of
features available on other Boeing 767 models, as well as changes required
for the military application. In the tanker role, total fuel capacity is
to be just over 200,000 pounds, including up to 41,000 pounds carried in
added auxiliary fuel tanks. The KC-767A would have the capability to
perform refueling by the hose/drogue and boom methods from the aircraft
centerline and would also be able to receive fuel from other refueling
aircraft. The cabin of the KC-767A would be convertible to three
configurations. In the passenger configuration, the KC-767A would
accommodate up to 190 passengers and 10 crewmembers. The freight
configuration would accommodate 19 cargo pallets and 10 crewmembers. The
combination (so-called "Combi") configuration would have the capacity for
10 pallets, 10 crewmembers, and 70 passengers, although this configuration
has been dropped by the Air Force to reduce the price of the proposed
aircraft.
(350437) 1 System specification for Air Force KC-767A tanker transport
aircraft, Boeing Integrated Defense Systems, December 19, 2002.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
GAO's Mission
Obtaining Copies of GAO Reports and Testimony
The General Accounting Office, the audit, evaluation and investigative arm
of Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO's Web site (www.gao.gov) contains abstracts
and full-text files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document files.
To have GAO e-mail this list to you every afternoon, go to www.gao.gov and
select "Subscribe to e-mail alerts" under the "Order GAO Products"
heading.
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C.
20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
Contact:
To Report Fraud, Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in E-mail: [email protected]
Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470
Jeff Nelligan, Managing Director, [email protected] (202) 512-4800
Public Affairs U.S. General Accounting Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***