Defense Budget: Review of DOD's Report on Budgeting for Fuel Cost
Fluctuations (26-APR-07, GAO-07-688R).
The Office of Management and Budget (OMB) establishes for the
Department of Defense (DOD) the price DOD will use for pricing
crude oil when constructing its budget for upcoming fiscal years.
DOD in turn uses OMB's price in establishing the standard price
to be used for a barrel of fuel for budgeting purposes by DOD
fuel customers such as the military services. Because of the
volatility of world petroleum prices, the standard price for a
barrel of fuel included in the President's annual budget request
for DOD may be lower or higher than the actual price established
by the world market at any point in time after DOD's budget
request is submitted to the Congress. During the fiscal year, DOD
pays for fuel at the actual market rate, which typically varies
from the budgeted rate. As a result, if the actual price of crude
oil increases above the price DOD charges its customers, more
dollars are needed to pay for fuel than originally budgeted. If
the actual price is lower than what DOD charges its customers,
DOD has more dollars than needed. Additionally, if DOD responds
to increases in the world market crude oil prices by increasing
the price it charges its customers above what they initially
budgeted, the customers will have additional funding needs to pay
their fuel bills. Concerned as to whether DOD's method for
setting fuel prices has produced realistic budget estimates, last
year the Congress required DOD to consider alternative methods.
Specifically, the John Warner National Defense Authorization Act
for Fiscal Year 2007 requires the Secretary of Defense to submit
a report on the fuel rate and cost projection used in the annual
DOD budget presentation. The act required that DOD identify
alternative approaches, including approaches used by other
federal departments and agencies and the feasibility of using
private economic forecasting organizations, for selecting fuel
rates that would produce more realistic estimates of the amounts
required for DOD to accommodate fuel rate fluctuations. DOD is
also required to discuss the advantages and disadvantages of each
approach and to identify the department's preferred approach
among the alternatives and provide a rationale for preferring
that approach. Finally, the act further requires that GAO review
DOD's report, including the basis for the Secretary's conclusions
for the preferred approach. DOD submitted its report to the
Congress on February 27, 2007. In response to the act, we
determined (1) the extent to which DOD identified and evaluated
alternative crude oil forecasts that could be used in setting its
fuel rates--such as forecasts used by other federal departments
and agencies and private economic forecasting organizations, and
(2) DOD's basis for selecting its preferred fuel rate setting
approach.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-07-688R
ACCNO: A68767
TITLE: Defense Budget: Review of DOD's Report on Budgeting for
Fuel Cost Fluctuations
DATE: 04/26/2007
SUBJECT: Comparative analysis
Crude oil
Defense budgets
Defense economic analysis
Defense procurement
Energy costs
Fuel prices
Fuels
Future budget projections
Policy evaluation
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-07-688R
* [1]end of correspond & Test.pdf
* [2]PDF6-Ordering Information.pdf
* [3]Order by Mail or Phone
April 26, 2007
The Honorable Carl Levin
Chairman
The Honorable John McCain
Ranking Member
Committee on Armed Services
United States Senate
The Honorable Ike Skelton
Chairman
The Honorable Duncan L. Hunter
Ranking Member
Committee on Armed Services
House of Representatives
Subject: Defense Budget: Review of DOD's Report on Budgeting for Fuel Cost
Fluctuations
The Office of Management and Budget (OMB) establishes for the Department
of Defense (DOD) the price DOD will use for pricing crude oil when
constructing its budget for upcoming fiscal years. DOD in turn uses OMB's
price in establishing the standard price to be used for a barrel of fuel
for budgeting purposes by DOD fuel customers such as the military
services. Because of the volatility of world petroleum prices, the
standard price for a barrel of fuel included in the President's annual
budget request for DOD may be lower or higher than the actual price
established by the world market at any point in time after DOD's budget
request is submitted to the Congress. During the fiscal year, DOD pays for
fuel at the actual market rate, which typically varies from the budgeted
rate. As a result, if the actual price of crude oil increases above the
price DOD charges its customers, more dollars are needed to pay for fuel
than originally budgeted. If the actual price is lower than what DOD
charges its customers, DOD has more dollars than needed. Additionally, if
DOD responds to increases in the world market crude oil prices by
increasing the price it charges its customers above what they initially
budgeted, the customers will have additional funding needs to pay their
fuel bills.
Concerned as to whether DOD's method for setting fuel prices has produced
realistic budget estimates, last year the Congress required DOD to
consider alternative methods. Specifically, the John Warner National
Defense Authorization Act for Fiscal Year 2007 requires the Secretary of
Defense to submit a report on the fuel rate and cost projection used in
the annual DOD budget presentation.^1 The act required that DOD identify
alternative approaches, including approaches used by other federal
departments and agencies and the feasibility of using private economic
forecasting organizations, for selecting fuel rates that would produce
more realistic estimates of the amounts required for DOD to accommodate
fuel rate fluctuations. DOD is also required to discuss the advantages and
disadvantages of each approach and to identify the department's preferred
approach among the alternatives and provide a rationale for preferring
that approach. Finally, the act further requires that GAO review DOD's
report, including the basis for the Secretary's conclusions for the
preferred approach. DOD submitted its report to the Congress on February
27, 2007.
In response to the act, we determined (1) the extent to which DOD
identified and evaluated alternative crude oil forecasts that could be
used in setting its fuel rates--such as forecasts used by other federal
departments and agencies and private economic forecasting organizations,
and (2) DOD's basis for selecting its preferred fuel rate setting
approach.
In conducting our work, we examined DOD's report to the Congress on
budgeting for fuel cost fluctuations and other pertinent documentation on
DOD's methodology to identify the alternative crude oil forecasts DOD
considered for selecting budgeted fuel rates. We interviewed responsible
officials from the Office of the Under Secretary of Defense, the DOD
Comptroller, and Defense Logistics Agency (DLA), and OMB to discuss DOD's
methodology and rationale for selecting its preferred fuel rate selection
approach. We also assessed DOD's comparative analyses and the basis for
its conclusions. We conducted our work from February 2007 to April 2007 in
accordance with generally accepted government auditing standards.
Summary
DOD identified and evaluated approaches for forecasting crude oil prices
used by other federal departments and forecasts from private forecasting
organizations. DOD received responses from eight agencies.^2 DOD
determined that seven of the eight respondents used DOD's forecast, did
not have a forecast method, or did not have a forecasting method that
would accommodate the size and complexity of DOD's fuel requirement. Thus,
DOD concluded that only the Department of Energy's (DOE) forecasting
method, which uses the Energy Information Administration (EIA)^3 to
provide forecast prices for crude oil, was an option that warranted
further evaluation. According to DOD's report, it conducted four^4
analyses using different scenarios to compare its forecast approach, which
is based on crude oil forecasts provided by OMB with the methods used by
DOE and 38 private forecasting organizations. DOD reported that DOE's EIA
analysis produced forecasts that were marginally closer to actual crude
oil prices than either DOD's current method or most other private
organizations' forecasts, and only in a few cases did the private
organizations produce forecasts that were closer to actual costs than
DOD's current method. DOD selected its current method as its preferred
forecasting approach over the DOE and 38 private forecasting alternatives.
According to the DOD report, DOD chose to continue to use its current rate
setting method because it believes none of the alternative forecasting
methods produced significantly more accurate results. Specifically, DOD
did not believe that the margin of difference between forecasts based on
its current method and alternative approaches was significant enough to
warrant a change.
^1Pub. L. No. 109-364, S 1006 (2006).
^2DOD solicited inquiries from at least 21 federal departments and several
independent agencies but did not have information on the exact number.
^3The Congress created EIA within the DOE in 1977. As a statistical
agency, it provides policy-independent data, forecasts, and analyses to
promote sound policy making, efficient markets, and public understanding
regarding energy and its interaction with the economy and the environment.
^4DOD actually analyzed five scenarios but eliminated one in its report to
the Congress because the data were similar to scenario four.
In assessing DOD's comparative analysis, we found limitations on the scope
of the analysis due to the lack of available historical forecast data.
Specifically, DOD included 18 months of data or less in each of the
comparative scenarios it analyzed and the scenarios did not involve
forecasts from the same time period DOD used to set its budget request.
According to DOD officials, historical forecasting data were not available
to conduct a more extensive multiyear comparison or to replicate the
time-frame in which the fiscal year 2006 budgeted bulk fuel rate was set.
DOD officials agreed that had it been available, these additional data
would have produced a more ideal comparison. Consequently, based on the
results of the analysis that DOD was able to conduct with the data
available and given the volatile crude oil market, we agree that the
results did not provide a compelling reason for DOD to adjust its rate
setting approach and that DOD's current forecast based on OMB's guidance
for crude oil prices is a reasonable approach. In order for DOD to conduct
a more robust and extensive analysis, it would need to begin capturing and
maintaining the necessary forecast data now to conduct such analyses in
the future.
Background
DOD is the largest federal consumer of energy, accounting for over 92
percent of the petroleum-based energy consumed by the federal government.
DOD bulk fuel pricing is based on OMB's guidance for crude oil prices. DOD
uses three components in establishing its budgeted bulk fuel price. The
first component is the price for crude oil, the second component is the
added expense for refining the crude oil, and the third component is a
factor to account for DOD's overhead costs associated with transportation
and storage of the fuel. To estimate the cost of crude oil, the Council of
Economic Advisors, the Department of the Treasury, and OMB, referred to as
the Troika, jointly prepare a set of economic assumptions for agencies to
use in preparing their overall budgets. From these economic assumptions,
OMB directly provides DOD the crude oil price projections it uses in
setting its bulk fuel price. In developing the crude oil price
projections, the Troika uses oil price projections coming from the prices
in the futures market for West Texas Intermediate crude oil in the New
York Mercantile Exchange. For the 2007 budget, futures prices projected 5
years ahead were used in developing crude oil price projections. DOD and
its customers use refined oil products, such as jet fuel and diesel fuel,
and as such DOD has to include in its bulk fuel budget prices the
additional cost of refining the fuel above crude oil projections. This
component is estimated using a statistical technique that analyzes the
historical data relationship between crude oil prices and refined oil
prices.^5 Finally, DOD adds an overhead factor that reflects
transportation and storage costs of the petroleum products to DOD. When
comparing DOD's current method for setting bulk fuel prices to alternative
methods, the refining crude oil expense and overhead factor should not be
considered. Specifically, DOD's refining crude oil expense and overhead
costs have to be added to any forecast DOD used to set fuel prices, and
are unique factors to DOD. Consequently, DOD's cost for crude oil (the
first component) is what is comparable to other forecasts for oil.
DOD Identified and Evaluated Alternative Crude Oil Forecasts
In meeting its legislative requirement, DOD identified and evaluated
alternative crude oil forecasts by other federal departments and forecasts
from 38 private forecasting organizations. DOD evaluated these different
forecasts by conducting a comparative analysis of four different
scenarios, in which DOD reported that only one method produced slightly
better estimates than its current method. However, DOD believed that the
improvement was marginal and, along with the limited scope of its analysis
due to the data limitations, did not warrant a change from DOD's current
methodology.
DOD Evaluated Federal Agencies' and Private Organizations' Crude Oil
Forecasts
In its report, DOD evaluated crude oil forecasts used by other federal
departments and agencies. DOD solicited information from several federal
departments and agencies as to what method they used to forecast crude
oil. DOD received responses from eight departments and agencies: the
Departments of Commerce, Energy, Justice, and State, the U.S. Postal
Service, U.S. Customs and Border Protection agency, General Services
Administration, and Federal Bureau of Investigation. Of the eight
respondents, DOD evaluated each approach and determined seven did not have
a forecast method that would meet its needs. Of these seven, one agency
used DOD's forecasts, another did not have a forecast method in place and
the remaining five did not have crude oil forecast methods that DOD
believed would accommodate the size and complexity of DOD's fuel
requirement. Consequently, DOD concluded that only one agency--DOE, which
uses EIA forecasts--had a forecast that warranted further evaluation.
In addition to evaluating the methods used by other departments and
agencies to forecast crude oil prices, DOD conducted research to determine
which private organizations had experience in forecasting energy-related
commodities. DOD determined that 38 private organizations had forecasts
for crude oil prices that could be included in its comparative analysis.
^5The statistical approach used for the 2007 budget and earlier budgets
was regression analysis. A DOD energy official indicated that they are
examining different statistical methods to look at this relationship in
the future.
DOD's Report Included a Comparative Analysis of Four Scenarios to Evaluate
Crude Oil Forecasts
To evaluate the different crude oil forecast, DOD's report included a
comparative analysis of four different scenarios. DOD compared its
OMB-based forecast, DOE's EIA forecast, and up to 38^6 other private
economic forecasting organizations' crude oil forecasts to EIA actual
price data to determine which method came closest to predicting actual
crude oil prices. DOD's report included four different comparison
scenarios that compared June 2005 forecasts to actual rates for (1) the
average price for the first quarter of fiscal year 2006, (2) the average
price for the third quarter of fiscal year 2006, (3) the average price for
the entire 2006 calendar year, and (4) each of the six quarters from the
fourth quarter of fiscal year 2005 to the first quarter of fiscal year
2007. The results of these four scenarios follow.
The first scenario compared OMB's, EIA's, and 38 private organizations'
June 2005 forecasts to EIA's actual crude oil price of $60.00 per barrel
for the first quarter of fiscal year 2006. Table 1 shows that 1 private
organization's forecast and the EIA forecast performed better--were closer
to the actual price--than the OMB forecast for this time period, and 37 of
the 38 private organizations had forecasts that performed below--were
farther away from the actual price-- than OMB's forecasts.
^6According to DOD officials, they were not able to obtain forecasts for
each of the 38 private organizations for each scenario DOD evaluated
because the historic forecasting data were not always available.
Therefore, the number of private organizations compared in each scenario
varies.
Table 1: Comparison of Forecasts from OMB, EIA, and 38 Private
Organizations in June 2005 to the EIA Actual Price Using the Average of
the First Quarter of Fiscal Year 2006
Dollars per barrel
Organization June 2005 forecasts Difference between June 2005
forecast and actual price of
$60.00
Private organization 57.10 (2.90)
A forecast
EIA forecast 55.00 (5.00)
OMB forecast 53.43 (6.57)
37 other private Ranging from 32.00 (28.00 to 7.00)
organizations' forecasts to 53.00
Source: DOD data.
The second scenario compared OMB's, EIA's, and 28 private organizations'
June 2005 forecasts to EIA's actual price of $70.41 per barrel for the
third quarter of fiscal year 2006. Table 2 shows that 3 private
organizations' and the EIA forecast performed better than the OMB forecast
for this time period and 25 of the 28 private organizations had forecasts
that performed below OMB's forecasts.
Table 2: Comparison of Forecasts from OMB, EIA, and 28 Private
Organizations in June 2005 to the EIA Actual Price Using the Average of
the Third Quarter of Fiscal Year 2006
Dollars per barrel
Organization June 2005 forecasts Difference between June 2005
forecast and actual price of
$70.41
Private organization A 59.30 (11.11)
forecast
Private organization B 55.00 (15.41)
forecast
Private organization C 54.00 (16.41)
forecast
EIA forecast 53.58 (16.83)
OMB forecast 52.80 (17.61)
25 other private Ranging from 37.00 (33.41 to 20.41)
organizations' forecasts to 52.80
Source: DOD data.
The third scenario compared OMB's, EIA's, and 13 private organizations'
June 2005 forecasts to EIA's actual price of $66.02 per barrel for
calendar year 2006. Table 3 shows that 1 private organization's forecast
and the EIA forecast performed better than the OMB forecast for this time
period and 12 of the 13 organizations had forecasts that performed below
OMB's forecasts.
Table 3: Comparison of Forecasts from OMB, EIA, and 13 Private
Organizations in June 2005 to the EIA Actual Prices for Calendar Year 2006
Dollars per barrel
Organization June 2005 forecasts Difference between June 2005
forecast and actual price of
$66.02
Private organization C 55.00 (11.02)
forecast
EIA forecast 54.23 (11.79)
OMB forecast 52.58 (13.44)
12 other private Ranging from 41.80 (24.22 to 14.77)
organizations' forecasts to 51.25
Source: DOD data.
The fourth scenario compared forecasts of OMB, EIA, and 1 private
organization from the fourth quarter of fiscal year 2005 to the first
quarter of 2007 to EIA actual prices for each of those quarters. As shown
in table 4, the EIA forecast performed better than the OMB forecast for
this time period, and the private organization's forecasts performed below
both OMB's forecasts and EIA's forecasts.
Table 4: Comparison of Forecasts from OMB, EIA and 1 Private Organization
to the EIA Actual Prices for the Fourth Quarter of Fiscal Year 2005 to the
First Quarter of 2007
Dollars per barrel
EIA actual EIA forecast OMB forecast Private organization D
price forecast
4th Quarter 63.19 52.83 52.68 52.50
2005
1st Quarter 60.00 55.00 53.43 53.00
2006
2nd Quarter 63.27 54.00 53.24 52.00
2006
3rd Quarter 70.41 53.58 52.80 50.00
2006
4th Quarter 70.42 54.33 52.37 49.00
2006
1st Quarter 59.98 55.00 51.76 48.00
2007
Source: DOD data.
The overall results of DOD's four comparative analyses show that EIA
forecasts produced forecasts that were closer to the actual price than
OMB's forecasts in each scenario^7 and 3 of the 38 private organizations
outperformed OMB and EIA forecasts in individual comparison scenarios;
however, no private organization consistently outperformed either OMB or
EIA in all scenarios.
DOD Selected Its Preferred Approach Based on Results of Its Comparative
Analysis
DOD selected its current method--based on OMB's forecasts--as its
preferred bulk fuel rate setting approach because it concluded that it's
comparative analysis showed that results using OMB's forecasts were
comparable to or better than the alternative forecasts it evaluated. Based
on the comparative analysis performed, DOD concluded that because the
results showed only marginal improvement (1.92 percent difference) in
using EIA's forecast over OMB's forecast and since no private organization
consistently outperformed OMB forecasts in all scenarios, it did not
warrant a change from DOD's current methodology. DOD also reported that
inaccurate forecasts are pervasive in the current economic conditions
based on market volatility. Thus, DOD concluded that changing forecasting
sources or methods may not provide more realistic estimates for DOD at
this time.
^7 The eliminated scenario compared the average price for the entire 2006
fiscal year for OMB's, EIA's and 1 private organization's forecasts. This
comparison also showed that EIA's June 2005 forecast was more accurate
than OMB's forecast by 1.92 percent.
Our Assessment Showed Data Limitations Constrained DOD's Comparative
Analysis, but Current Fuel Rate Setting Approach Seems Reasonable
In assessing DOD's comparative analysis, we found that lack of available
historical forecast data limited the scope of DOD's analysis. DOD included
18 months or less of data in each of the comparative scenarios it
conducted. An economist in DOD's Defense Energy Support Center, who
performed the analysis, agreed that a thorough evaluation of a group's
long-term forecasting performance should include several different time
periods. However, according to DOD officials, DOD was not able to obtain
historical forecasting data to perform comparisons prior to June 2005
because these data are not retained by private forecast organizations.
Consequently, DOD used June 2005 forecasts to predict crude oil prices for
the various fiscal year 2006 scenarios. However, DOD set its fiscal year
2006 budgeted fuel prices in December 2004, 6 months earlier than the
forecasts DOD used in the analyses. When DOD set its fuel rate in December
2004, the forecast it used predicted fuel costs 10 to 21 months into the
future (December 2004 to October 2005 through September 2006). Most of the
scenarios DOD included in its analysis did not predict far enough into the
future to replicate what DOD does each year when its sets its budgeted
fuel rates--
for example, scenario one only predicted 3 months into the future. In
addition, OMB's December 2004 forecast for crude oil was $40.45 a barrel,
which was $12.98 a barrel lower than the June 2005 OMB forecast DOD used
in scenario one. The closer the forecast is to the actual time-frame being
estimated, the easier it is to predict a more accurate forecast. The DOD
economist we spoke with agreed that it would have been a more meaningful
analysis to use December 2004 forecasts, but again stated that forecasts
were not available from EIA or private forecasting organizations for that
time-frame. In fact, the EIA forecast data that DOD used in the analysis
are only available from the month starting the forecast to the end of the
next calendar year. For example, in December 2004, EIA forecasted data
only through December 2005--which would have been insufficient to provide
a basis for DOD in establishing fiscal year 2006 rates. According to DOD
officials, the timing of the EIA forecast data does not match up to the
time-frame when DOD has to set its rates for budgeting purposes.
Consequently, based on the results of the analysis that DOD was able to
conduct with the data available, we agree that the results did not provide
a compelling reason for DOD to adjust its rate setting approach at this
time. Because of DOD's decision to retain its current approach, we
reviewed the basis for OMB's forecast to determine if it is based on
reasonable assumptions. OMB's forecast is based on the futures market of
crude oil. Using futures' prices for crude oil seems like a reasonable
approach because buyers and sellers are actually making purchases and
sales based on their ideas and information about the future price of crude
oil. The futures price for crude oil for a particular date would reflect
the interaction between buyers and sellers and the best estimate for the
price of crude oil at that time. Barring more extensive analysis that
might provide additional information to consider in selecting an
alternative forecast, we believe that DOD's current method for producing
fuel rates is a reasonable approach. In order for DOD to conduct a more
robust analysis, it would need to begin capturing and maintaining the
necessary forecast data now to conduct such analysis in the future.
Agency Comments
DOD officials reviewed a draft of this report and had no comments.
_ _ _ _ _
We are sending copies of this report to the Senate and House Armed
Services Committees. We also will make copies available to others upon
request. In addition, this report will be available at no charge on GAO's
Web site at http://www.gao.gov .
If you have any questions concerning this report, please contact me at
(202) 512-9619 or pickups@gao.gov . Contact points for our Offices of
Congressional Relations and Public Affairs may be found on the last page
of this report. Key contributors to this assignment were Bonita P.
Anderson, Renee S. Brown, Laura L. Durland, Julia Matta, Charles W.
Perdue, and Jeanett H. Reid.
Sharon Pickup
Director, Defense Capabilities and Management
(350960)
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548
Public Affairs
Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***