Defense Travel System: Reported Savings Questionable and	 
Implementation Challenges Remain (26-SEP-06, GAO-06-980).	 
                                                                 
In 1995, the Department of Defense (DOD) began an effort to	 
implement a standard departmentwide travel system. The Defense	 
Travel System (DTS) is envisioned as DOD's standard end-to-end	 
travel system. This report is a follow-up to GAO's January 2006, 
report which highlighted DTS implementation problems. Because of 
continued congressional interest in DTS, GAO initiated this	 
follow-up audit under the Comptroller General's statutory	 
authority. GAO determined whether (1) two key assumptions made in
the September 2003 economic analysis were reasonable, (2) DOD is 
taking action to ensure full utilization of DTS and gathering the
data needed to monitor DTS utilization, and (3) DOD has resolved 
the previously identified problems with DTS flight information.  
To address the above objectives, GAO (1) reviewed the September  
2003 DTS economic analysis, (2) analyzed DTS utilization data,	 
and (3) analyzed DTS flight information.			 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-06-980 					        
    ACCNO:   A61428						        
  TITLE:     Defense Travel System: Reported Savings Questionable and 
Implementation Challenges Remain				 
     DATE:   09/26/2006 
  SUBJECT:   Cost analysis					 
	     Cost effectiveness analysis			 
	     Data collection					 
	     Defense cost control				 
	     Defense economic analysis				 
	     Financial management systems			 
	     Internal controls					 
	     Program evaluation 				 
	     Program management 				 
	     Travel						 
	     Travel costs					 
	     Program implementation				 
	     Defense Travel System				 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-06-980

     

     * Results in Brief
     * Background
     * Validity of DTS Economic Analysis Questionable
          * Personnel Savings Are Unrealistic
          * Savings Associated with Reduction of CTO Fees Are Unknown
               * "No Touch" Transaction Volume Estimates Are Not Supported
               * Navy Impact of CTO Management Fees Not Adequately Considered
          * Effective Implementation of Existing Policies Should Have Id
          * New Economic Analysis Not Required by DOD Criteria
     * DTS Remains Underutilized by the Military Services
          * Metrics to Measure DTS Utilization Are Inadequate
          * DOD Has Taken Steps to Improve DTS Utilization, but Further
     * Previously Reported DTS Requirements Management and Testing
          * Providing Complete Flight Information Has Been a Continuing
          * DOD's Planned Corrective Actions Will Not Address Fundamenta
     * Conclusions
     * Recommendations for Executive Action
     * Agency Comments and Our Evaluation
     * Appendix I: Scope and Methodology
     * Appendix II: Comments from the Department of Defense
     * Appendix III: GAO Contacts and Staff Acknowledgments
          * GAO Contacts
          * Acknowledgments
               * Order by Mail or Phone

Report to Congressional Addressees

United States Government Accountability Office

GAO

September 2006

DEFENSE TRAVEL SYSTEM

Reported Savings Questionable and Implementation Challenges Remain

GAO-06-980

Contents

Letter 1

Results in Brief 3
Background 7
Validity of DTS Economic Analysis Questionable 9
DTS Remains Underutilized by the Military Services 17
Previously Reported DTS Requirements Management and Testing Deficiencies
Have Not Been Resolved 23
Conclusions 31
Recommendations for Executive Action 31
Agency Comments and Our Evaluation 32
Appendix I Scope and Methodology 38
Appendix II Comments from the Department of Defense 40
Appendix III GAO Contacts and Staff Acknowledgments 43

Tables

Table 1: Summary of DTS Estimated Annual Net Savings Reported in the
September 2003 Economic Analysis 8
Table 2: Summary of Estimated Annual Personnel Savings 10
Table 3: DTS Reported Utilization Percentage for the Period October 2005
through April 2006 18

Figure

Figure 1: March 2006 DTS Display of Flights between Santiago, Chile, and
San Antonio, Texas 27

Abbreviations

BTA Business Transformation Agency CTO commercial travel office DFAS
Defense Finance and Accounting Service DOD Department of Defense DTS
Defense Travel System FTR Federal Travel Regulation GDS Global
Distribution System GSA General Services Administration IOC Initial
Operating Capability NCAD Naval Cost Analysis Division OMB Office of
Management and Budget PMO-DTS Program Management Office-Defense Travel
System RTS Reserve Travel System TDY temporary duty

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

United States Government Accountability Office

Washington, DC 20548

September 26, 2006

Congressional Addressees

In 1995, the Department of Defense (DOD) embarked upon the daunting
challenge of implementing a standard, departmentwide travel system in
response to a report by the DOD Task Force to Reengineer Travel.1 The
report pinpointed three principal causes for DOD's inefficient travel
system: (1) travel policies and programs were focused on compliance with
rigid rules rather than mission performance, (2) travel practices did not
keep pace with travel management improvements implemented by industry, and
(3) the various existing travel systems were not integrated. To address
these concerns, DOD established the Program Management Office-Defense
Travel System (PMO-DTS) to acquire travel services that would be used
DOD-wide. The department launched this program with the goal of replacing
existing travel systems with a single departmentwide system to more
effectively support nonintegrated travel processes and procedures across
its component organizations. The Defense Travel System (DTS) is envisioned
as being the department's standard end-to-end travel system.2

The department estimates that DTS will be fully deployed at all 11,000
intended locations during fiscal year 2007.3 The September 2003 economic
analysis noted that DTS, when fully implemented, would result in annual
net savings of over $56 million. The economic analysis noted that savings
would be realized by the department during fiscal years 2009-2016. In
December 2003, the department's Chief Information Officer approved a DTS
funding level of approximately $564 million. Of this amount, the contract
for the design, development, and deployment of DTS was for about $264
million. The remaining costs are associated with areas such as the
operation and maintenance of DTS, operation of the PMO-DTS, the voucher
payment process, and management and oversight of the numerous contracted
commercial travel offices (CTO).

1 Department of Defense, Report of the Department of Defense Task Force to
Reengineer Travel (Arlington, Va.: January 1995).

2 DOD expects DTS to perform all functions related to travel or ensure
that other systems are provided with adequate information to provide this
functionality. For example, obligating funds associated with travel is a
necessary function, and DTS is expected to (1) make sure that adequate
funds are available before authorizing travel either through information
contained in its system or by obtaining the necessary information from
another system, (2) obligate funds through issuance of approved travel
orders, and (3) provide DOD's financial management systems with the
necessary information so that those systems can record the obligation.
Since DTS is required to ensure that all travel-related functionality is
properly performed, DOD commonly refers to DTS as an "end-to-end travel
system."

3 As of September 2005, the department had estimated that DTS would be
fully deployed during fiscal year 2006.

This report is a follow-up to our September 2005 testimony and January
2006 report in which we highlighted problems encountered by the department
in its efforts to successfully implement DTS.4 One of our major findings
was that DOD did not have reasonable assurance that flight information was
properly displayed for DOD travelers because the department failed to
properly test the system interfaces through which the data are accessed
for display. We further noted that the continued use of the existing
legacy travel systems at locations where DTS has been deployed results in
underutilization of DTS and reduces the envisioned savings.

The objectives of this audit were to determine whether (1) two key
assumptions related to the estimated cost savings in the September 2003
economic analysis were reasonable, (2) DOD is taking action to ensure full
utilization of DTS and gathering the data needed to monitor DTS
utilization, and (3) DOD has resolved the previously identified problems
with properly displaying DTS flight information. To address the first
objective, we obtained and reviewed the September 2003 economic analysis
to (1) ascertain if the economic analysis was prepared in accordance with
criteria prescribed by the Office of Management and Budget (OMB) and DOD;
(2) analyze two key assumptions on which the majority of the estimated
savings were predicated-personnel savings and reduced CTO fees; and (3)
analyze the underlying supporting documentation related to these two
assumptions. In addition, we interviewed PMO-DTS and military service
officials to obtain an understanding of the basis they used to compute the
reported annual net savings of over $56 million.5

4 GAO, DOD Business Transformation: Defense Travel System Continues to
Face Implementation Challenges, GAO-06-18 (Washington, D.C.: Jan. 18,
2006), and DOD Business Transformation: Preliminary Observations on the
Defense Travel System, GAO-05-998T (Washington, D.C.: Sept. 29, 2005).

To address the second objective, we obtained and analyzed DTS utilization
data from the PMO-DTS. We also met with military service officials to
obtain an understanding of the efforts they have underway to help ensure
the full utilization of DTS. Further, we obtained from the military
services an understanding of the data they used to monitor DTS
utilization. Finally, to address the third objective, we analyzed 246 U.S.
General Services Administration (GSA) city pair flights to determine if
the information being displayed to the traveler was consistent with DTS's
stated requirement. We also met with PMO-DTS and contractor officials.

Because of the continued widespread congressional interest in DTS, this
assignment was performed at our initiative under the statutory authority
provided to the Comptroller General of the United States. Our work focused
on the validity of the assumptions that were the principal drivers of the
net annual estimated savings of over $56 million. We determined that the
data were sufficiently reliable for this purpose. We did not review the
accuracy and reliability of the specific dollar amounts shown in the
September 2003 economic analysis. Our work was performed from October 2005
through July 2006 in accordance with U.S. generally accepted government
auditing standards. Details on our scope and methodology are included in
appendix I. We requested comments on a draft of this report from the
Secretary of Defense or his designee. We received written comments from
the Under Secretary of Defense (Personnel and Readiness), which are
reprinted in appendix II.

                                Results in Brief

Our analysis of the September 2003 DTS economic analysis found that two
key assumptions used to estimate cost savings were not based on reliable
information. Two primary areas represented the majority of the over $56
million of estimated annual net savings DTS was expected to
realize-personnel savings of $24.2 million and reduced CTO fees of $31
million. The $24.2 million estimated annual personnel savings were
attributed to the Air Force and Navy.6 However, the Naval Cost Analysis
Division has stated that the Navy will not realize any tangible personnel
cost savings from the implementation of DTS. In regard to the estimated
annual savings of $31 million attributed to lower CTO fees, we requested,
but the PMO-DTS could not provide, any analysis of travel data to support
the assumption that 70 percent of all airline tickets would be considered
"no touch"-meaning that there would be no or minimal intervention by the
CTO, thereby resulting in lower CTO fees. We found that the 70 percent
assumption was based solely upon an article that appeared in a travel
industry trade publication.

5 The total estimated annual savings were $123.5 million and the total
estimated annual costs were $67.1 million for a net annual savings of
$56.4 million. The annual net savings are in constant fiscal year 2003
dollars. The department estimated that savings would start in fiscal year
2009.

In addition, the economic analysis was not prepared in accordance with
guidance prescribed by OMB and DOD. Both sets of guidance require that an
economic analysis be based on facts and data and be explicit about the
underlying assumptions used to arrive at future benefits and costs. DOD
guidance also states that life-cycle cost estimates should be
independently validated. An independent review is intended, in part, to
provide program management some degree of assurance that the life-cycle
cost estimates are reasonable and the cost estimates are built on
realistic program assumptions. However, an independent validation was not
performed.

Based on these factors, the estimated annual net savings of over $56
million included in the 2003 economic analysis is highly questionable.
While the reliability of the economic analysis is questionable, the
department's system acquisition criteria do not require that a new
economic analysis be prepared because DTS has already completed all of the
major milestones related to a major automated system. However, the
department's business system investment management guidance stipulates
that all business systems must be reviewed annually and provides an
opportunity for DOD management to assess whether DTS is meeting its
planned cost, schedule, and functionality goals.

Our analysis also found that the department did not have quantitative
metrics to measure the extent to which DTS is actually being used. The
reported DTS utilization rates were based on a methodology that was
developed using estimated data, and PMO-DTS program officials acknowledged
that the model had not been completely updated with actual data as DTS
continued to be implemented at the 11,000 sites. As a result, the PMO-DTS
continues to rely on outdated information in calculating DTS utilization
rates that are reported to DOD management and the Congress. Additionally,
while the military services have initiated actions to help increase the
utilization of DTS, they pointed out that ineffective DTS training is a
contributing factor to the lower than expected usage rate by the military
services.

6 The economic analysis identified annual savings of $11.3 million and
$12.9 million for the Air Force and Navy, respectively.

Finally, DOD still has not addressed the several functional problems
associated with weak requirements management and system testing.
Requirements represent the blueprint that system developers and program
managers use to design, develop, test, and implement a system. Because
requirements provide the foundation for system testing, they must be
complete, clear, and well documented to design and implement an effective
testing program. Our February 2006 analysis disclosed that DOD still did
not have reasonable assurance that the flight information was being
properly displayed to DOD travelers. We identified 246 unique GSA city
pair flights that should have been identified on one or more DTS flight
displays according to the DOD requirements. However, 87 of these flights
did not appear on one or more of the required listings. We also identified
instances in which DTS displayed flights for selection that did not appear
to comply with the Fly America Act.7 By not displaying flights in
accordance with the Fly America Act's criteria, DTS places the traveler
who purchases a ticket or the individual authorizing, certifying, or
disbursing a payment made when a ticket is paid for directly by DOD
through a centrally billed account at unnecessary risk of personal
liability, because the travelers can be held accountable for the cost of
the trip. While the PMO-DTS has taken action to address our concerns,
these actions do not fully address the fundamental problems we found
during this audit and on which we have previously reported.8 For example,
the DTS requirements we reviewed were still ambiguous and conflicting.
Adequately defined and tested requirements are one of the key elements to
help reduce a project's risks to acceptable levels.9

7 49 U.S.C. S: 40118(c). The act requires federal employees and their
dependents, consultants, contractors, grantees, and others performing U.S.
government-funded air travel to travel by U.S. certificated flag air
carriers except under certain circumstances, such as when travel by a
foreign air carrier is a matter of necessity as defined by the statute or
when U.S. certificated flag air carrier service is not available.

8 GAO-05-998T and GAO-06-18 .

We are making four recommendations to the Secretary of Defense aimed at
improving the department's management and oversight of DTS. More
specifically, we recommend that the Secretary of Defense (1) evaluate the
cost effectiveness of the Navy continuing with the CTO management fee
structure, (2) update the DTS Voucher Analysis Model to report DTS actual
utilization rates, (3) require the PMO-DTS to provide periodic reports on
the utilization of DTS, and (4) resolve inconsistencies in DTS
requirements.

In written comments on a draft of this report, DOD agreed with three and
partially agreed with one of the recommendations. For those
recommendations the department agreed with, the comments briefly outlined
its actions for addressing two of them, but did not comment on the third.
In regard to the recommendations to which it responded, the department's
planned actions are in keeping with the intent of our recommendations.

DOD disagreed with our finding that the estimated personnel savings are
unrealistic. DOD stated that recognizing fiscal constraints, the
department continues to identify efficiencies and eliminate redundancies
to help leverage available funds. As noted in our report, DOD officials
responsible for reviewing economic analyses stated that while shifting
personnel to other functions is considered a benefit, it should be
considered an intangible benefit rather than tangible dollar savings since
the shifting of personnel does not result in a reduction of DOD
expenditures. Because none of the military services could validate an
actual reduction in the number of personnel as a result of DTS
implementation, we continue to believe that the estimated annual personnel
savings of $54.1 million is unrealistic. The Agency Comments and Our
Evaluation section of this report provides a more detailed discussion of
the department's comments. We have reprinted DOD's written comments in
appendix II.

9 Acceptable levels refer to the fact that any systems acquisition effort
will have risks and will suffer the adverse consequences associated with
defects in the processes. However, effective implementation of disciplined
processes, which includes project planning and management, requirements
management, risk management, quality assurance, and testing, reduces the
possibility of the potential risks actually occurring and prevents
significant defects from materially affecting the cost, timeliness, and
performance of the project.

                                   Background

In September 1993, the National Performance Review recommended an overhaul
of DOD's temporary duty (TDY) travel system. In response, DOD created the
DOD Task Force to Reengineer Travel to examine the travel process. The
task force found that the current process was expensive to administer and
was neither customer nor mission oriented with the net result being a
travel process that was costly, inefficient, fragmented, and did not
support DOD's needs. On December 13, 1995, the Under Secretary of Defense
for Acquisition and Technology and the Under Secretary of Defense
(Comptroller)/Chief Financial Officer issued a memorandum, "Reengineering
Travel Initiative," establishing the PMO-DTS to acquire travel services
that would be used DOD-wide. Additionally, in a 1997 report to Congress,
the DOD Comptroller pointed out that the existing DOD TDY travel system
was never designed to be an integrated system.10 The report stated that
because there was no centralized focus on the department's travel
practices, the travel policies were issued by different offices and the
process had become fragmented and "stovepiped." The report further noted
that there was no vehicle in the current structure to overcome these
deficiencies, as no one individual within the department had specific
responsibility for management control of DOD TDY travel.

DOD management and oversight of the DTS program has varied over the years.
DTS was designated a "Special Interest" program in 1995. It retained this
status until May 2002 when it was designated a major automated information
system,11 with the Defense Finance and Accounting Service (DFAS) being
designated as the lead component for the program. This meant that DFAS was
responsible for the management oversight of DTS program acquisition,
including DTS compliance with the required DOD acquisition guidance.

10 Office of the Under Secretary of Defense (Comptroller), Department of
Defense Travel Reengineering Pilot Report to Congress (Arlington, Va.:
June 1997).

11 A major automated information system is one in which the DOD component
head estimates that (1) program costs in any single year will exceed $32
million in fiscal year 2000 constant dollars, (2) total program costs will
exceed $126 million in fiscal year 2000 constant dollars, or (3) total
life-cycle costs will exceed $378 million in fiscal year 2000 constant
dollars. The life-cycle cost is the total cost to the government for an
information system over its expected useful life and includes the costs to
acquire, operate, maintain, and dispose of the system. DOD Instruction
5000.2, Operation of the Defense Acquisition System, specifies current
mandatory policies and procedures for major acquisitions. The policy also
specifies that the DOD Chief Information Officer is the milestone decision
authority, responsible for program approval, for all major automated
information systems.

In September 2003, DOD finalized its economic analysis for DTS in
preparation for a milestone decision review.12 The highlights of the
economic analysis are shown in table 1. In December 2003, the DOD Chief
Information Officer granted approval for DTS to proceed with full
implementation throughout the department.

Table 1: Summary of DTS Estimated Annual Net Savings Reported in the
September 2003 Economic Analysis

Constant fiscal year 2003 dollars in millions
Cost components                    Estimated annual net savings 
Records management                                        $19.8 
Centrally billed accounts                                   1.7 
CTO acquisition and administration                          2.4 
CTO services                                               31.0 
Voucher process and compute                                54.1 
Voucher pay                                                   0 
Legacy systems                                             14.5 
PMO                                                       (8.8) 
Help desk/DTA                                            (36.8) 
System operations                                        (21.5) 
Total net savings                                         $56.4 

Source: September 2003 economic analysis provided by the PMO-DTS.

Note: In arriving at the estimated annual net savings of over $56 million,
the economic analysis took into consideration the estimated costs of over
$2.1 billion, which covers fiscal years 2003-2016. The estimated costs
included the costs that are estimated to be incurred by the PMO-DTS, the
Army, the Navy, the Air Force, and the defense agencies.

In October 2005, DOD established the Business Transformation Agency (BTA)
to advance DOD-wide business transformation efforts, particularly with
regard to business systems modernization. DOD believes it can better
address managing defensewide business transformation, which includes
planning, management, organizational structures, and processes related to
all key business areas, by first transforming business operations to
support the warfighter, while also enabling financial accountability
across DOD. BTA operates under the authority, direction, and control of
the Under Secretary of Defense for Acquisition, Technology, and Logistics,
who is the vice chair of the Defense Business Systems Management
Committee-which serves as the highest ranking governing body for business
systems modernization activities. Among other things, BTA includes a
Defense Business Systems Acquisition Executive who is responsible for
centrally managing 28 DOD-wide business projects, programs, systems, and
initiatives-one of which is DTS.13 In October 2004, responsibility for the
policies and procedures related to the management of commercial travel
throughout DOD transferred to the Office of the Under Secretary of Defense
(Personnel and Readiness).

12 This is an addendum to the July 2003 DTS economic analysis.

                 Validity of DTS Economic Analysis Questionable

Our analysis of the September 2003 DTS economic analysis found that two
key assumptions used to estimate cost savings were not based on reliable
information. Consequently, the economic analysis did not serve to help
ensure that the funds invested in DTS were used in an efficient and
effective manner. Two primary areas represented the majority of the over
$56 million of estimated annual net savings DTS was expected to
realize-personnel savings and reduced CTO fees. However, the estimates
used to generate these savings were unreliable. Further, DOD did not
effectively implement the policies relating to developing economic
analyses for programs such as DTS. Effective implementation of these
policies should have highlighted the problems that we found and allowed
for appropriate adjustments so that the economic analysis could have
served as a useful management tool in making funding decisions related to
DTS-which is the primary purpose of this analysis. While the department's
system acquisition criteria do not require that a new economic analysis be
prepared, the department's business system investment management structure
provides an opportunity for DOD management to assess whether DTS is
meeting its planned cost, schedule, and functionality goals.

Personnel Savings Are Unrealistic

The economic analysis estimated that the annual personnel savings was over
$54 million,14 as shown in table 2.

13 Examples of some of these DOD-wide programs, systems, and initiatives
besides DTS include the Standard Procurement System, the Defense
Integrated Military Human Resources System, and the Standard Financial
Information Structure.

14 During fiscal years 2009 through 2016.

Table 2: Summary of Estimated Annual Personnel Savings

Constant fiscal year 2003 dollars in millions
DOD component               Estimated annual savings 
Army                                           $16.0 
Navy                                            12.9 
Air Force                                       11.3 
Marine Corps                                     5.8 
Defense agencies                                 6.3 
Permanent change of station                      1.8 
Total savings                                  $54.1 

Source: September 2003 economic analysis provided by the PMO-DTS.

As shown in table 2, approximately 45 percent of the estimated savings, or
$24.2 million was attributable to the Air Force and Navy. The assumption
behind the personnel savings computation was that there would be less
manual intervention in the processing of travel vouchers for payment, and
therefore fewer staff would be needed. However, based on our discussions
with Air Force and Navy DTS program officials, it is questionable as to
how the estimated savings will be achieved. Air Force and Navy DTS program
officials stated that they did not anticipate a reduction in the number of
personnel with the full implementation of DTS, but rather the shifting of
staff to other functions. According to DOD officials responsible for
reviewing economic analyses, while shifting personnel to other functions
is considered a benefit, it should be considered an intangible benefit
rather than tangible dollar savings since the shifting of personnel does
not result in a reduction of DOD expenditures. Also, as part of the Navy's
overall evaluation of the economic analysis, program officials stated that
"the Navy has not identified, and conceivably will not recommend, any
personnel billets for reduction." Finally, the Naval Cost Analysis
Division (NCAD) October 2003 report on the economic analysis noted that it
could not validate approximately 40 percent of the Navy's total costs,
including personnel costs, in the DTS life-cycle cost estimates because
credible supporting documentation was lacking. The report also noted that
the PMO-DTS used unsound methodologies in preparing the DTS economic
analysis.

The extent of personnel savings for the Army and defense agencies, which
are reported as $16 million and $6.3 million respectively, is also
unclear. The Army and many defense agencies use DFAS to process their
travel vouchers, so the personnel savings for the Army and the defense
agencies were primarily related to reductions in DFAS's costs. In
discussions with DFAS officials, they were unable to estimate the actual
personnel savings that would result since they did not know (1) the number
of personnel, like those at the Air Force and Navy, that would simply be
transferred to other DFAS functions or (2) the number of personnel that
could be used to avoid additional hiring. For example, DFAS expects that
some of the individuals assigned to support the travel function could be
moved to support its ePayroll program. Since these positions would need to
be filled regardless of whether the travel function is reduced,
transferring personnel from travel to ePayroll would reduce DOD's overall
costs since DFAS would not have to hire additional individuals.

Savings Associated with Reduction of CTO Fees Are Unknown

According to the September 2003 economic analysis, DOD expected to realize
annual net savings of $31 million through reduced fees paid to the CTOs
because the successful implementation of DTS would enable the majority of
airline tickets to be acquired with either no or minimal intervention by
the CTOs. These are commonly referred to as "no touch" transactions.
However, DOD did not have a sufficient basis to estimate the number of
transactions that would be considered "no touch" since (1) the estimated
percentage of transactions that can be processed using the "no touch" was
not supported and (2) the analysis did not properly consider the effects
of components that use management fees, rather than transaction fees, to
compensate the CTOs for services provided. The weaknesses we identified
with the estimating process raise serious questions as to whether DOD will
realize substantial portions of the estimated annual net savings of $31
million.

  "No Touch" Transaction Volume Estimates Are Not Supported

DOD arrived at the $31 million of annual savings in CTO fees by estimating
that 70 percent of all DTS airline tickets would be considered "no touch"
and then multiplying these tickets by the savings per ticket in CTO fees.
However, a fundamental flaw in this analysis was that the 70 percent
assumption had no solid basis. We requested, but the PMO-DTS could not
provide, any analysis of travel data to support the assertion. Rather, the
sole support provided by the PMO-DTS was an article in a travel industry
trade publication.15 The article was not based on information related to
DTS, but rather on the experience of one private sector company.

15 American Express News Releases: American Express' Interactive Travel
Update (New York: Aug. 11, 2003),
http://corp.americanexpress.com/gcs/cards/us/ni/pr/081303.aspx .

  Navy Impact of CTO Management Fees Not Adequately Considered

The economic analysis assumed that DOD could save about $13.50 per "no
touch" ticket. Since that analysis, DOD has awarded one contract that
specifically prices transactions using the same model as that envisioned
by the economic analysis. This contract applies to the Defense Travel
Region 6 travel area.16 During calendar year 2005, the difference in fees
for "no touch" transactions and the transactions supported by the current
process averaged between $10 and $12, depending on when the fees were
incurred because the contract rates changed during 2005.17 In analyzing
travel voucher data for Region 6 for calendar year 2005, we found that the
reported "no touch" rate was, at best 47 percent-far less than the 70
percent envisioned in the economic analysis.

PMO-DTS program officials stated they are uncertain as to why the
anticipated 70 percent "no touch" was not being achieved. According to
PMO-DTS program officials, this could be attributed, in part, to the DOD
travelers being uncomfortable with the system and making reservations
without using a CTO. Although this may be one reason, other factors may
also affect the expected "no touch" fee. For example, we were informed
that determining the airline availability and making the associated
reservation can be accomplished, in most cases, rather easily. However,
obtaining information related to hotels and rental cars and making the
associated reservation can be more problematic because of the limitations
in the data that DTS is able to obtain from its commercial sources.
Accordingly, while a traveler may be able to make a "no touch" reservation
for the airline portion of the trip, the individual may need to contact
the CTO in order to make hotel or rental car reservations. When this
occurs, rather than paying a "no touch" fee to the CTO, DOD ends up paying
a higher fee, which eliminates the savings estimated in the economic
analysis.

16 Defense Travel Region 6 includes the Air Force and defense agencies in
the states of Kentucky, Illinois, Indiana, Iowa, Michigan, Minnesota,
Missouri, Nebraska, North Dakota, South Dakota, and Wisconsin. The
contract also applies to Army activities in 8 of the 11 states (excluding
Kentucky, Missouri, and Nebraska). As discussed later, the Navy uses a
management fee contract, and is therefore not included in the Defense
Travel Region 6 contract.

17 According to DTS officials, these savings are consistent with the DTS
contracts that have been awarded to small businesses. The average savings
per "no touch" ticket under these contracts is about $12.88. Because the
contractors are paid these fees directly by the traveler, they are unable
to determine the percentage of transactions that are actually paid using
the "no touch" rate.

The economic analysis assumed that (1) DOD would be able to modify the
existing CTO contracts to achieve a substantial reduction in fees paid to
a CTO when DTS was fully implemented across the department and (2) all
services would use the fee structure called for in the new CTO contracts.
The first part of the assumption is supported by results of the CTO
contract for DOD Region 6 travel. The fees for the DTS "no touch"
transactions were at least $10 less than if a CTO was involved in the
transactions. However, to date, the department has experienced difficulty
in awarding new contracts with the lower fee structure. On May 10, 2006,
the department announced the cancellation of the solicitation for a new
contract. According to the department, it decided that the solicitation
needed to be rewritten based on feedback from travel industry
representatives at a March 28, 2006, conference. The department
acknowledged that the "DTS office realized its solicitation didn't reflect
what travel agency services it actually needed."18 The department would
not say how the solicitation would be refined, citing the sensitivity of
the procurement process. The department also noted that the new
solicitation would be released soon, but provided no specific date.

The economic analysis assumed that the Navy would save about $7.5 million,
almost 25 percent, of the total savings related to CTO fees once DTS is
fully deployed. The economic analysis averaged the CTO fees paid by the
Army, the Air Force, and the Marine Corps-which amounted to about $18.71
per transaction-to compute the savings in Navy CTO fees. Using these data,
the assumption was made in the economic analysis that a fee of $5.25 would
be assessed for each ticket, resulting in an average savings of $13.46 per
ticket for the Navy ($18.71 minus $5.25).19 While this approach may be
valid for the organizations that pay individual CTO fees, it may not be
representative for organizations such as the Navy that pay a management
fee. The management fee charged the Navy is the same regardless of the
involvement of the CTO-therefore, the reduced "no touch" fee would not
apply.

We were informed by Navy DTS program officials that they were considering
continuing the use of management fees after DTS is fully implemented.
According to Navy DTS program officials, they paid about $14.5 million
during fiscal year 2005 for CTO management fees, almost $19 per ticket for
approximately 762,700 tickets issued. Accordingly, even if the department
arrives at a new CTO contract containing the new fee structure or fees
similar to those of Region 6, the estimated savings related to CTO fees
for the Navy will not be realized if the Navy continues to use the
management fee concept.

18 "DOD Retracts Solicitation for Travel Agency Services,"
FederalTimes.com (May 16, 2006), http://www.federaltimes.com/index.php ?
(downloaded June 14, 2006).

19 These savings translate to about 572,000 tickets annually.

Effective Implementation of Existing Policies Should Have Identified Problems
with the Economic Analysis

Effective implementation of DOD guidance would have detected the types of
problems discussed above and resulted in an economic analysis that would
have accomplished the stated objective of the process-to help ensure that
the funds invested in DTS were used efficiently and effectively. DOD
policy20 and OMB guidance21 require that an economic analysis be based on
facts and data and be explicit about the underlying assumptions used to
arrive at estimates of future benefits and costs. Since an economic
analysis deals with costs and benefits occurring in the future,
assumptions must be made to account for uncertainties. DOD policy
recognizes this and provides a systematic approach to the problem of
choosing the best method of allocating scarce resources to achieve a given
objective.

A sound economic analysis recognizes that there are alternative ways to
meet a given objective and that each alternative requires certain
resources and produces certain results. The purpose of the economic
analysis is to give the decision maker insight into economic factors
bearing on accomplishing the objectives. Therefore, it is important to
identify factors, such as cost and performance risks and drivers, which
can be used to establish and defend priorities and resource allocations.
The DTS economic analysis did not comply with the DOD policy, and the
weaknesses we found should have been detected had the DOD policy been
effectively implemented. The PMO-DTS had adequate warning signs of the
potential problems associated with not following the OMB and DOD guidance
for developing an effective economic analysis. For example, as noted
earlier, the Air Force and Navy provided comments when the economic
analysis was being developed that the expected benefits being claimed were
unrealistic. Just removing the benefits associated with personnel savings
from the Air Force and Navy would have reduced the overall estimated
program cost savings by almost 45 percent. This would have put increased
pressure on the credibility of using a 70 percent "no touch" utilization
rate. The following are examples of failures to effectively implement the
DOD policy on conducting economic analyses and the adverse effects on the
DTS economic analysis.

20 DOD Instruction 7041.3, Economic Analysis for Decisionmaking, November
7, 1995.

21 Office of Management and Budget, Circular No. A-94, Guidelines and
Discount Rates for Benefit-Cost Analysis of Federal Programs (Revised Jan.
18, 2006).

           o  The DTS life-cycle cost estimates portion of the economic
           analysis was not independently validated as specified in DOD's
           guidance.22 PMO-DTS officials acknowledged that there was not an
           independent assessment of the DTS life-cycle cost estimates.
           However, they noted that the department's Office of Program
           Analysis and Evaluation had provided comments on the economic
           analysis.23 Program Analysis and Evaluation officials informed us
           that they did not perform an independent assessment of the DTS
           economic analysis because the data were not available to validate
           the reliability of that analysis. Program Analysis and Evaluation
           officials also noted that they had raised similar concerns about
           the July 2003 economic analysis, but those issues had not been
           resolved when the September 2003 economic analysis was provided
           for their review. Because the September 2003 DTS life-cycle cost
           estimates were not independently assessed, the department did not
           have reasonable assurance that the reported estimates were
           realistic, that the assumptions on which the analysis was based
           were valid, or that the estimated rate of return on the investment
           could reasonably be expected to be realized.

           o  The September 2003 DTS economic analysis did not undertake an
           assessment of the effects of the uncertainty inherent in the
           estimates of benefits and costs, as required by DOD and OMB
           guidance.24 Because an economic analysis uses estimates and
           assumptions, it is critical that a sensitivity analysis be
           performed to understand the effects of the imprecision in both
           underlying data and modeling assumptions. This analysis is
           required since the estimates of future benefits and costs are
           subject to varying degrees of uncertainty. For example, according
           to

           DOD officials, the number of travel transactions has remained
           relatively stable over the years. On the other hand, as discussed
           previously, the number of transactions that can be processed as
           "no touch" is unknown. Sensitivity analysis refers to changing the
           value of a given variable in a model to gauge the effect of change
           on model results. More importantly, it identifies key
           elements-data and assumptions-as discussed above-and varies a
           single element while holding the others constant to determine what
           amount of change in that element is required to raise or lower the
           resulting dominant benefit and cost elements by a set amount. In
           this way, data and assumptions can be risk-ranked for
           decisionmaking and auditing. In the case of DTS, we requested that
           the PMO-DTS determine the effects of a change in "no touch"
           transaction percentage. With all other factors remaining the same,
           DTS would have to achieve a 35 percent "no touch" transaction rate
           just to break even-where tangible costs and benefits are equal.
           Had DOD performed such an analysis, it would have understood that
           depending solely on an industry trade publication as its support
           for the "no touch" transaction percentage had major implications
           on the potential savings.

           New Economic Analysis Not Required by DOD Criteria

           Although the September 2003 economic analysis was not based on
           supportable data, the department's criteria do not require that a
           new economic analysis be prepared. DTS has already completed all
           of the major milestones related to a major automated system, which
           require that an economic analysis be prepared or at least updated
           to reflect the current assumptions and the related costs and
           benefits. However, the fiscal year 2005 defense authorization
           act25 requires the periodic review, but not less than annually, of
           every defense business system investment. Further, the
           department's April 2006 guidance26 notes that the annual review
           process "provides follow-up assurance that information technology
           investments, which have been previously approved and certified,
           are managed properly, and that promised capabilities are delivered
           on time and within budget." If effectively implemented, this
           annual review process provides an excellent opportunity for DOD
           management to assess whether DTS is meeting its planned cost,
           schedule, and functionality goals. Going forward, such a review
           could serve as a useful management tool in making funding and
           other management decisions related to DTS.

                          DTS Remains Underutilized by the Military Services
           
			  Our September 2005 testimony and January 2006 report27 noted the
           challenge facing the department in attaining the anticipated DTS's
           utilization. While DOD has acknowledged the underutilization, we
           found that across DOD, the department does not have reasonable
           quantitative metrics to measure the extent to which DTS is
           actually being used. Presently, the reported DTS utilization is
           based on a DTS Voucher Analysis Model28 that was developed in
           calendar year 2003 using estimated data, but over the years has
           not been completely updated with actual data. While the military
           services have initiated actions to help increase the utilization
           of DTS, they pointed out that ineffective DTS training is a
           contributing factor to the lower than expected usage rate by the
           military services.

           
           Metrics to Measure DTS Utilization Are Inadequate

           The DTS Voucher Analysis Model was prepared in calendar year 2003
           and based on airline ticket and voucher count data that were
           reported by the military services and defense agencies, but the
           data were not verified or validated. Furthermore, PMO-DTS
           officials acknowledged that the model has not been completely
           updated with actual data as DTS continues to be implemented at the
           11,000 sites. We found that the Air Force is the only military
           service that submits monthly metrics to the PMO-DTS officials for
           their use in updating the DTS Voucher Analysis Model. Rather than
           reporting utilization based on individual site system utilization
           data, the PMO-DTS continues to rely on outdated information in the
           reporting of DTS utilization to DOD management and Congress. We
           have previously reported29 that best business practices indicate
           that a key factor of project management and oversight is the
           ability to effectively monitor and evaluate a project's actual
           performance against what was planned.

           In order to perform this critical task, best business practices
           require the adoption of quantitative metrics to help measure the
           effectiveness of a business system implementation and to
           continually measure and monitor results, such as system
           utilization. This lack of accurate and pertinent utilization data
           hinders management's ability to monitor its progress toward the
           DOD vision of DTS as the standard travel system, as well as to
           provide consistent and accurate data to Congress. With the shift
           of the DTS program to BTA, which now makes DTS an enterprisewide
           endeavor, improved metrics and training are essential if DTS is to
           be DOD's standard, integrated, end-to-end travel system for
           business travel.

           Table 3 presents DTS's reported percentage of utilization during
           the period October 2005 through April 2006. PMO-DTS officials
           calculated these utilization percentages by comparing the actual
           number of travel vouchers processed through DTS to the outdated
           universe of travel transaction data per the model, as described
           previously. Because the PMO-DTS was not able to identify the total
           number of travel vouchers that should have been processed through
           DTS (total universe of travel vouchers), the utilization
           percentages shown in table 3 may be over- or understated.

           Table 3: DTS Reported Utilization Percentage for the Period
           October 2005 through April 2006

           Source: PMO-DTS.

           PMO-DTS program officials confirmed that the reported utilization
           data were not based on complete data because the department did
           not have comprehensive information to identify the universe or the
           total number of travel vouchers that should be processed through
           DTS. PMO-DTS program and DTS military service officials agreed
           that the actual DTS utilization rate should be calculated by
           comparing actual vouchers being processed in DTS to the total
           universe of vouchers that should be processed in DTS. The universe
           would exclude those travel vouchers that cannot be processed
           through DTS, such as those related to permanent change of station
           travel.

           The Air Force was the only military service that attempted to
           obtain data on (1) the actual travel vouchers processed through
           DTS and (2) those travel vouchers eligible to be processed through
           DTS, but were not. These data were site specific. For example,
           during the month of December 2005, the PMO-DTS reported that at
           Wright-Patterson Air Force Base, 2,880 travel vouchers were
           processed by DTS, and the Air Force reported that another 2,307
           vouchers were processed through the legacy system-the Reserve
           Travel System (RTS). Of those processed through RTS, Air Force DTS
           program officials stated that 338 travel vouchers should have been
           processed through DTS. DTS Air Force program officials further
           stated that they submitted to the PMO-DTS the number of travel
           vouchers processed through RTS each month. These data are used by
           the PMO-DTS to update the DTS Voucher Analysis Model. However,
           neither the Air Force nor the PMO-DTS have verified the accuracy
           and reliability of the data. Therefore, the accuracy of the
           utilization rates reported for the Air Force by the PMO-DTS is not
           known. As shown in table 3, PMO-DTS officials reported utilization
           data for the Air Force from a low of 29 percent (January 2006) to
           a high of 48 percent (November 2005) during the 7-month period
           ending April 2006.

           Because Army and Navy DTS program officials did not have the
           information to identify the travel transactions that should have
           been processed through DTS, the Army and Navy did not have a basis
           for evaluating DTS utilization at their respective military
           locations and activities. Furthermore, Navy DTS program officials
           indicated that the utilization data that the PMO-DTS program
           officials reported for the Navy were not accurate. According to
           Navy DTS program officials, the Navy's primary source of
           utilization data was the monthly metrics reports provided by the
           PMO-DTS, but Navy DTS program officials questioned the accuracy of
           the Navy utilization reports provided by the PMO-DTS.

           o  For example, the Navy PMO-DTS utilization site report has a
           site name of Ballston, Va.; however, Ballston, Va. is not listed
           on the map site names on the DTS contractor's database. As a
           result, the PMO-DTS Navy utilization report for this location
           indicates no usage every month. Our analysis indicated that this
           was 1 of at least 33 similar instances where no usage was reported
           for a nonexistent location. Navy DTS program officials stated that
           an effort is underway to "re-map" all Navy organizations to the
           correct site name, but as of June 2006 this effort had not been
           completed.

           o  Another example indicates the inconsistencies that exist in the
           different information used by the Navy and the PMO-DTS program
           officials to report utilization rates for the Navy. The PMO-DTS
           program officials reported that the Navy had a total of 9,400
           signed, original vouchers processed through DTS during December
           2005; however, this is less than the 10,523 reported by the DTS
           contractor for the same month. According to Navy DTS program
           officials, they have not been able to confirm whether either
           figure is correct. Since the number of DTS vouchers is required to
           calculate utilization, the Navy is unable to determine the
           accuracy of the utilization metrics reported by the PMO-DTS
           officials, as shown in table 3.

           DOD Has Taken Steps to Improve DTS Utilization, but Further Action
			  Is Needed
			  
			  While the military services have issued various memorandums that
           direct or mandate the use of DTS to the fullest extent possible at
           those sites where DTS has been deployed, resistance still exists.
           As highlighted below, deployed sites are still using non-DTS
           systems, or legacy systems, to process TDY travel.

           o  The Army issued a memorandum in September 2004 directing each
           Army installation to fully disseminate DTS to all travelers within
           90 to 180 days after Initial Operating Capability30 (IOC) at each
           installation.31 Subsequently in September 2005, DFAS officials
           reported that 390,388 travel vouchers were processed through the
           Army's legacy system-the Windows Integrated Automated Travel
           System, but DFAS officials could not provide a breakout of how
           many of the 390,388 travel vouchers should have been processed
           through DTS.

           o  The Air Force issued a memorandum in November 2004 that
           stressed the importance of using DTS once it was implemented at an
           installation. The Air Force memorandum specifically stated that
           business, local, and group travel vouchers should be
           electronically processed through DTS and that travel claims should
           not be submitted to the local finance office for processing.
           However, we found that Air Force travelers continued to process
           travel claims through legacy systems, such as RTS. For example,
           during the month of November 2005, the Air Force reported that
           3,277 business vouchers, 1,875 local vouchers, and 1,815 group
           vouchers were processed through RTS that should have been
           processed through DTS. Additionally, a DFAS internal review32
           analyzed Air Force vouchers during the period January 2005 through
           June 2005, at locations where DTS was deployed, and found that Air
           Force travelers used legacy systems to process 79 percent of all
           routine TDY transactions.

           o  The Navy issued a memorandum in May 2005 that directed the use
           of DTS to generate travel orders throughout all Navy locations.
           Navy DTS program officials reported in an April 2006 briefing that
           18,300 travel vouchers were processed in DTS during the month of
           March 2006, but that over 90,000 travel vouchers were still being
           processed monthly through the Integrated Automated Travel System-a
           legacy system.

           Thus, despite memoranda issued by the military services, it
           appears that DTS continues to be underutilized by the military
           services. As discussed in our September 2005 testimony and January
           2006 report,33 the unnecessary continued use of the legacy travel
           systems results in the inefficient use of funds because the
           department is paying to operate and maintain duplicative systems
           that perform the same function-travel.

           Besides the memorandums, DOD is taking other actions to increase
           DTS utilization as the following examples illustrate.

           o  The Assistant Secretary of the Army for Financial Management
           (Financial & Accounting Oversight Directorate) holds monthly
           Senior Focus Group meetings with the installation leadership of
           major commands to discuss DTS utilization issues and possible
           corrective actions.

           o  The Navy conducts quarterly video and telephone conferences
           with major commands and contacts commands with low usage to
           determine the causes for low DTS usage.

           o  The PMO-DTS conducts monthly working group meetings with the
           military service and defense agency DTS program officials to
           discuss DTS functionality issues and concerns, DTS usage, and
           other related DTS issues.

           Although the military services have issued various memorandums
           aimed at increasing the utilization of DTS, the military service
           DTS program officials all pointed to ineffective training as a
           primary cause of DTS not being utilized to a far greater extent.
           The following examples highlight the concerns raised by the
           military service officials.

           o  Army DTS program officials emphasized that the DTS system is
           complex and the design presents usability challenges for
           users-especially for first-time or infrequent users. They added
           that a major concern is that there is no PMO-DTS training for
           existing DTS users as new functionality is added to DTS. These
           officials stated that the PMO-DTS does not do a good job of
           informing users about functionality changes made to the system. We
           inquired if the Help Desk was able to resolve the users' problems,
           and the Army DTS officials simply stated "no." The Army officials
           further pointed out that it would be beneficial if the PMO-DTS
           improved the electronic training on the DTS Web site and made the
           training documentation easier to understand. Also, improved
           training would help infrequent users adapt to system changes. The
           Army officials noted that without some of these improvements to
           resolve usability concerns, DTS will continue to be extremely
           frustrating and cumbersome for travelers.

           o  Navy DTS program officials stated that DTS lacks adequate
           user/traveler training. The train-the-trainer concept of training
           system administrators who could then effectively train all their
           travelers has been largely unsuccessful. According to Navy
           officials, this has resulted in many travelers and users
           attempting to use DTS with no or insufficient training. The effect
           has frustrated users at each step of the travel process and has
           discouraged use of DTS.

           o  Air Force officials stated that new DTS system releases are
           implemented with known problems, but the sites are not informed of
           the problems. Workarounds are not provided until after the sites
           begin encountering problems. Air Force DTS program officials
           stated that DTS releases did not appear to be well tested prior to
           implementation. Air Force officials also stated that there was
           insufficient training on new functionality. PMO-DTS and DTS
           contractor program officials believed that conference calls to
           discuss new functionality with the sites were acceptable training,
           but Air Force officials did not agree. The Air Force finance
           office was expected to fully comprehend the information received
           from those conference calls and provide training on the new
           functionality to users/approvers, but these officials stated that
           this was an unrealistic expectation.

           Previously Reported DTS Requirements Management and Testing
			  Deficiencies Have Not Been Resolved
			  
			  Our September 2005 testimony and January 200634 report noted
           problems with DTS's ability to properly display flight information
           and traced those problems to inadequate requirements management
           and testing. DOD stated that it had addressed those deficiencies
           and in February 2006, we again tested the system to determine
           whether the stated weaknesses had been addressed. We found that
           similar problems continue to exist. We also identified additional
           deficiencies in DTS's ability to display flights that comply with
           the Fly America Act.35 DTS's inability to display flights that
           comply with the Fly America Act places the traveler who purchases
           a ticket or the individual authorizing, certifying, or disbursing
           a payment made when a ticket is paid for directly by DOD through a
           centrally billed account at unnecessary risk of personal
           liability. Once again, these problems can be traced to ineffective
           requirements management and testing processes. Properly defined
           requirements are a key element in systems that meet their cost,
           schedule, and performance goals since they define (1) the
           functionality that is expected to be provided by the system and
           (2) the quantitative measures by which to determine through
           testing whether that functionality is operating as expected.

           We briefed PMO-DTS officials on the results of our tests and in
           May 2006 the officials agreed that our continued concerns about
           the proper display of flight information and compliance with the
           Fly America Act were valid. PMO-DTS officials stated that the DTS
           technology refresh, which is to be completed in September 2006,
           should address some of our concerns. While these actions are a
           positive step forward, they do not address the fundamental problem
           that DTS's requirements are still ambiguous and conflicting-a
           primary cause of the previous problems. Until a viable
           requirements management process is developed and effectively
           implemented, the department (1) cannot develop an effective
           testing process and (2) will not have reasonable assurance the
           project risks have been reduced to acceptable levels.

           Providing Complete Flight Information Has Been a Continuing Problem
			  
			  In our earlier testimony and report,36 we noted that DOD did not
           have reasonable assurance that the flights displayed met the
           stated DOD requirements. Although DOD stated in each case that our
           concerns had been addressed, subsequent tests found that the
           problems had not been corrected. Requirements represent the
           blueprint that system developers and program managers use to
           design, develop, and acquire a system. Requirements should be
           consistent with one another, verifiable, and directly traceable37
           to higher-level business or functional requirements. It is
           critical that requirements be carefully defined and that they flow
           directly from the organization's concept of operations (how the
           organization's day-to-day operations are or will be carried out to
           meet mission needs). Improperly defined or incomplete requirements
           have been commonly identified as a cause of system failure and
           systems that do not meet their cost, schedule, or performance
           goals.

           Requirements represent the foundation on which the system should
           be developed and implemented. As we have noted in previous
           reports,38 because requirements provide the foundation for system
           testing, significant defects in the requirements management
           process preclude an entity from implementing a disciplined testing
           process. That is, requirements must be complete, clear, and well
           documented to design and implement an effective testing program.
           Absent this, an organization is taking a significant risk that its
           testing efforts will not detect significant defects until after
           the system is placed into production. Our February 2006 analysis
           of selected flight information disclosed that DOD still did not
           have reasonable assurance that DTS displayed flights in accordance
           with its stated requirements. We analyzed 15 U.S. General Services
           Administration (GSA) city pairs,39 which should have translated
           into 246 GSA city pair flights for the departure times selected.
           However, we identified 87 flights that did not appear on one or
           more of the required listings based on the DTS requirements. For
           instance, our analysis identified 44 flights appearing on other
           DTS listings or airline sites that did not appear on the 9:00 am
           DTS listing even though those flights (1) met the 12-hour flight
           window40 and (2) were considered GSA city pair flights-two of the
           key DTS requirements the system was expected to meet.

           After briefing PMO officials on the results of our analysis in
           February 2006, the PMO-DTS employed the services of a contractor
           to review DTS to determine the specific cause of the problems and
           recommend solutions. In a March 2006 briefing, the PMO-DTS
           acknowledged the existence of the problems, and identified two
           primary causes. First, part of the problem was attributed to the
           methodology used by DTS to obtain flights from the Global
           Distribution System (GDS). The PMO-DTS stated that DTS was
           programmed to obtain a "limited" amount of data from GDS in order
           to reduce the costs associated with accessing GDS. This helps to
           explain why flight queries we reviewed did not produce the
           expected results. To resolve this particular problem, the PMO-DTS
           proposed increasing the amount of data obtained from GDS. Second,
           the PMO-DTS acknowledged that the system testing performed by the
           contractor responsible for developing and operating DTS was
           inadequate and, therefore, there was no assurance that DTS would
           provide the data in conformance with the stated requirements. This
           weakness was not new, but rather reconfirms the concerns discussed
           in our September 2005 testimony and January 2006 report41 related
           to the testing of DTS.

           Our analysis also found that DOD did not have reasonable assurance
           that the system displayed flights in compliance with the
           requirements of the Fly America Act. In 1996, Congress assigned
           the Administrator, GSA, the responsibility42 to determine the
           situations for which appropriated funds could be used consistent
           with the Fly America Act, and GSA has published its rules in the
           Federal Travel Regulation (FTR).43 Within the basic guidelines
           that GSA publishes, agencies must establish "internal procedures"
           to ensure that agency reimbursements with federal funds for
           travelers' air carrier expenses are made only in compliance with
           the Fly America Act and the FTR rules. As a result, DTS places the
           traveler who purchases a ticket or the individual authorizing,
           certifying, or disbursing a payment made when a ticket is paid for
           directly by DOD-such as those tickets purchased using a centrally
           billed account-at unnecessary risk of personal liability. DOD
           guidance expressly states that for code-sharing44 airline tickets
           related to foreign travel (1) the entire airline ticket must be
           issued by and on the U.S.-flag carrier (not necessarily the
           carrier operating the aircraft) and (2) the flight must be between
           a centennial United States and a foreign destination. If these
           conditions are not met, DOD requires a determination that a
           U.S.-flag carrier is not available or use of a non-U.S.-flag
           carrier is necessary.45 These requirements are commonly referred
           to as the Fly America Act requirements. According to PMO-DTS
           officials, DTS's requirements are intended to comply with the Fly
           America Act. However, our analysis of March 2006 flight display
           data identified several instances in which flights were displayed
           to the DOD traveler that did not meet the requirements of the Fly
           America Act. For example, six of the first seven flights displayed
           between Santiago, Chile, and San Antonio, Texas, did not appear to
           comply with the Fly America Act requirements since they did not
           involve a U.S.-flag carrier. More importantly, several flights
           that appeared later in the listing and involved U.S.-flag carriers
           were more advantageous to the traveler because they required less
           actual travel time. Figure 1 shows the DTS display of flights.

22 Department of Defense Instruction 5000.2, Operation of the Defense
Acquisition System, May 12, 2003.

23 Department of Defense Regulation 5000.2-R, Interim Defense Acquisition
Guidebook (Oct. 30, 2002), para.c4.5.1.61, required the Office of Program
Analysis and Evaluation to assess certain aspects of the economic
analysis.

24 Department of Defense Instruction 7041.3, Economic Analysis of
Decisionmaking (Nov. 7, 1995), and Office of Management and Budget Revised
Circular No. A-94, Guidelines and Discount Rates for Benefit-Cost Analysis
of Federal Programs (Oct. 29, 1992).

25 Ronald W. Reagan National Defense Authorization Act for Fiscal Year
2005, Pub. L. No. 108-375, S: 332, 118 Stat. 1811, 1851-56 (Oct. 28, 2004)
(codified, in part, at 10 U.S.C. S:S: 186, 2222).

26 DOD, DOD IT Business Systems Investment Review Process: Investment
Certification and Annual Review Process User Guidance (Apr. 10, 2006).

27 GAO-05-998T and GAO-06-18 .

28 DOD developed a model in calendar year 2003 that compares the expected
usage against the actual usage. The expected usage is obtained by using
historical data, such as ticket counts, to determine the expected number
of vouchers processed by a given location. For example, if a location had
1,000 vouchers as its expected number of vouchers per the model, but now
processes 750 actual vouchers through DTS, then the PMO model considers
that that location has achieved a 75 percent utilization rate. It then
takes the individual computations for each DTS location and "rolls them
up" to determine the total utilization for individual service performance
on a monthly basis.

29 GAO, Financial Management Systems: Additional Efforts Needed to Address
Key Causes of Modernization Failures, GAO-06-184 (Washington, D.C.: Mar.
15, 2006), and Financial Management Systems: Lack of Disciplined Processes
Puts Implementation of HHS' Financial System at Risk, GAO-04-1008
(Washington, D.C.: Sept. 23, 2004).

Month         Army Navy Air Force 
October 2005    46   28        33 
November 2005   59   32        48 
December 2005   50   27        38 
January 2006    40   20        29 
February 2006   54   30        40 
March 2006      66   39        47 
April 2006      59   35        40 
Average         53   30        39 

30 When a military service location has declared Initial Operating
Capability (IOC), that location moves into an "operational phase" in which
all units/activities are fully proliferated for use of DTS.

31 The memorandum included a list of sites to which DTS should be fully
disseminated and the types of vouchers that must be processed through DTS.

32 Department of Defense, Defense Finance and Accounting Service, Internal
Review, Audit of the Defense Travel System (DTS), October 2005 - February
2006, CO06SRP005AR (Arlington, Va.: Feb. 22, 2006).

33 GAO-05-998T and GAO-06-18 .

 Previously Reported DTS Requirements Management and Testing Deficiencies Have
                               Not Been Resolved

34 GAO-05-998T and GAO-06-18 .

35 49 U.S.C. S:40118, commonly referred to as the Fly America Act,
requires federal employees and their dependents, consultants, contractors,
grantees, and others performing U.S. government-funded air travel to
travel by U.S. certificated flag air carriers except under certain
circumstances, such as when travel by foreign air carrier is a matter of
necessity as defined by the statute or when U.S. certificated flag air
carrier service is not available. See 41C.F.R. S: 301-10.135.

Providing Complete Flight Information Has Been a Continuing Problem

36 GAO-05-998T and GAO-06-18 .

37 Traceability allows the user to follow the life of the requirement both
forward and backward through these documents and from origin through
implementation. Traceability is also critical to understanding the
parentage, interconnections, and dependencies among the individual
requirements. This information in turn is critical to understanding the
impact when a requirement is changed or deleted.

38 See, for example, GAO-04-1008 and Army Depot Maintenance: Ineffective
Oversight of Depot Maintenance Operations and System Implementation
Efforts, GAO-05-441 (Washington, D.C.: June 30, 2005).

39 GSA awards contracts to airlines to provide flight services between
pairs of cities. This is commonly referred to as the GSA city pair
program. Under this program (1) no advanced ticket purchases are required,
(2) no minimum or maximum length of stay is required, (3) tickets are
fully refundable and no charges are assessed for cancellations or changes,
(4) seating is not capacity controlled (i.e., as long as there is a
coach-class seat on the plane, the traveler may purchase it), (5) no
blackout dates apply, (6) fare savings average 70 percent over regular
walk-up fares, and (7) fares are priced on one-way routes permitting
agencies to plan for multiple destinations. We selected the first 15 city
pairs that were provided by DOD to GSA in support of a GSA study on
accuracy of flight displays and fare information by DTS and the GSA
eTravel providers.

40 A flight window is the amount of time before and after a specified time
and is used for determining the flights that should be displayed. DTS uses
a 12-hour flight window for domestic flights and a 24-hour flight window
for foreign flights. The system is also expected to display up to 25
flights for the flight window.

41 GAO-05-998T and GAO-06-18 .

42 See 49 U.S.C. S: 40118(c).

43 See 41 C.F.R. S:301-10.143.

44 A code-share agreement is a marketing arrangement in which an airline
places its designator code on a flight operated by another airline and
sells, advertises, and issues tickets as its own flights. U.S. carriers
must obtain authorization for foreign code-share operations from the
Department of Transportation.

45 JTR, C2204-C, and JFTR, U3125-C.

Figure 1: March 2006 DTS Display of Flights between Santiago, Chile, and
San Antonio, Texas

According to DTS program officials, after our discussions relating to the
flight displays and compliance with the Fly America Act, they did a
"requirements scrub" to define the requirements that should be used to
display flights, including those requirements relating to displaying
flights that comply with the Fly America Act. The previous requirement
stated that "DTS shall examine international trip records for compliance
with DOD policy on the use of non-U.S.-flag carriers." The revised
requirement relating to international flights stated that the system
should display flights that are (1) part of the GSA city pair program or
(2) offered by U.S. carriers. If the system cannot find flights that meet
these criteria, then the system is expected to instruct the user to
contact their CTO to arrange the flight. According to PMO-DTS officials,
this change has been incorporated into the production system. We conducted
a limited nonstatistical test to determine if the examples of flights not
complying with the Fly America Act identified in our earlier tests had
been eliminated and found that these flights no longer appeared on the DTS
displayed flights. However, as we noted, the DOD policy is compliant with
the Fly America Act requirements and this was a DTS requirement in effect
when we identified the examples of flight displays not complying with the
Fly America Act. In effect, this is another example of (1) inadequate
testing by the DTS contractor and (2) DOD's inability to ensure the system
is meeting its requirements. Until DOD effectively analyzes and properly
documents the functionality it desires, it has little assurance that the
proper requirements have been defined.

DOD's Planned Corrective Actions Will Not Address Fundamental Requirements
Management Problems

While DOD's planned actions, if effectively implemented, should address
several of the specific weaknesses we identified related to flight
displays and the Fly America Act, they fall short of addressing the
fundamental problems that caused those weaknesses-inadequate requirements
management. DTS's requirements continue to be ambiguous. For example, a
system requirement was changed to "display," that is, show the fares
relating to the full GSA city pair fare only if the GSA city pair fare
with capacity limits46 was not available. Based upon information provided
by PMO-DTS officials, after the requirement was supposed to have been
implemented, both fare types were shown on the DTS display screen. PMO-DTS
officials stated that although both fares were shown, DTS was still
expected to book the lower fare and that the requirement was really
designed to ensure that the lower fare was booked. This requirement is
ambiguous because it is not clear what the word "display" means in this
context. Based upon the stated requirement, the most common interpretation
would be that the word display implies information that is provided (or
shown) to the DOD traveler. However, based on the PMO-DTS official's
explanation, the word display, in fact, means the fare that is booked.
This type of ambiguity was one cause of problems we noted in the past
where testing did not identify system defects and DTS did not properly
display the proper flight information to the user.

46 Several GSA city pair flights have two contract fares. These fares are
commonly referred to as an unrestricted GSA city pair fare and a GSA city
pair fare with capacity limits. The latter fare is cheaper than the
unrestricted GSA city pair fare and applies to a limited number of seats
when available. However, it has no other restrictions.

Furthermore, DOD is currently undergoing a technology upgrade of DTS that
is scheduled for completion by September 30, 2006. This technology upgrade
is expected to provide additional functionality; however, DOD still has
not adequately defined the requirements that are needed to define flight
displays for DOD travelers. According to DTS program officials and the
contractor responsible for the technology upgrade, the upgrade is intended
to do the following:

           o  Replace the current display of up to 25 flights on one page in
           a predetermined order47 and separate the 25 flights into three
           categories-GSA city pair flights, Other Government Fares, and
           Other Unrestricted Flights-and then sort the flights by additional
           criteria such as elapsed travel time (rather than the current
           flight time), time difference from the requested departure time,
           number of stops, and whether the flight is considered a direct
           flight. This approach, if effectively implemented, addresses one
           problem we noted with the current process where flight time48
           rather than elapsed travel time is used as one of the sorting
           criteria. It will also present flights that have the shortest
           duration in relation to the requested departure time at the top of
           the listing.

           o  Display the prices on all flights returned to the traveler. The
           current system displays the prices for the GSA city pair flights
           and allows the traveler to request prices for up to 10 additional
           flights at a time. This significantly improves the ability of the
           system to present information to the traveler that can be used to
           select the best flight for the government and allows the system to
           help ensure that the lowest cost flights are selected by the
           user.49 This is especially true when a GSA city pair fare is not
           available. According to DOD officials, it is cost prohibitive to
           obtain the pricing information for non-GSA city pair flights using
           the current technology.50

           Although these planned improvements should provide the DOD
           traveler with better travel information, they still fall short of
           adequately defining the requirements that should be used for
           displaying flights. For example, DOD has retained a requirement to
           display 25 flights for each inquiry. However, it has not
           determined (1) whether the rationale for that requirement is valid
           and (2) under what conditions flights that are not part of the GSA
           city pair program should be displayed. For example, we found that
           several DTS flights displayed to the user "overlap"51 other
           flights. Properly validating the requirements would allow DOD to
           obtain reasonable assurance that its requirements properly define
           the functionality needed and the business rules necessary to
           properly implement that functionality. As previously noted,
           requirements that are unambiguous and consistent are fundamental
           to providing reasonable assurance that a system will provide the
           desired functionality. Until DOD improves DTS requirement
           management practices, it will not have this assurance.

           Conclusions
			  
			  Overhauling the department's antiquated travel management
           practices and systems has been a daunting challenge for DOD. While
           it was widely recognized that this was a task that needed to be
           accomplished and savings could result, the underlying assumptions
           in support of those savings are not based on reliable data and
           therefore it is questionable whether the anticipated savings will
           materialize. Even though the overall savings are questionable, the
           successful implementation of DTS is critical to reducing the
           number of stovepiped, duplicative travel systems throughout the
           department. We have reported on numerous occasions that reducing
           the number of business systems within DOD can translate into
           savings that can be used for other mission needs. Furthermore, the
           shift of DTS to BTA, which makes DTS an enterprisewide endeavor,
           should help in making DTS the standard integrated, end-to-end
           travel system for business travel. Management oversight is
           essential for this to become a reality. Equally important,
           however, will be the department's ability to resolve the
           long-standing difficulties that DTS has encountered with its
           requirements management and system testing. Until these issues are
           resolved, more complete utilization of DTS will be problematic.

           Recommendations for Executive Action
			  
			  To improve the department's management and oversight of DTS, which
           has been declared a DOD enterprise business system, we recommend
           that the Secretary of Defense direct the Under Secretary of
           Defense (Personnel and Readiness) and the Director, Business
           Transformation Agency, to jointly take the following four actions:

           o  Evaluate the cost effectiveness of the Navy continuing with the
           CTO management fee structure versus adopting the revised CTO fee
           structure, once the new contracts have been awarded.

           o  Develop a process by which the military services develop and
           use quantitative data from DTS and their individual legacy systems
           to clearly identify the total universe of DTS-eligible
           transactions on a monthly basis. At a minimum, these data should
           be used to update the DTS Voucher Analysis Model to report DTS
           actual utilization rates.

           o  Require the PMO-DTS to provide a periodic report on the
           utilization of DTS to the Under Secretary of Defense (Personnel
           and Readiness) and the Director, Business Transformation Agency,
           once accurate data are available. The report should continue until
           the department has reasonable assurance that DTS is operating as
           intended at all 11,000 locations. The report should identify at a
           minimum (1) the number of defense locations at which DTS has been
           deployed, (2) the extent of DTS utilization at these sites, (3)
           steps taken or to be taken by the department to improve DTS
           utilization, and (4) any continuing problems in the implementation
           and utilization of DTS.

           o  Resolve inconsistencies in DTS requirements, such as the 25
           flight display, by properly defining the (1) functionality needed
           and (2) business rules necessary to properly implement the needed
           functionality.

           Agency Comments and Our Evaluation
			  
			  We received written comments on a draft of this report from the
           Under Secretary of Defense (Personnel and Readiness), which are
           reprinted in appendix II. DOD concurred with three and partially
           concurred with one of the recommendations. In regard to the
           recommendations with which the department concurred, it briefly
           outlined the actions it planned to take in addressing two of the
           three recommendations. For example, the department noted the
           difficulties in obtaining accurate utilization data from the
           existing legacy systems, but stated that the Office of the Under
           Secretary of Defense (Personnel and Readiness) and BTA will
           evaluate methods for reporting actual DTS utilization.

           Additionally, DOD noted that the Defense Travel Management Office
           developed and implemented a requirements change management process
           on May 1, 2006. In commenting on the report, the department stated
           that this process is intended to define requirements and track the
           entire life cycle of the requirements development process. As
           reiterated in this report, and discussed in our September 2005
           testimony and January 2006 report,52 effective requirements
           management has been an ongoing concern, and we fully support the
           department's efforts to improve its management oversight of DTS's
           requirements. In this regard, the department needs to have in
           place a process that provides DOD reasonable assurance that (1)
           requirements are properly documented and (2) requirements are
           adequately tested as recommended in our January 2006 report.53
           This process should apply to all existing requirements as well as
           any new requirements. As discussed in this report, we reviewed
           some of the requirements in May 2006, that were to have followed
           the new requirements management process, and found problems
           similar to those noted in our January 2006 report. While we did
           not specifically review the new process, if it does not include an
           evaluation of existing requirements, the department may continue
           to experience problems similar to those we previously identified.

           DOD partially concurred with our recommendation to evaluate the
           cost effectiveness of the Navy continuing with the CTO management
           fee structure. DOD stated that all military service secretaries
           should participate in an evaluation to determine the most
           cost-effective payment method to the CTOs. DOD's response
           indicated that the Defense Travel Management Office is currently
           procuring commercial travel services for DOD worldwide in a manner
           that will ensure evaluation of cost effectiveness for all
           services. If DOD proceeds with the actions outlined in its
           comments, it will meet the intent of our recommendation.

           Finally, DOD strongly objected to our finding that the personnel
           savings are unrealistic. In its comments, the department stated
           that DOD is facing an enormous challenge and the department
           continues to identify efficiencies and eliminate redundancies to
           help leverage available funds. We fully recognize that the
           department is attempting to improve the efficiency and
           effectiveness of its business operations. In fact, the Comptroller
           General of the United States testified in August 2006 that
           increased commitment by the department to address DOD's numerous
           challenges represents an improvement over past efforts.54

           The fact remains, however, that the results of an economic
           analysis are intended to help management decide if future
           investments in a given endeavor are worthwhile. In order to
           provide management with this information it is imperative that the
           underlying assumptions in an economic analysis be supported by
           valid assumptions. The September 2003 economic analysis noted that
           personnel savings of $54.1 million, as shown in table 2 of this
           report, would be realized by the department annually for fiscal
           years 2009 through 2016. However, based upon our review and
           analysis of documentation and discussion with department personnel
           we found that the underlying assumptions in support of the $54.1
           million were not valid.

           Furthermore, as noted in the report Air Force and Navy DTS program
           officials stated that they did not anticipate a reduction in the
           number of personnel with the full implementation of DTS. Further,
           as discussed in the report, the Naval Cost Analysis Division
           review of the DTS economic analysis noted that approximately 40
           percent of the Navy's total costs, including personnel costs, in
           the DTS life-cycle cost estimates could not be validated because
           credible supporting documentation was lacking.

           The report does note that Air Force and Navy DTS program officials
           noted that while they did not anticipate a reduction in the number
           of personnel, there would be a shifting of personnel to other
           functions. The report further points out that DOD officials
           responsible for reviewing economic analyses stated that while
           shifting personnel to other functions is considered a benefit, it
           should be considered an intangible benefit rather than tangible
           dollar savings since the shifting of personnel does not result in
           a reduction of DOD expenditures. Additionally, in its comments the
           department provided no new data that was counter to our finding.

           We are sending copies of this report to the Secretary of Defense;
           Under Secretary of Defense (Comptroller); the Under Secretary of
           Defense for Acquisition, Technology, and Logistics; the Under
           Secretary of Defense (Personnel and Readiness); the Director,
           Business Transformation Agency; and the Director, Office of
           Management and Budget. Copies of this report will be made
           available to others upon request. In addition, the report is
           available at no charge on the GAO Web site at http://www.gao.gov.

           If you or your staff have any questions on matters discussed in
           this report, please contact McCoy Williams at (202) 512-9095 or
           [email protected] or Keith A. Rhodes at (202) 512-6412 or
           [email protected] . Contact points for our Offices of Congressional
           Relations and Public Affairs may be found

           on the last page of this report. Key contributors to this report
           are listed in appendix III.

           McCoy Williams Director Financial Management and Assurance

           Keith A. Rhodes Chief Technologist Applied Research and Methods
           Center for Technology and Engineering

           List of Congressional Addressees

           The Honorable John Warner Chairman The Honorable Carl Levin
           Ranking Minority Member Committee on Armed Services United States
           Senate

           The Honorable Ted Stevens Chairman The Honorable Daniel K. Inouye
           Ranking Minority Member Subcommittee on Defense Committee on
           Appropriations United States Senate

           The Honorable Susan M. Collins Chairman The Honorable Joseph I.
           Lieberman Ranking Minority Member Committee on Homeland Security
           and Governmental Affairs United States Senate

           The Honorable Duncan L. Hunter Chairman The Honorable Ike Skelton
           Ranking Minority Member Committee on Armed Services House of
           Representatives

           The Honorable C. W. Bill Young Chairman The Honorable John P.
           Murtha Ranking Minority Member Subcommittee on Defense Committee
           on Appropriations House of Representatives

           The Honorable Tom Davis Chairman The Honorable Henry A. Waxman
           Ranking Minority Member Committee on Government Reform House of
           Representatives

           The Honorable Norm Coleman Chairman Permanent Subcommittee on
           Investigations Committee on Homeland Security and Governmental
           Affairs United States Senate

           The Honorable Tom Coburn Chairman Subcommittee on Federal
           Financial Management, Government Information and International
           Security Committee on Homeland Security and Governmental Affairs
           United States Senate

           Appendix I: Scope and Methodology
			  
			  To assess the reasonableness of the key assumptions made by DOD to
           arrive at the net annual estimated savings of over $56 million
           shown in the September 2003 economic analysis addendum, we (1)
           ascertained if the economic analysis was prepared in accordance
           with the prescribed standards, (2) analyzed two key assumptions
           that represent the largest dollar savings for the DTS program, and
           (3) analyzed the supporting documentation related to these two
           assumptions to determine whether the assumptions were valid.
           Furthermore, we met with the military services and DFAS officials
           to ascertain their specific concerns with the estimated savings.
           Further, we met with Program Analysis and Evaluation officials to
           identify any issues they had with the DTS estimated savings. In
           performing this body of work, we relied heavily upon the expertise
           of our Applied Research and Method's Center for Economics.

           To determine the actions being taken to enhance the utilization of
           DTS, we met with military services officials to obtain an
           understanding of the specific actions that were being taken. In
           addition, we obtained and reviewed various memorandums related to
           the utilization of DTS. We also obtained an overview of the method
           and data used by the PMO-DTS to report the rate of DTS utilization
           for the various DOD components. We also met with the military
           services to ascertain how they use the PMO-DTS data to monitor
           their respective utilization and whether they augment these data
           with any other data and if so, the source of those data.

           To ascertain whether DOD has reasonable assurance that the testing
           of DTS was adequate, and thereby ensure accurate flight
           information was displayed, we met with Northrop Grumman and the
           PMO-DTS officials to obtain an explanation of the corrective
           actions that were to have been implemented. To ascertain if the
           noted corrective actions have been successfully implemented, we
           analyzed 246 GSA city pair flights to determine if the information
           being displayed to the traveler was consistent with DTS's stated
           requirement.

           We did not review the accuracy and reliability of the specific
           dollar amounts shown in the September 2003 economic analysis.
           Given the department's previously reported problems related to
           financial management,1 we have no assurance that the underlying
           data supporting the economic analysis were complete. Furthermore,
           our emphasis was directed more towards the validity of the
           assumptions that were used to arrive at the net annual estimated
           savings of over $56 million. We determined that the data were
           sufficiently reliable for the purpose of this audit. We performed
           our audit work from October 2005 through July 2006 in accordance
           with U.S. generally accepted government auditing standards.

           We requested comments on a draft of this report from the Secretary
           of Defense or his designee. We received written comments from the
           Under Secretary of Defense (Personnel and Readiness), which are
           reprinted in appendix II.

           Appendix II: Comments from the Department of Defense


Appendix III: GAO Contacts and Staff Acknowledgments

                                  GAO Contacts

           McCoy Williams, (202) 512-9095 or [email protected] Keith A.
           Rhodes, (202) 512-6412 or [email protected]
			  
			   Acknowledgments

           In addition to the above contacts, the following individuals made
           key contributions to this report: Darby Smith, Assistant Director;
           J. Christopher Martin, Senior-Level Technologist; F. Abe Dymond,
           Assistant General Counsel; Beatrice Alff; Harold Brumm, Jr.;
           Francine DelVecchio; Jason Kelly; and Tarunkant Mithani.
			  

47 Under the current release, DTS will attempt to display up to 25 flights
in two categories-GSA city pairs and other. The flights within GSA city
pairs are then displayed according to elapsed travel time.

48 Flight time is the actual time a plane is in the air while elapsed
travel time is the total time from the original departure to the ultimate
arrival. For non stop flights, the times are the same. However, in cases
of connecting flights, the "layover" time is only included in the elapsed
travel time.

49 According to DOD officials, once the display of pricing information is
implemented, the system will require a justification when the lowest cost
flight is not selected. This edit would be similar to the edit for GSA
city pair fares, which requires the user to provide a justification if a
GSA city pair flight is available but not selected. When a user does not
select the lowest cost fare, unless it is a GSA city pair fare, the user
would be required to provide a justification. A justification is not
expected when the user selects a GSA city pair fare rather than a lower
cost fare, assuming one is available, which is displayed by DTS because of
government policy that encourages the use of the GSA city pair program.

50 DOD estimates that the current technological approach-querying the GDS
for the prices of the flights displayed to the user-would cost about $6
million for the 3 million trips that DTS is expected to book each year.
DOD did not provide the estimated fees that will be paid for the
technology that will enable these prices to be displayed.

51 For example, DTS displayed a GSA city pair flight between Washington,
D.C., and Atlanta, Ga. that departed at 10:05 a.m. and arrived at 1:50
p.m. This flight "overlapped" two other GSA city pair direct flights that
were available and required less travel time. One flight left at 10:05
a.m. and arrived at 12:02 p.m. while another left at 11:05 a.m. and
arrived at 12:56 p.m. Furthermore, DTS displayed a non-GSA city pair
flight that left at 9:20 a.m. and arrived at 1:05 p.m. This flight did not
meet any of the acceptable criteria for not using a GSA city pair flight.

52 GAO-05-998T and GAO-06-18 .

53 GAO-06-18 .

54 GAO, Department of Defense: Sustained Leadership Is Critical to
Effective Financial and Business Management Transformation, GAO-06-1006T
(Washington, D.C.: Aug. 3, 2006).

1 GAO, High-Risk Series: An Update, GAO-05-207 (Washington, D.C.: Jan.
2005).
			  

           GAOï¿½s Mission
			  
			  The Government Accountability Office, the audit, evaluation and
           investigative arm of Congress, exists to support Congress in
           meeting its constitutional responsibilities and to help improve
           the performance and accountability of the federal government for
           the American people. GAO examines the use of public funds;
           evaluates federal programs and policies; and provides analyses,
           recommendations, and other assistance to help Congress make
           informed oversight, policy, and funding decisions. GAO's
           commitment to good government is reflected in its core values of
           accountability, integrity, and reliability.

           Obtaining Copies of GAO Reports and Testimony
			  
			  The fastest and easiest way to obtain copies of GAO documents at
           no cost is through GAO's Web site ( www.gao.gov ). Each weekday,
           GAO posts newly released reports, testimony, and correspondence on
           its Web site. To have GAO e-mail you a list of newly posted
           products every afternoon, go to www.gao.gov and select "Subscribe
           to Updates."

           Order by Mail or Phone
			  
			  The first copy of each printed report is free. Additional copies
           are $2 each. A check or money order should be made out to the
           Superintendent of Documents. GAO also accepts VISA and Mastercard.
           Orders for 100 or more copies mailed to a single address are
           discounted 25 percent. Orders should be sent to:

           U.S. Government Accountability Office 441 G Street NW, Room LM
           Washington, D.C. 20548

           To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
           (202) 512-6061

           To Report Fraud, Waste, and Abuse in Federal Programs
			  
			  Contact:

           Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
           [email protected] Automated answering system: (800) 424-5454 or
           (202) 512-7470

           Congressional Relations
			  
			  Gloria Jarmon, Managing Director, [email protected] (202) 512-4400
           U.S. Government Accountability Office, 441 G Street NW, Room 7125
           Washington, D.C. 20548

           Public Affairs
			  
			  aul Anderson, Managing Director, [email protected] (202)
           512-4800 U.S. Government Accountability Office, 441 G Street NW,
           Room 7149 Washington, D.C. 20548

(195072)

www.gao.gov/cgi-bin/getrpt? GAO-06-980 .

To view the full product, including the scope 
and methodology, click on the link above.

For more information, contact McCoy Williams at (202) 512-9095 or Keith
Rhodes at (202) 512-6412.

Highlights of GAO-06-980 , a report to congressional addressees

September 2006

DEFENSE TRAVEL SYSTEM

Reported Savings Questionable and Implementation Challenges Remain

In 1995, the Department of Defense (DOD) began an effort to implement a
standard departmentwide travel system. The Defense Travel System (DTS) is
envisioned as DOD's standard end-to-end travel system. This report is a
follow-up to GAO's January 2006, report which highlighted DTS
implementation problems. Because of continued congressional interest in
DTS, GAO initiated this follow-up audit under the Comptroller General's
statutory authority. GAO determined whether (1) two key assumptions made
in the September 2003 economic analysis were reasonable, (2) DOD is taking
action to ensure full utilization of DTS and gathering the data needed to
monitor DTS utilization, and (3) DOD has resolved the previously
identified problems with DTS flight information. To address the above
objectives, GAO (1) reviewed the September 2003 DTS economic analysis, (2)
analyzed DTS utilization data, and (3) analyzed DTS flight information.

What GAO Recommends

GAO is making four recommendations to DOD aimed at improving the
management oversight of DTS including periodic reports on DTS utilization
and resolution of inconsistencies in DTS's requirements. DOD generally
agreed with the recommendations and described its efforts to address them.
DOD also strongly objected to a finding that the reported personnel
savings were unrealistic.

GAO's analysis of the September 2003 DTS economic analysis found that the
two key assumptions used to estimate annual net savings were not based on
reliable information. Two cost components represent the majority of the
over $56 million in estimated net savings-personnel savings and reduced
commercial travel office (CTO) fees. In regard to the personnel savings,
GAO's analysis found that the $24.2 million of personnel savings related
to the Air Force and the Navy was not supported.

           o  Air Force and Navy DTS program officials stated that they did
           not anticipate a reduction in the number of personnel, but rather
           the shifting of staff from the travel function to other functions.
           o  The Naval Cost Analysis Division stated that the Navy will not
           realize any tangible personnel cost savings from the
           implementation of DTS.

In regard to the CTO fees, the economic analysis assumed that 70 percent
of all DTS airline tickets would either require no intervention or minimal
intervention from the CTOs, resulting in an estimated annual net savings
of $31 million. However, the sole support provided by the DTS program
office was an article in a trade industry publication. The article was not
based on information related to DTS, but rather on the experience of one
private sector company. Furthermore, the economic analysis was not
prepared in accordance with guidance prescribed by OMB and DOD.

           o  DOD guidance stated that the life-cycle cost estimates should
           be verified by an independent party, but this did not occur.
           o  The economic analysis did not undertake an assessment of the
           effects of the uncertainty inherent in the estimates of benefits
           and costs. Because an economic analysis uses estimates and
           assumptions, it is critical that the imprecision in both the
           underlying data and assumptions be understood. Such an assessment
           is referred to as a sensitivity analysis.

DOD acknowledged that DTS is not being used to the fullest extent
possible, but lacks comprehensive data to effectively monitor its
utilization. DOD's utilization data are based on a model that was
developed in calendar year 2003. However, the model has not been
completely updated to reflect actual DTS usage. The lack of accurate
utilization data hinders management's ability to monitor progress toward
the DOD vision of DTS as the standard travel system. GAO also found that
the military services have initiated actions that are aimed at increasing
the utilization of DTS.

Finally, GAO found that DTS still has not addressed the underlying
problems associated with weak requirement management and system testing.
While DOD has acted to address concerns GAO previously raised, GAO found
that DTS's requirements are still ambiguous and conflicting. For example,
DTS displaying up to 25 flights for each inquiry is questionable because
it is unclear whether this is a valid requirement. Until DOD improves
DTS's requirement management practices, the department will not have
reasonable assurance that DTS can provide the intended functionality.
*** End of document. ***