Defense Travel System: Estimated Savings Are Questionable and	 
Improvements Are Needed to Ensure Functionality and Increase	 
Utilization (16-NOV-06, GAO-07-208T).				 
                                                                 
In 1995, the Department of Defense (DOD) began an effort to	 
implement a standard departmentwide travel system. The Defense	 
Travel System (DTS) is envisioned as DOD's standard end-to-end	 
travel system. This testimony is based on GAO's September 2006	 
related report. Today's testimony highlights GAO's key findings  
with regard to the following objectives: (1) Were the two key	 
assumptions made in the September 2003 economic analysis	 
reasonable? (2) Was DOD taking action to ensure full utilization 
of DTS and gathering the data needed to monitor DTS utilization? 
and (3) Has DOD resolved several functional problems associated  
with weak system requirements and testing? To address these	 
objectives, GAO (1) reviewed the September 2003 DTS economic	 
analysis, (2) analyzed DTS utilization data, and (3) analyzed DTS
flight information.						 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-07-208T					        
    ACCNO:   A63444						        
  TITLE:     Defense Travel System: Estimated Savings Are Questionable
and Improvements Are Needed to Ensure Functionality and Increase 
Utilization							 
     DATE:   11/16/2006 
  SUBJECT:   Cost analysis					 
	     Cost effectiveness analysis			 
	     Data collection					 
	     Defense cost control				 
	     Defense economic analysis				 
	     Financial management systems			 
	     Internal controls					 
	     Program evaluation 				 
	     Program management 				 
	     Travel						 
	     Travel costs					 
	     Program implementation				 
	     Defense Travel System				 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-07-208T

   

     * [1]Summary
     * [2]Validity of DTS Economic Analysis Questionable

          * [3]Personnel Savings Are Unrealistic
          * [4]Savings Associated with Reduction of CTO Fees Are Unknown

               * [5]"No Touch" Transaction Volume Estimates Are Not Supported
               * [6]Navy Impact of CTO Management Fees Not Adequately
                 Considered

          * [7]Effective Implementation of Existing Policies Should Have Id

     * [8]DTS Remains Underutilized by the Military Services

          * [9]Metrics to Measure DTS Utilization Are Inadequate
          * [10]DOD Has Taken Steps to Improve DTS Utilization, but Further

     * [11]Previously Reported DTS Requirements Management and Testing

          * [12]Providing Complete Flight Information Has Been a Continuing
          * [13]DOD's Planned Corrective Actions Will Not Address Fundamenta

     * [14]Recommendations to Improve DTS Management and Oversight
     * [15]Contacts and Acknowledgments
     * [16]GAO's Mission
     * [17]Obtaining Copies of GAO Reports and Testimony

          * [18]Order by Mail or Phone

     * [19]To Report Fraud, Waste, and Abuse in Federal Programs
     * [20]Congressional Relations
     * [21]Public Affairs

Testimony

Before the Permanent Subcommittee on Investigations, Committee on Homeland
Security and Governmental Affairs, U.S. Senate

United States Government Accountability Office

GAO

For Release on Delivery Expected at 10:00 a.m. EST

Thursday, November 16, 2006

DEFENSE TRAVEL SYSTEM

Estimated Savings Are Questionable and Improvements Are Needed to Ensure
Functionality and Increase Utilization

Statement of McCoy Williams, Director Financial Management and Assurance

Keith A. Rhodes, Chief Technologist Applied Research and Methods Center
for Technology and Engineering

GAO-07-208T

Mr. Chairman and Members of the Subcommittee:

Thank you for the opportunity to discuss our recent report^1 related to
problems encountered by the Department of Defense (DOD) in its efforts to
successfully implement the Defense Travel System (DTS). As you know, DOD
envisions DTS as the department's standard end-to-end travel system.^2 The
department estimates that DTS will be fully deployed at all 11,000
intended locations during fiscal year 2007.^3 The September 2003 economic
analysis noted that DTS, when fully implemented, would result in annual
net savings of over $56 million. The economic analysis noted that savings
would be realized by the department during fiscal years 2009-2016. In
December 2003, the department's Chief Information Officer approved a DTS
funding level of approximately $564 million. Of this amount, the contract
for the design, development, and deployment of DTS was for about $264
million. The remaining costs are associated with areas such as the
operation and maintenance of DTS, operation of the Program Management
Office-Defense Travel System (PMO-DTS), the voucher payment process, and
management and oversight of the numerous contracted commercial travel
offices (CTO).

My testimony today is based on our September 2006 report,^4 which followed
up on our September 2005 testimony and January 2006 report.^5 One of the
major findings in our previous work was that DOD did not have reasonable
assurance that flight information was properly displayed for DOD travelers
because the department failed to properly test the system interfaces
through which the data are accessed for display. We further noted that the
continued use of the existing legacy travel systems at locations where DTS
has been deployed results in underutilization of DTS and reduces the
envisioned savings.

^1GAO, Defense Travel System: Reported Savings Questionable and
Implementation Challenges Remain, [22]GAO-06-980 (Washington, D.C.: Sept.
26, 2006).

^2DOD expects DTS to perform all functions related to travel or ensure
that other systems are provided with adequate information to provide this
functionality. For example, obligating funds associated with travel is a
necessary function, and DTS is expected to (1) make sure that adequate
funds are available before authorizing travel either through information
contained in its system or by obtaining the necessary information from
another system, (2) obligate funds through issuance of approved travel
orders, and (3) provide DOD's financial management systems with the
necessary information so that those systems can record the obligation.
Since DTS is required to ensure that all travel-related functionality is
properly performed, DOD commonly refers to DTS as an "end-to-end travel
system."

^3As of September 2005, the department had estimated that DTS would be
fully deployed during fiscal year 2006.

^4 [23]GAO-06-980 .

^5GAO, DOD Business Transformation: Defense Travel System Continues to
Face Implementation Challenges, [24]GAO-06-18 (Washington, D.C.: Jan. 18,
2006), and DOD Business Transformation: Preliminary Observations on the
Defense Travel System, [25]GAO-05-998T (Washington, D.C.: Sept. 29, 2005).

Today, I will highlight three key findings from our September 2006 report:

           o Two key assumptions related to the estimated cost savings in the
           September 2003 DTS economic analysis were not reasonable. DOD
           strongly objected to this finding, and I will discuss why we
           continue to believe that our finding is accurate.

           o The department did not have quantitative metrics to measure the
           extent to which DTS is actually being used.

           o DOD still has not addressed several functional problems
           associated with weak requirements management and system testing.

           Finally, I will discuss our recommendations to improve the
           department's management and oversight of DTS.

           Our work focused on the validity of the assumptions that were the
           principal drivers of DOD's net annual estimated savings of over
           $56 million. We determined that the data were sufficiently
           reliable for this purpose. We did not review the accuracy and
           reliability of the specific dollar amounts shown in the September
           2003 economic analysis. To address our objectives we also (1)
           reviewed the September 2003 economic analysis and met with
           cognizant officials, (2) analyzed DTS utilization data and
           obtained an overview of the method and data used by DTS program
           officials to report the rate of DTS utilization for the various
           DOD components, and (3) analyzed DTS flight information. Our work
           was performed from October 2005 through July 2006 in accordance
           with U.S. generally accepted government auditing standards.
           Further details on our scope and methodology are included in our
           September 2006 report.^6
			  
			  Summary

           Our analysis of the September 2003 DTS economic analysis found
           that two key assumptions used to estimate cost savings were not
           based on reliable information. Two primary areas represented the
           majority of the over $56 million of estimated annual net savings
           DTS was expected to realize--personnel savings of $24.2 million
           and reduced CTO fees of $31 million. The $24.2 million estimated
           annual personnel savings were attributed to the Air Force and
           Navy.^7 However, Air Force and Navy DTS officials stated that they
           did not anticipate a reduction in the number of personnel with the
           full implementation of DTS, but rather the shifting of staff to
           other functions. Further, the Naval Cost Analysis Division has
           stated that the Navy will not realize any tangible personnel cost
           savings from the implementation of DTS. In written comments on a
           draft of our report, the Under Secretary of Defense (Personnel and
           Readiness), strongly objected to our finding that the estimated
           personnel savings are unrealistic and stated that recognizing
           fiscal constraints, the department continues to identify
           efficiencies and eliminate redundancies to help leverage available
           funds. As noted in our report, DOD officials responsible for
           reviewing economic analyses stated that while shifting personnel
           to other functions is considered a benefit, it should be
           considered an intangible benefit rather than tangible dollar
           savings since the shifting of personnel does not result in a
           reduction of DOD expenditures. Because none of the military
           services could validate an actual reduction in the number of
           personnel as a result of DTS implementation, and DOD's comments
           did not include any additional support or documentation for its
           position, we continue to believe that the estimated annual
           personnel savings of $54.1 million are unrealistic.

           In regard to the estimated annual savings of $31 million
           attributed to lower CTO fees, we requested, but the PMO-DTS could
           not provide, any analysis of travel data to support the assumption
           that 70 percent of all airline tickets would be considered "no
           touch"--meaning that there would be no or minimal intervention by
           the CTO, thereby resulting in lower CTO fees. We found that the 70
           percent assumption was based solely upon an article that appeared
           in a travel industry trade publication. Further, the economic
           analysis assumed that the Navy would save about $7.5 million,
           almost 25 percent, of the total savings related to CTO fees once
           DTS is fully deployed. Again, this figure was based on a reduction
           in the fees the Navy would pay for "no touch" transactions.
           However, the Navy paid a flat management fee that was the same
           regardless of the involvement of the CTO--therefore, the reduced
           "no touch" fee would not apply.

           In addition, the economic analysis was not prepared in accordance
           with guidance prescribed by the Office of Management and Budget
           (OMB) and DOD. Both sets of guidance require that an economic
           analysis be based on facts and data and be explicit about the
           underlying assumptions used to arrive at future benefits and
           costs. DOD guidance also states that life-cycle cost estimates
           should be independently validated. An independent review is
           intended, in part, to provide program management some degree of
           assurance that the life-cycle cost estimates are reasonable and
           the cost estimates are built on realistic program assumptions.
           However, an independent validation was not performed.

           Our analysis also found that the department did not have
           quantitative metrics to measure the extent to which DTS is
           actually being used. The reported DTS utilization rates were based
           on a methodology that was developed using estimated data, and
           PMO-DTS program officials acknowledged that the model had not been
           completely updated with actual data as DTS continued to be
           implemented at the 11,000 sites. As a result, the PMO-DTS
           continues to rely on outdated information in calculating DTS
           utilization rates that are reported to DOD management and the
           Congress. Additionally, while the military services have initiated
           actions to help increase the utilization of DTS, they pointed out
           that ineffective DTS training is a contributing factor to the
           lower than expected usage rate by the military services.

           Finally, DOD still has not addressed several functional problems
           associated with weak requirements management and system testing.
           Requirements represent the blueprint that system developers and
           program managers use to design, develop, test, and implement a
           system. Because requirements provide the foundation for system
           testing, they must be complete, clear, and well documented to
           design and implement an effective testing program. Adequately
           defined and tested requirements are one of the key elements to
           help reduce a project's risks to acceptable levels.^8 Our February
           2006 analysis disclosed that DOD still did not have reasonable
           assurance that the flight information was being properly displayed
           to DOD travelers. We identified 246 unique GSA city pair flights
           that should have been identified on one or more DTS flight
           displays according to the DOD requirements. However, 87 of these
           flights did not appear on one or more of the required listings.
           While the PMO-DTS has taken action to address our concerns, these
           actions do not fully address the fundamental problems we found
           during this audit and on which we have previously reported.^9 For
           example, the DTS requirements we reviewed were still ambiguous and
           conflicting.

           Our September 2006 report includes four recommendations to the
           Secretary of Defense aimed at improving the department's
           management and oversight of DTS. We recommended that the Secretary
           of Defense (1) evaluate the cost effectiveness of the Navy
           continuing with the CTO management fee structure, (2) update the
           DTS Voucher Analysis Model to report DTS actual utilization rates,
           (3) require the PMO-DTS to provide periodic reports on the
           utilization of DTS, and (4) resolve inconsistencies in DTS
           requirements. DOD generally agreed with the recommendations and
           described its efforts to address them.
			  
			  Validity of DTS Economic Analysis Questionable

           In September 2003, DOD finalized its economic analysis for DTS in
           preparation for a milestone decision review.^10 The highlights of
           the economic analysis are shown in table 1. In December 2003, the
           DOD Chief Information Officer granted approval for DTS to proceed
           with full implementation throughout the department.

           Table 1: Summary of DTS Estimated Annual Net Savings Reported in
           the September 2003 Economic Analysis
			  
			  Constant fiscal year 2003 dollars in millions
Cost components                    Estimated annual net savings 
Records management                                        $19.8 
Centrally billed accounts                                   1.7 
CTO acquisition and administration                          2.4 
CTO services                                               31.0 
Voucher process and compute                                54.1 
Voucher pay                                                   0 
Legacy systems                                             14.5 
PMO                                                       (8.8) 
Help desk/DTA                                            (36.8) 
System operations                                        (21.5) 
Total net savings                                         $56.4 

           Source: September 2003 economic analysis provided by the PMO-DTS.

           Note: In arriving at the estimated annual net savings of over $56
           million, the economic analysis took into consideration the
           estimated costs of over $2.1 billion, which covers fiscal years
           2003-2016. The estimated costs included the costs that are
           estimated to be incurred by the PMO-DTS, the Army, the Navy, the
           Air Force, and the defense agencies.

           Our analysis of the September 2003 DTS economic analysis found
           that two key assumptions used to estimate cost savings were not
           based on reliable information. Consequently, the economic analysis
           did not serve to help ensure that the funds invested in DTS were
           used in an efficient and effective manner. Two primary
           areas--personnel savings and reduced CTO fees--represented the
           majority of the over $56 million of estimated annual net savings
           DTS was expected to realize. However, the estimates used to
           generate these savings were unreliable. Further, DOD did not
           effectively implement the policies relating to developing economic
           analyses for programs such as DTS. Effective implementation of
           these policies should have highlighted the problems that we found
           and allowed for appropriate adjustments so that the economic
           analysis could have served as a useful management tool in making
           funding decisions related to DTS--which is the primary purpose of
           this analysis. While the department's system acquisition criteria
           do not require that a new economic analysis be prepared, the
           department's business system investment management structure
           provides an opportunity for DOD management to assess whether DTS
           is meeting its planned cost, schedule, and functionality goals.
			  
			  Personnel Savings Are Unrealistic

           The economic analysis estimated that the annual personnel savings
           was over $54 million,^11 as shown in table 2.

           Table 2: Summary of Estimated Annual Personnel Savings
			  
			  Constant fiscal year 2003 dollars in millions
DOD component               Estimated annual savings 
Army                                           $16.0 
Navy                                            12.9 
Air Force                                       11.3 
Marine Corps                                     5.8 
Defense agencies                                 6.3 
Permanent change of station                      1.8 
Total savings                                  $54.1 


           Source: September 2003 economic analysis provided by the PMO-DTS.

           As shown in table 2, approximately 45 percent of the estimated
           savings, or $24.2 million, was attributable to the Air Force and
           Navy. The assumption behind the personnel savings computation was
           that there would be less manual intervention in the processing of
           travel vouchers for payment, and therefore fewer staff would be
           needed. However, based on our discussions with Air Force and Navy
           DTS program officials, it is questionable as to how the estimated
           savings will be achieved. Air Force and Navy DTS program officials
           stated that they did not anticipate a reduction in the number of
           personnel with the full implementation of DTS, but rather the
           shifting of staff to other functions. According to DOD officials
           responsible for reviewing economic analyses, while shifting
           personnel to other functions is considered a benefit, it should be
           considered an intangible benefit rather than tangible dollar
           savings since the shifting of personnel does not result in a
           reduction of DOD expenditures. Also, as part of the Navy's overall
           evaluation of the economic analysis, program officials stated that
           "the Navy has not identified, and conceivably will not recommend,
           any personnel billets for reduction." Finally, the Naval Cost
           Analysis Division (NCAD) October 2003 report on the economic
           analysis noted that it could not validate approximately 40 percent
           of the Navy's total costs, including personnel costs, in the DTS
           life-cycle cost estimates because credible supporting
           documentation was lacking. The report also noted that the PMO-DTS
           used unsound methodologies in preparing the DTS economic analysis.

           The extent of personnel savings for the Army and defense agencies,
           which are reported as $16 million and $6.3 million respectively,
           is also unclear. The Army and many defense agencies use the
           Defense Finance and Accounting Service (DFAS) to process their
           travel vouchers, so the personnel savings for the Army and the
           defense agencies were primarily related to reductions in DFAS's
           costs. In discussions with DFAS officials, they were unable to
           estimate the actual personnel savings that would result since they
           did not know (1) the number of personnel, like those at the Air
           Force and Navy, that would simply be transferred to other DFAS
           functions or (2) the number of personnel that could be used to
           avoid additional hiring. For example, DFAS expects that some of
           the individuals assigned to support the travel function could be
           moved to support its ePayroll program. Since these positions would
           need to be filled regardless of whether the travel function is
           reduced, transferring personnel from travel to ePayroll would
           reduce DOD's overall costs since DFAS would not have to hire
           additional individuals.

           DOD strongly objected to our finding that the personnel savings
           are unrealistic. In its written comments, the department stated
           that it is facing an enormous challenge and continues to identify
           efficiencies and eliminate redundancies to help leverage available
           funds. We fully recognize that the department is attempting to
           improve the efficiency and effectiveness of its business
           operations. The Comptroller General of the United States testified
           in August 2006 that increased commitment by the department to
           address DOD's numerous challenges represents an improvement over
           past efforts.^12

           The fact remains, however, that the results of an economic
           analysis are intended to help management decide if future
           investments in a given endeavor are worthwhile. In order to
           provide management with this information, it is imperative that
           the underlying assumptions in an economic analysis be supported by
           valid assumptions. The September 2003 economic analysis noted that
           personnel savings of $54.1 million would be realized by the
           department annually for fiscal years 2009 through 2016. However,
           based on our review and analysis of documentation and discussion
           with department personnel, we found that the underlying
           assumptions in support of the $54.1 million were not valid,
           particularly in regard to the amounts estimated for the Navy and
           Air Force. For example, we agree with the statements of DOD
           officials who indicated that the shifting of personnel to other
           functions cannot be counted towards tangible dollar savings, since
           such actions do not result in a reduction of DOD expenditures.
           Moreover, the department did not provide any new data or related
           documentation in its comments that were counter to our finding. As
           a result of these factors, we continue to believe that the
           estimated annual personnel savings of $54.1 million are
           unrealistic.
			  
			  Savings Associated with Reduction of CTO Fees Are Unknown

           According to the September 2003 economic analysis, DOD expected to
           realize annual net savings of $31 million through reduced fees
           paid to the CTOs because the successful implementation of DTS
           would enable the majority of airline tickets to be acquired with
           either no or minimal intervention by the CTOs. These are commonly
           referred to as "no touch" transactions. However, DOD did not have
           a sufficient basis to estimate the number of transactions that
           would be considered "no touch" since the (1) estimated percentage
           of transactions that can be processed using the "no touch" was not
           supported and (2) analysis did not properly consider the effects
           of components that use management fees, rather than transaction
           fees, to compensate the CTOs for services provided. The weaknesses
           we identified with the estimating process raise serious questions
           as to whether DOD will realize substantial portions of the
           estimated annual net savings of $31 million.
			  
			  ï¿½No Touchï¿½ Transaction Volume Estimates Are Not Supported

           DOD arrived at the $31 million of annual savings in CTO fees by
           estimating that 70 percent of all DTS airline tickets would be
           considered "no touch" and then multiplying these tickets by the
           savings per ticket in CTO fees. However, a fundamental flaw in
           this analysis was that the 70 percent assumption had no solid
           basis. We requested, but the PMO-DTS could not provide, any
           analysis of travel data to support the assertion. Rather, the sole
           support provided by the PMO-DTS was an article in a travel
           industry trade publication.^13 The article was not based on
           information related to DTS, but rather on the experience of one
           private sector company.

           The economic analysis assumed that DOD could save about $13.50 per
           "no touch" ticket. Since that analysis, DOD has awarded one
           contract that specifically prices transactions using the same
           model as that envisioned by the economic analysis. This contract
           applies to the Defense Travel Region 6 travel area.^14 During
           calendar year 2005, the difference in fees for "no touch"
           transactions and the transactions supported by the current process
           averaged between $10 and $12, depending on when the fees were
           incurred because the contract rates changed during 2005.^15 In
           analyzing travel voucher data for Region 6 for calendar year 2005,
           we found that the reported "no touch" rate was, at best 47
           percent--far less than the 70 percent envisioned in the economic
           analysis.

           PMO-DTS program officials stated they are uncertain as to why the
           anticipated 70 percent "no touch" was not being achieved.
           According to PMO-DTS program officials, this could be attributed,
           in part, to the DOD travelers being uncomfortable with the system
           and with making reservations without using a CTO. Although this
           may be one reason, other factors may also affect the expected "no
           touch" fee. For example, we were informed that determining the
           airline availability and making the associated reservation can be
           accomplished, in most cases, rather easily. However, obtaining
           information related to hotels and rental cars and making the
           associated reservation can be more problematic because of the
           limitations in the data that DTS is able to obtain from its
           commercial sources. Accordingly, while a traveler may be able to
           make a "no touch" reservation for the airline portion of the trip,
           the individual may need to contact the CTO in order to make hotel
           or rental car reservations. When this occurs, rather than paying a
           "no touch" fee to the CTO, DOD ends up paying a higher fee, which
           eliminates the savings estimated in the economic analysis.

           The economic analysis assumed that (1) DOD would be able to modify
           the existing CTO contracts to achieve a substantial reduction in
           fees paid to a CTO when DTS was fully implemented across the
           department and (2) all services would use the fee structure called
           for in the new CTO contracts. The first part of the assumption is
           supported by results of the CTO contract for DOD Region 6 travel.
           The fees for the DTS "no touch" transactions were at least $10
           less than if a CTO was involved in the transactions. However, to
           date, the department has experienced difficulty in awarding new
           contracts with the lower fee structure. On May 10, 2006, the
           department announced the cancellation of the solicitation for a
           new contract. According to the department, it decided that the
           solicitation needed to be rewritten based on feedback from travel
           industry representatives at a March 28, 2006, conference. The
           department acknowledged that the "DTS office realized its
           solicitation didn't reflect what travel agency services it
           actually needed."^16 The department would not say how the
           solicitation would be refined, citing the sensitivity of the
           procurement process. The department also noted that the new
           solicitation would be released soon, but provided no specific
           date.
			  
			  Navy Impact of CTO Management Fees Not Adequately Considered

           The economic analysis assumed that the Navy would save about $7.5
           million, almost 25 percent, of the total savings related to CTO
           fees once DTS is fully deployed. The economic analysis averaged
           the CTO fees paid by the Army, the Air Force, and the Marine
           Corps--which amounted to about $18.71 per transaction--to compute
           the savings in Navy CTO fees. Using these data, the assumption was
           made in the economic analysis that a fee of $5.25 would be
           assessed for each ticket, resulting in an average savings of
           $13.46 per ticket for the Navy ($18.71 minus $5.25).^17 While this
           approach may be valid for the organizations that pay individual
           CTO fees, it may not be representative for organizations such as
           the Navy that pay a management fee. The management fee charged the
           Navy is the same regardless of the involvement of the
           CTO--therefore, the reduced "no touch" fee would not apply.

           We were informed by Navy DTS program officials that they were
           considering continuing the use of management fees after DTS is
           fully implemented. According to Navy DTS program officials, they
           paid about $14.5 million during fiscal year 2005 for CTO
           management fees, almost $19 per ticket for approximately 762,700
           tickets issued. Accordingly, even if the department arrives at a
           new CTO contract containing the new fee structure or fees similar
           to those of Region 6, the estimated savings related to CTO fees
           for the Navy will not be realized if the Navy continues to use the
           management fee concept.
			  
			  Effective Implementation of Existing Policies Should Have
			  Identified Problems with the Economic Analysis

           Effective implementation of DOD guidance would have detected the
           types of problems discussed above and resulted in an economic
           analysis that would have accomplished the stated objective of the
           process--to help ensure that the funds invested in DTS were used
           efficiently and effectively. DOD policy^18 and OMB guidance^19
           require that an economic analysis be based on facts and data and
           be explicit about the underlying assumptions used to arrive at
           estimates of future benefits and costs. Since an economic analysis
           deals with costs and benefits occurring in the future, assumptions
           must be made to account for uncertainties. DOD policy recognizes
           this and provides a systematic approach to the problem of choosing
           the best method of allocating scarce resources to achieve a given
           objective.

           A sound economic analysis recognizes that there are alternative
           ways to meet a given objective and that each alternative requires
           certain resources and produces certain results. The purpose of the
           economic analysis is to give the decision maker insight into
           economic factors bearing on accomplishing the objectives.
           Therefore, it is important to identify factors, such as cost and
           performance risks and drivers, that can be used to establish and
           defend priorities and resource allocations. The DTS economic
           analysis did not comply with the DOD policy, and the weaknesses we
           found should have been detected had the DOD policy been
           effectively implemented. The PMO-DTS had adequate warning signs of
           the potential problems associated with not following the OMB and
           DOD guidance for developing an effective economic analysis. For
           example, as noted earlier, the Air Force and Navy provided
           comments when the economic analysis was being developed that the
           expected benefits being claimed were unrealistic. Just removing
           the benefits associated with personnel savings from the Air Force
           and Navy would have reduced the overall estimated program cost
           savings by almost 45 percent. This would have put increased
           pressure on the credibility of using a 70 percent "no touch"
           utilization rate.

           Specific examples of failures to effectively implement the DOD
           policy on conducting economic analyses include the (1) DTS
           life-cycle cost estimates portion of the economic analysis was not
           independently validated as specified in DOD's guidance^20 and (2)
           September 2003 DTS economic analysis did not undertake an
           assessment of the effects of the uncertainty inherent in the
           estimates of benefits and costs, as required by DOD and OMB
           guidance.^21 Because an economic analysis uses estimates and
           assumptions, it is critical that a sensitivity analysis^22 be
           performed to understand the effects of the imprecision in both
           underlying data and modeling assumptions.
			  
			  DTS Remains Underutilized by the Military Services

           Our September 2005 testimony and January 2006 report^23 noted the
           challenge facing the department in attaining the anticipated DTS
           utilization. While DOD has acknowledged the underutilization, we
           found that, across DOD, the department does not have reasonable
           quantitative metrics to measure the extent to which DTS is
           actually being used. Presently, the reported DTS utilization is
           based on a DTS Voucher Analysis Model^24 that was developed in
           calendar year 2003 using estimated data, but over the years has
           not been completely updated with actual data. While the military
           services have initiated actions to help increase the utilization
           of DTS, they pointed out that ineffective DTS training is a
           contributing factor to the lower than expected usage rate by the
           military services.
			  
			  Metrics to Measure DTS Utilization Are Inadequate

           The DTS Voucher Analysis Model was prepared in calendar year 2003
           and based on airline ticket and voucher count data that were
           reported by the military services and defense agencies, but the
           data were not verified or validated. Furthermore, PMO-DTS
           officials acknowledged that the model has not been completely
           updated with actual data as DTS continues to be implemented at the
           11,000 sites. We found that the Air Force is the only military
           service that submits monthly metrics to the PMO-DTS officials for
           their use in updating the DTS Voucher Analysis Model. Rather than
           reporting utilization based on individual site system utilization
           data, the PMO-DTS continues to rely on outdated information in the
           reporting of DTS utilization to DOD management and the Congress.
           We have previously reported^25 that best business practices
           indicate that a key factor of project management and oversight is
           the ability to effectively monitor and evaluate a project's actual
           performance against what was planned.

           In order to perform this critical task, best business practices
           require the adoption of quantitative metrics to help measure the
           effectiveness of a business system implementation and to
           continually measure and monitor results, such as system
           utilization. This lack of accurate and pertinent utilization data
           hinders management's ability to monitor its progress toward the
           DOD vision of DTS as the standard travel system, as well as to
           provide consistent and accurate data to Congress. With the shift
           of the DTS program to the Business Transformation Agency (BTA),^26
           which now makes DTS an enterprisewide endeavor, improved metrics
           and training are essential if DTS is to be DOD's standard,
           integrated, end-to-end travel system for business travel.

           DTS's reported utilization rates for the period October 2005
           through April 2006 averaged 53 percent for Army, 30 percent for
           Navy, and 39 percent for Air Force. Because the PMO-DTS was not
           able to identify the total number of travel vouchers that should
           have been processed through DTS (total universe of travel
           vouchers), these utilization rates may be over- or understated.
           PMO-DTS program officials confirmed that the reported utilization
           data were not based on complete data because the department did
           not have comprehensive information to identify the universe or the
           total number of travel vouchers that should be processed through
           DTS. PMO-DTS program and DTS military service officials agreed
           that the actual DTS utilization rate should be calculated by
           comparing actual vouchers being processed in DTS to the total
           universe of vouchers that should be processed in DTS. The universe
           would exclude those travel vouchers that cannot be processed
           through DTS, such as those related to permanent change of station
           travel.

           The Air Force was the only military service that attempted to
           obtain data on (1) the actual travel vouchers processed through
           DTS and (2) those travel vouchers that were eligible to be
           processed through DTS, but were not. These data were
           site-specific. For example, during the month of December 2005, the
           PMO-DTS reported that at Wright-Patterson Air Force Base, 2,880
           travel vouchers were processed by DTS, and the Air Force reported
           that another 2,307 vouchers were processed through the legacy
           system--the Reserve Travel System (RTS). Of those processed
           through RTS, Air Force DTS program officials stated that 338
           travel vouchers should have been processed through DTS. DTS Air
           Force program officials further stated that they submitted to the
           PMO-DTS the number of travel vouchers processed through RTS each
           month. These data are used by the PMO-DTS to update the DTS
           Voucher Analysis Model. However, neither the Air Force nor the
           PMO-DTS have verified the accuracy and reliability of the data.
           Therefore, the accuracy of the utilization rates reported for the
           Air Force by the PMO-DTS is not known.

           Because Army and Navy DTS program officials did not have the
           information to identify the travel transactions that should have
           been processed through DTS, the Army and Navy did not have a basis
           for evaluating DTS utilization at their respective military
           locations and activities. Furthermore, Navy DTS program officials
           indicated that the utilization data that the PMO-DTS program
           officials reported for the Navy were not accurate. According to
           Navy DTS program officials, the Navy's primary source of
           utilization data was the monthly metrics reports provided by the
           PMO-DTS, but Navy DTS program officials questioned the accuracy of
           the Navy utilization reports provided by the PMO-DTS.
			  
			  DOD Has Taken Steps to Improve DTS Utilization, but Further Action
			  Is Needed

           Although the military services have issued various memorandums
           aimed at increasing the utilization of DTS, the military service
           DTS program officials all pointed to ineffective training as a
           primary cause of DTS not being utilized to a far greater extent.
           The following examples highlight the concerns raised by the
           military service officials:

           o Army DTS program officials emphasized that the DTS system is
           complex and the design presents usability challenges for
           users--especially for first-time or infrequent users. They added
           that a major concern is that there is no PMO-DTS training for
           existing DTS users as new functionality is added to DTS. These
           officials stated that the PMO-DTS does not do a good job of
           informing users about functionality changes made to the system. We
           inquired if the Help Desk was able to resolve the users' problems,
           and the Army DTS officials simply stated "no." The Army officials
           further pointed out that it would be beneficial if the PMO-DTS
           improved the electronic training on the DTS Web site and made the
           training documentation easier to understand. Also, improved
           training would help infrequent users adapt to system changes. The
           Army officials noted that without some of these improvements to
           resolve usability concerns, DTS will continue to be extremely
           frustrating and cumbersome for travelers.

           o Navy DTS program officials stated that DTS lacks adequate
           user/traveler training. The train-the-trainer concept of training
           system administrators who could then effectively train all their
           travelers has been largely unsuccessful. According to Navy
           officials, this has resulted in many travelers and users
           attempting to use DTS with no or insufficient training. The effect
           has frustrated users at each step of the travel process and has
           discouraged use of DTS.

           o Air Force officials stated that new DTS system releases are
           implemented with known problems, but the sites are not informed of
           the problems. Workarounds are not provided until after the sites
           begin encountering problems. Air Force DTS program officials
           stated that DTS releases did not appear to be well tested prior to
           implementation. Air Force officials also stated that there was
           insufficient training on new functionality. PMO-DTS and DTS
           contractor program officials believed that conference calls to
           discuss new functionality with the sites were acceptable training,
           but Air Force officials did not agree. The Air Force finance
           office was expected to fully comprehend the information received
           from those conference calls and provide training on the new
           functionality to users/approvers, but these officials stated that
           this was an unrealistic expectation.

           As discussed in our September 2005 testimony and January 2006
           report,^27 the unnecessary continued use of the legacy travel
           systems results in the inefficient use of funds because the
           department is paying to operate and maintain duplicative systems
           that perform the same function--travel.
			  
			  Previously Reported DTS Requirements Management and Testing
			  Deficiencies Have Not Been Resolved

           Our September 2005 testimony and January 2006 report noted
           problems with DTS's ability to properly display flight information
           and traced those problems to inadequate requirements management
           and testing. DOD stated that it had addressed those deficiencies,
           and in February 2006, we again tested the system to determine
           whether the stated weaknesses had been addressed. We found that
           similar problems continue to exist. Once again, these problems can
           be traced to ineffective requirements management and testing
           processes. Properly defined requirements are a key element in
           systems that meet their cost, schedule, and performance goals
           since the requirements define the (1) functionality that is
           expected to be provided by the system and (2) quantitative
           measures by which to determine through testing whether that
           functionality is operating as expected.

           We briefed PMO-DTS officials on the results of our tests and in
           May 2006 the officials agreed that our continued concerns about
           the proper display of flight information were valid. PMO-DTS
           officials stated that the DTS technology refresh, which was to be
           completed in September 2006, should address some of our concerns.
           While these actions are a positive step forward, they do not
           address the fundamental problem that DTS's requirements are still
           ambiguous and conflicting--a primary cause of the previous
           problems. Until a viable requirements management process is
           developed and effectively implemented, the department (1) cannot
           develop an effective testing process and (2) will not have
           reasonable assurance the project risks have been reduced to
           acceptable levels.
			  
			  Providing Complete Flight Information Has Been a Continuing Problem

           In our earlier testimony and report,^28 we noted that DOD did not
           have reasonable assurance that the flights displayed met the
           stated DOD requirements. Although DOD stated in each case that our
           concerns had been addressed, subsequent tests found that the
           problems had not been corrected. Requirements represent the
           blueprint that system developers and program managers use to
           design, develop, and acquire a system. Requirements should be
           consistent with one another, verifiable, and directly traceable^29
           to higher-level business or functional requirements. It is
           critical that requirements be carefully defined and that they flow
           directly from the organization's concept of operations (how the
           organization's day-to-day operations are or will be carried out to
           meet mission needs). Improperly defined or incomplete requirements
           have been commonly identified as a cause of system failure and
           systems that do not meet their cost, schedule, or performance
           goals.

           Requirements represent the foundation on which the system should
           be developed and implemented. As we have noted in previous
           reports,^30 because requirements provide the foundation for system
           testing, significant defects in the requirements management
           process preclude an entity from implementing a disciplined testing
           process. That is, requirements must be complete, clear, and well
           documented to design and implement an effective testing program.
           Absent this, an organization is taking a significant risk that its
           testing efforts will not detect significant defects until after
           the system is placed into production. Our February 2006 analysis
           of selected flight information disclosed that DOD still did not
           have reasonable assurance that DTS displayed flights in accordance
           with its stated requirements. We analyzed 15 U.S. General Services
           Administration (GSA) city pairs,^31 which should have translated
           into 246 GSA city pair flights for the departure times selected.
           However, we identified 87 flights that did not appear on one or
           more of the required listings based on the DTS requirements. For
           instance, our analysis identified 44 flights appearing on other
           DTS listings or airline sites that did not appear on the 9:00 am
           DTS listing even though those flights (1) met the 12-hour flight
           window^32 and (2) were considered GSA city pair flights--two of
           the key DTS requirements the system was expected to meet.

           After briefing PMO officials on the results of our analysis in
           February 2006, the PMO-DTS employed the services of a contractor
           to review DTS to determine the specific cause of the problems and
           recommend solutions. In a March 2006 briefing, the PMO-DTS
           acknowledged the existence of the problems, and identified two
           primary causes. First, part of the problem was attributed to the
           methodology used by DTS to obtain flights from the Global
           Distribution System (GDS). The PMO-DTS stated that DTS was
           programmed to obtain a "limited" amount of data from GDS in order
           to reduce the costs associated with accessing GDS. This helps to
           explain why flight queries we reviewed did not produce the
           expected results. To resolve this particular problem, the PMO-DTS
           proposed increasing the amount of data obtained from GDS. Second,
           the PMO-DTS acknowledged that the system testing performed by the
           contractor responsible for developing and operating DTS was
           inadequate and, therefore, there was no assurance that DTS would
           provide the data in conformance with the stated requirements. This
           weakness was not new, but rather reconfirms the concerns discussed
           in our September 2005 testimony and January 2006 report^33 related
           to the testing of DTS.
			  
			  DODï¿½s Planned Corrective Actions Will Not Address Fundamental
			  Requirements Management Problems

           While DOD's planned actions, including a recent technology
           upgrade, should address several of the specific weaknesses we
           identified related to flight displays, they fall short of
           addressing the fundamental problems that caused those
           weaknesses--inadequate requirements management. DTS's requirements
           continue to be ambiguous. For example, DOD has retained a
           requirement to display 25 flights for each inquiry. However, it
           has not determined (1) whether the rationale for that requirement
           is valid and (2) under what conditions flights that are not part
           of the GSA city pair program should be displayed. For example, we
           found that several DTS flights displayed to the user "overlap"^34
           other flights. Properly validating the requirements would allow
           DOD to obtain reasonable assurance that its requirements properly
           define the functionality needed and the business rules necessary
           to properly implement that functionality. As previously noted,
           requirements that are unambiguous and consistent are fundamental
           to providing reasonable assurance that a system will provide the
           desired functionality. Until DOD improves DTS requirement
           management practices, it will not have this assurance.
			  
			  Recommendations to Improve DTS Management and Oversight

           Our recent report^35 included four recommendations to improve the
           department's management and oversight of DTS. We recommended that
           DOD (1) evaluate the cost effectiveness of the Navy continuing
           with the CTO management fee structure versus adopting the revised
           CTO fee structure, once the new contracts have been awarded, (2)
           develop a process by which the military services develop and use
           quantitative data from DTS and their individual legacy systems to
           clearly identify the total universe of DTS-eligible transactions
           on a monthly basis, (3) require the PMO-DTS to provide periodic
           reports on the utilization of DTS, once accurate data are
           available, and (4) resolve inconsistencies in DTS requirements by
           properly defining the functionality needed and business rules
           necessary to properly implement the needed functionality. DOD
           concurred with three and partially concurred with one of the
           recommendations. In regard to the recommendations with which the
           department concurred, it briefly outlined the actions it planned
           to take in addressing two of the three recommendations. For
           example, the department noted the difficulties in obtaining
           accurate utilization data from the existing legacy systems, but
           stated that the Office of the Under Secretary of Defense
           (Personnel and Readiness) and BTA will evaluate methods for
           reporting actual DTS utilization.

           Additionally, DOD noted that the Defense Travel Management Office
           developed and implemented a requirements change management process
           on May 1, 2006. In commenting on the report, the department stated
           that this process is intended to define requirements and track the
           entire life cycle of the requirements development process. While
           we fully support the department's efforts to improve its
           management oversight of DTS's requirements, we continue to believe
           that the department needs to have in place a process that provides
           DOD reasonable assurance that (1) requirements are properly
           documented and (2) requirements are adequately tested as
           recommended in our January 2006 report.^36 This process should
           apply to all existing requirements as well as any new
           requirements. As discussed in this report, we reviewed in May 2006
           some of the requirements that were to have followed the new
           requirements management process and found problems similar to
           those noted in our January 2006 report. Although we did not
           specifically review the new process, if it does not include an
           evaluation of existing requirements, the department may continue
           to experience problems similar to those we previously identified.

           DOD partially concurred with our recommendation to evaluate the
           cost effectiveness of the Navy continuing with the CTO management
           fee structure. However, DOD's response indicated that the Defense
           Travel Management Office is currently procuring commercial travel
           services for DOD worldwide in a manner that will ensure evaluation
           of cost effectiveness for all services. If DOD proceeds with the
           actions outlined in its comments, it will meet the intent of our
           recommendation.

           Effective implementation of these recommendations as well as those
           included in our January 2006 report^37 will go a long way towards
           improving DTS functionality and increasing utilization.
           Furthermore, the shift of DTS to the BTA, which makes DTS an
           enterprisewide endeavor, should help in making DTS the standard
           integrated, end-to-end travel system for business travel.
           Management oversight is essential for this to become a reality. As
           I stated previously, in written comments on a draft of our report,
           the Under Secretary of Defense (Personnel and Readiness), strongly
           objected to our finding that the estimated personnel savings
           included in the economic analysis are unrealistic. Because none of
           the military services could validate an actual reduction in the
           number of personnel as a result of DTS implementation, and DOD's
           comments did not include any additional support or documentation
           for its position, we continue to believe that the estimated annual
           personnel savings of $54.1 million are unrealistic. Although the
           department's criteria do not require that a new economic analysis
           be prepared, the fiscal year 2005 defense authorization act^38
           requires the periodic review, but not less than annually, of every
           defense business system investment. If effectively implemented,
           this annual review process provides an excellent opportunity for
           DOD management to assess whether DTS is meeting its planned cost,
           schedule, and functionality goals. Going forward, such a review
           could serve as a useful management tool in making funding and
           other management decisions related to DTS.

           In conclusion, overhauling the department's antiquated travel
           management practices and systems has been a daunting challenge for
           DOD. While it was widely recognized that this was a task that
           needed to be accomplished and savings could result, the underlying
           assumptions in support of those savings are not based on reliable
           data and therefore it is questionable whether the anticipated
           savings will materialize. Even though the overall savings are
           questionable, the successful implementation of DTS is critical to
           reducing the number of stovepiped, duplicative travel systems
           throughout the department. We have reported on numerous occasions
           that reducing the number of business systems within DOD can
           translate into savings that can be used for other mission needs.
           As noted above, management oversight will be an important factor
           in DTS achieving its intended goals. Equally important, however,
           will be the department's ability to resolve the long-standing
           difficulties that DTS has encountered with its requirements
           management and system testing. Until these issues are resolved,
           more complete utilization of DTS will be problematic.

           Mr. Chairman, this concludes my prepared statement. We would be
           happy to answer any questions that you or other members of the
           Subcommittee may have at this time.
			  
			  Contacts and Acknowledgments

           For further information about this testimony, please contact McCoy
           Williams at (202) 512-9095 or [email protected] , or Keith A.
           Rhodes at (202) 512-6412 or [email protected] .

           Contact points for our Offices of Congressional Relations and
           Public Affairs may be found on the last page of this report. In
           addition to the above contacts, the following individuals made key
           contributions to this testimony: Darby Smith, Assistant Director;
           J. Christopher Martin, Senior-Level Technologist; F. Abe Dymond,
           Assistant General Counsel; Beatrice Alff; Harold Brumm, Jr.;
           Francine DelVecchio; and Tarunkant Mithani.
			  
			  GAOï¿½s Mission

           The Government Accountability Office, the audit, evaluation and
           investigative arm of Congress, exists to support Congress in
           meeting its constitutional responsibilities and to help improve
           the performance and accountability of the federal government for
           the American people. GAO examines the use of public funds;
           evaluates federal programs and policies; and provides analyses,
           recommendations, and other assistance to help Congress make
           informed oversight, policy, and funding decisions. GAO's
           commitment to good government is reflected in its core values of
           accountability, integrity, and reliability.
			  
			  Obtaining Copies of GAO Reports and Testimony

           The fastest and easiest way to obtain copies of GAO documents at
           no cost is through GAO's Web site ( www.gao.gov ). Each
           weekday, GAO posts newly released reports, testimony, and
           correspondence on its Web site. To have GAO e-mail you a list of
           newly posted products every afternoon, go to www.gao.gov and
           select "Subscribe to Updates."
			  
			  Order by Mail or Phone

           The first copy of each printed report is free. Additional copies
           are $2 each. A check or money order should be made out to the
           Superintendent of Documents. GAO also accepts VISA and Mastercard.
           Orders for 100 or more copies mailed to a single address are
           discounted 25 percent. Orders should be sent to:

           U.S. Government Accountability Office 441 G Street NW, Room LM
           Washington, D.C. 20548

           To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
           (202) 512-6061
			  
			  To Report Fraud, Waste, and Abuse in Federal Programs

           Contact:

           Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
           [email protected] Automated answering system: (800) 424-5454 or
           (202) 512-7470
			  
			  Congressional Relations

           Gloria Jarmon, Managing Director, [email protected] (202)
           512-4400 U.S. Government Accountability Office, 441 G Street NW,
           Room 7125 Washington, D.C. 20548
			  
			  Public Affairs

           Paul Anderson, Managing Director, [email protected] (202)
           512-4800 U.S. Government Accountability Office, 441 G Street NW,
           Room 7149 Washington, D.C. 20548

^6 [34]GAO-06-980 .

^7The economic analysis identified annual savings of $11.3 million and
$12.9 million for the Air Force and Navy, respectively.

^8Acceptable levels refer to the fact that any systems acquisition effort
will have risks and will suffer the adverse consequences associated with
defects in the processes. However, effective implementation of disciplined
processes, which includes project planning and management, requirements
management, risk management, quality assurance, and testing, reduces the
possibility of the potential risks actually occurring and prevents
significant defects from materially affecting the cost, timeliness, and
performance of the project.

^9 [35]GAO-05-998T and [36]GAO-06-18 .

^10The September 2003 economic analysis is an addendum to the July 2003
DTS economic analysis.

^11During fiscal years 2009 through 2016.

^12GAO, Department of Defense: Sustained Leadership Is Critical to
Effective Financial and Business Management Transformation,
[37]GAO-06-1006T (Washington, D.C.: Aug. 3, 2006).

^13American Express News Releases: American Express' Interactive Travel
Update, (New York, N.Y.: Aug. 11, 2003),
[38]http://corp.americanexpress.com/gcs/cards/us/ni/pr/081303.aspx .

^14Defense Travel Region 6 includes the Air Force and defense agencies in
the states of Kentucky, Illinois, Indiana, Iowa, Michigan, Minnesota,
Missouri, Nebraska, North Dakota, South Dakota, and Wisconsin. The
contract also applies to Army activities in 8 of the 11 states (excluding
Kentucky, Missouri, and Nebraska). As discussed later, the Navy uses a
management fee contract, and is therefore not included in the Defense
Travel Region 6 contract.

^15According to DTS officials, these savings are consistent with the DTS
contracts that have been awarded to small businesses. The average savings
per "no touch" ticket under these contracts is about $12.88. Because the
contractors are paid these fees directly by the traveler, they are unable
to determine the percentage of transactions that are actually paid using
the "no touch" rate.

^16"DOD Retracts Solicitation for Travel Agency Services,"
FederalTimes.com (May 16, 2006), [39]http://www.federaltimes.com/index.php
? (downloaded June 14, 2006).

^17These savings translate to about 572,000 tickets annually.

^18DOD Instruction 7041.3, Economic Analysis for Decisionmaking, November
7, 1995.

^19Office of Management and Budget, Circular No. A-94, Guidelines and
Discount Rates for Benefit-Cost Analysis of Federal Programs (Revised Jan.
18, 2006).

^20Department of Defense Instruction 5000.2, Operation of the Defense
Acquisition System, May 12, 2003.

^21Department of Defense Instruction 7041.3, Economic Analysis of
Decisionmaking, (Nov. 7, 1995), and Office of Management and Budget
Revised Circular No. A-94, Guidelines and Discount Rates for Benefit-Cost
Analysis of Federal Programs (Oct. 29, 1992).

^22Sensitivity analysis refers to changing the value of a given variable
in a model to gauge the effect of change on model results.

^23 [40]GAO-05-998T and [41]GAO-06-18 .

^24DOD developed a model in calendar year 2003 that compares the expected
usage against the actual usage. The expected usage is obtained by using
historical data, such as ticket counts, to determine the expected number
of vouchers processed by a given location. For example, if a location had
1,000 vouchers as its expected number of vouchers per the model, but now
processes 750 actual vouchers through DTS, then the PMO model considers
that that location has achieved a 75 percent utilization rate. It then
takes the individual computations for each DTS location and "rolls them
up" to determine the total utilization for individual service performance
on a monthly basis.

^25GAO, Financial Management Systems: Additional Efforts Needed to Address
Key Causes of Modernization Failures, [42]GAO-06-184 (Washington, D.C.:
Mar. 15, 2006), and Financial Management Systems: Lack of Disciplined
Processes Puts Implementation of HHS' Financial System at Risk,
[43]GAO-04-1008 (Washington, D.C.: Sept. 23, 2004).

^26In October 2005, DOD established BTA to advance DOD-wide business
transformation efforts, particularly with regard to business systems
modernization. BTA operates under the authority, direction, and control of
the Under Secretary of Defense for Acquisition, Technology, and Logistics,
who is the vice chair of the Defense Business Systems Management
Committee--which serves as the highest ranking governing body for business
systems modernization activities.

^27 [44]GAO-05-998T and [45]GAO-06-18 .

^28 [46]GAO-05-998T and [47]GAO-06-18 .

^29Traceability allows the user to follow the life of the requirement both
forward and backward through these documents and from origin through
implementation. Traceability is also critical to understanding the
parentage, interconnections, and dependencies among the individual
requirements. This information in turn is critical to understanding the
impact when a requirement is changed or deleted.

^30See, for example, [48]GAO-04-1008 and Army Depot Maintenance:
Ineffective Oversight of Depot Maintenance Operations and System
Implementation Efforts, [49]GAO-05-441 (Washington, D.C.: June 30, 2005).

^31GSA awards contracts to airlines to provide flight services between
pairs of cities. This is commonly referred to as the GSA city pair
program. Under this program (1) no advanced ticket purchases are required,
(2) no minimum or maximum length of stay is required, (3) tickets are
fully refundable and no charges are assessed for cancellations or changes,
(4) seating is not capacity controlled (i.e., as long as there is a
coach-class seat on the plane, the traveler may purchase it), (5) no
blackout dates apply, (6) fare savings average 70 percent over regular
walk-up fares, and (7) fares are priced on one-way routes permitting
agencies to plan for multiple destinations. We selected the first 15 city
pairs that were provided by DOD to GSA in support of a GSA study on
accuracy of flight displays and fare information by DTS and the GSA
eTravel providers.

^32A flight window is the amount of time before and after a specified time
and is used for determining the flights that should be displayed. DTS uses
a 12-hour flight window for domestic flights and a 24-hour flight window
for foreign flights. The system is also expected to display up to 25
flights for the flight window.

^33 [50]GAO-05-998T and [51]GAO-06-18 .

^34For example, DTS displayed a GSA city pair flight between Washington,
D.C., and Atlanta, Georgia, that departed at 10:05 a.m. and arrived at
1:50 p.m. This flight "overlapped" two other GSA city pair direct flights
that were available and required less travel time. One flight left at
10:05 a.m. and arrived at 12:02 p.m. while another left at 11:05 a.m. and
arrived at 12:56 p.m. Furthermore, DTS displayed a non-GSA city pair
flight that left at 9:20 a.m. and arrived at 1:05 p.m. This flight did not
meet any of the acceptable criteria for not using a GSA city pair flight.

^35 [52]GAO-06-980 .

^36 [53]GAO-06-18 .

^37 [54]GAO-06-18 .

^38Ronald W. Reagan National Defense Authorization Act for Fiscal Year
2005, Pub. L. No. 108-375, S 332, 118 Stat. 1811, 1851-56 (Oct. 28, 2004)
(codified, in part, at 10 U.S.C. SS 186, 2222).

(195104)

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

www.gao.gov/cgi-bin/getrpt?GAO-07-208T .

To view the full product, including the scope
and methodology, click on the link above.

For more information, contact McCoy Williams at (202) 512-9095 or Keith
Rhodes at (202) 512-6412.

Highlights of [56]GAO-07-208T , a testimony to the Permanent Subcommittee
on Investigations, Committee on Homeland Security and Governmental
Affairs,

U.S. Senate

November 16, 2006

DEFENSE TRAVEL SYSTEM

Estimated Savings Are Questionable and Improvements Are Needed to Ensure
Functionality and Increase Utilization

In 1995, the Department of Defense (DOD) began an effort to implement a
standard departmentwide travel system. The Defense Travel System (DTS) is
envisioned as DOD's standard end-to-end travel system. This testimony is
based on GAO's September 2006 related report. Today's testimony highlights
GAO's key findings with regard to the following objectives: (1) Were the
two key assumptions made in the September 2003 economic analysis
reasonable? (2) Was DOD taking action to ensure full utilization of DTS
and gathering the data needed to monitor DTS utilization? and

(3) Has DOD resolved several functional problems associated with weak
system requirements and testing? To address these objectives, GAO (1)
reviewed the September 2003 DTS economic analysis, (2) analyzed DTS
utilization data, and (3) analyzed DTS flight information.

[57]What GAO Recommends

GAO made four recommendations aimed at improving the management of DTS,
including periodic reports on DTS utilization and resolution of
inconsistencies in DTS's requirements. DOD generally agreed with the
recommendations and described efforts to address them. DOD also strongly
objected to the finding that the reported personnel savings were
unrealistic. However, DOD provided no new data that were counter to GAO's
findings.

GAO's analysis of the September 2003 DTS economic analysis found that the
two key assumptions used to estimate annual net savings were not based on
reliable information. Two cost components represent the majority of the
over $56 million in estimated net savings------personnel savings and
reduced commercial travel office (CTO) fees. In regard to the personnel
savings, GAO's analysis found that the $24.2 million of personnel savings
related to the Air Force and the Navy were not supported.

           o Air Force and Navy DTS program officials stated that they did
           not anticipate a reduction in the number of personnel, but rather
           the shifting of staff from the travel function to other functions.
           o The Naval Cost Analysis Division stated that the Navy will not
           realize any tangible personnel cost savings from the
           implementation of DTS.

In regard to the CTO fees, the economic analysis assumed that 70 percent
of all DTS airline tickets would either require no intervention or minimal
intervention from the CTOs, resulting in an estimated annual net savings
of $31 million. However, the sole support provided by the DTS program
office was an article in a trade industry publication. The article was not
based on information related to DTS, but rather on the experience of one
private sector company. Furthermore, the economic analysis was not
prepared in accordance with guidance prescribed by OMB and DOD.

           o DOD guidance stated that the life-cycle cost estimates should be
           verified by an independent party, but this did not occur.
           o The economic analysis did not undertake an assessment of the
           effects of the uncertainty inherent in the estimates of benefits
           and costs. Because an economic analysis uses estimates and
           assumptions, it is critical that the imprecision in both the
           underlying data and assumptions be understood. Such an assessment
           is referred to as a sensitivity analysis.

DOD acknowledged that DTS is not being used to the fullest extent
possible, but lacks comprehensive data to effectively monitor its
utilization. DOD's utilization data are based on a model that was
developed in calendar year 2003. However, the model has not been
completely updated to reflect actual DTS usage. The lack of accurate
utilization data hinders management's ability to monitor progress toward
the DOD vision of DTS as the standard travel system. GAO also found that
the military services have initiated actions that are aimed at increasing
the utilization of DTS.

Finally, GAO found that DTS still has not addressed the underlying
problems associated with weak requirements management and system testing.
While DOD has acted to address concerns GAO previously raised, GAO found
that DTS's requirements are still ambiguous and conflicting. For example,
DTS displaying up to 25 flights for each inquiry is questionable because
it is unclear whether this is a valid requirement. Until DOD improves
DTS's requirements management practices, the department will not have
reasonable assurance that DTS can provide the intended functionality.

References

Visible links
  22. http://www.gao.gov/cgi-bin/getrpt?GAO-06-980
  23. http://www.gao.gov/cgi-bin/getrpt?GAO-06-980
  24. http://www.gao.gov/cgi-bin/getrpt?GAO-06-18
  25. http://www.gao.gov/cgi-bin/getrpt?GAO-05-998T
  34. http://www.gao.gov/cgi-bin/getrpt?GAO-06-980
  35. http://www.gao.gov/cgi-bin/getrpt?GAO-05-998T
  36. http://www.gao.gov/cgi-bin/getrpt?GAO-06-18
  37. http://www.gao.gov/cgi-bin/getrpt?GAO-06-1006T
  38. http://corp.americanexpress.com/gcs/cards/us/ni/pr/081303.aspx
  39. http://www.federaltimes.com/index.php
  40. http://www.gao.gov/cgi-bin/getrpt?GAO-05-998T
  41. http://www.gao.gov/cgi-bin/getrpt?GAO-06-18
  42. http://www.gao.gov/cgi-bin/getrpt?GAO-06-184
  43. http://www.gao.gov/cgi-bin/getrpt?GAO-04-1008
  44. http://www.gao.gov/cgi-bin/getrpt?GAO-05-998T
  45. http://www.gao.gov/cgi-bin/getrpt?GAO-06-18
  46. http://www.gao.gov/cgi-bin/getrpt?GAO-05-998T
  47. http://www.gao.gov/cgi-bin/getrpt?GAO-06-18
  48. http://www.gao.gov/cgi-bin/getrpt?GAO-04-1008
  49. http://www.gao.gov/cgi-bin/getrpt?GAO-05-441
  50. http://www.gao.gov/cgi-bin/getrpt?GAO-05-998T
  51. http://www.gao.gov/cgi-bin/getrpt?GAO-06-18
  52. http://www.gao.gov/cgi-bin/getrpt?GAO-06-980
  53. http://www.gao.gov/cgi-bin/getrpt?GAO-06-18
  54. http://www.gao.gov/cgi-bin/getrpt?GAO-06-18
  56. http://www.gao.gov/cgi-bin/getrpt?GAO-07-208T
*** End of document. ***