Defense Transportation: Monitoring Costs and Benefits Needed
While Implementing a New Program for Moving Household Goods
(18-APR-03, GAO-03-367).
The Department of Defense (DOD) spends more than $1.7 billion
each year to move and store over 600,000 household goods
shipments when relocating military personnel. It conducted and
evaluated several pilot program studies aimed at fixing its
problem-plagued program and, in 2002, issued a report to Congress
with three recommendations. The 1997 Defense Appropriations Act
Conference Report directed GAO to validate the results achieved
by the pilot programs. In response, GAO examined the extent to
which DOD's recommendations to Congress (1) offer solutions to
long-standing problems in the current program and (2) are
supported by the evaluation's findings and should be implemented.
GAO also assessed the soundness of methodologies used by DOD to
develop cost estimates to implement the recommendations.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-03-367
ACCNO: A06688
TITLE: Defense Transportation: Monitoring Costs and Benefits
Needed While Implementing a New Program for Moving Household
Goods
DATE: 04/18/2003
SUBJECT: Household goods
Internal controls
Military personnel
Personal property
Relocation allowances
Strategic planning
DOD Full Service Moving Project
Military Traffic Management Command
Reengineered Personal Property Program
Navy Service Member Arranged Move Pilot
Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-03-367
Report to Congressional Committees
United States General Accounting Office
GAO
April 2003 DEFENSE TRANSPORTATION
Monitoring Costs and Benefits Needed While Implementing a New Program for
Moving Household Goods
GAO- 03- 367
The recommendations in DOD*s report to Congress have the potential to
resolve several long- standing problems found in the current personal
property program, which manages the transportation and storage of
household goods. The recommendations, if implemented, would
reengineer the claims process to reduce the length of time it currently
takes to resolve claims for lost, destroyed, or damaged household goods
and increase the reimbursement rates that military personnel currently
receive for their losses;
use performance- based service contracts to improve the generally low
quality of service that DOD currently gets from the moving industry; and
put in place new information technology with interface capabilities to
enable program managers and users to monitor in- transit shipments and
track the number and cost of shipments processed each year.
The recommendations in DOD*s report to Congress are supported by the
Transportation Command*s evaluation of the pilot programs* findings and
should be implemented within budget constraints. DOD*s approach in
conducting the evaluation was methodologically sound: It developed an
evaluation plan to guide its work and adjusted the plan when necessary to
address differences in the pilot programs* approaches. While the shipments
included in the evaluation do not represent all shipment types managed
annually by DOD, GAO believes that the evaluation results provide
sufficient information to allow DOD to initiate actions
to improve its current personal property program. GAO found that the
soundness of methodologies used to develop DOD*s cost estimates varied.
Therefore, DOD*s ability to implement changes to the existing program
within the cost estimates DOD reported to Congress is uncertain. GAO
found that the estimate to implement the information technology
recommendation was $7 million rather than the $4 million to $6 million
estimate DOD reported to Congress. In developing cost estimates for the
remaining recommendations, DOD did not provide the same level of
evidentiary support for one of the three adjustments it used to align the
pilot programs* costs to current program costs. As a result, GAO questions
the extent to which these recommendations can be implemented within DOD*s
estimated 13 percent increase over current program costs. While DOD
believes it used a conservative approach in developing this 13 percent
estimate, it has not quantified the risk associated with the projection,
which could provide the military services and Congress information needed
to develop and review future budget requests for this program. Without
providing the range of possible cost increases and the risk
regarding the likelihood of achieving this 13 percent projection within
that range, DOD may find a repeat of what occurred during the pilots,
where the military services terminated participation in one of the pilot
programs due to costs exceeding projections. GAO also found that without
carefully monitoring costs during the
implementation phase and assessing costs and benefits from a period
succeeding full implementation of the recommendations, DOD would not have
the information needed to determine if anticipated improvements in the
program are being achieved at a reasonable cost. DEFENSE TRANSPORTATION
Monitoring Costs and Benefits Needed While Implementing a New Program for
Moving Household Goods
www. gao. gov/ cgi- bin/ getrpt? GAO- 03- 367. To view the full report,
including the scope and methodology, click on the link above. For more
information, contact William M. Solis at (202) 512- 8365 or solisw@ gao.
gov. Highlights of GAO- 03- 367, a report to
Congressional Committees.
April 2003
The Department of Defense (DOD) spends more than $1. 7 billion each year
to move and store over 600,000 household goods shipments when relocating
military personnel. It conducted and evaluated several pilot program
studies aimed at fixing its problemplagued program and, in 2002, issued a
report to Congress with three recommendations. The 1997 Defense
Appropriations Act
Conference Report directed GAO to validate the results achieved by the
pilot programs. In response,
GAO examined the extent to which DOD*s recommendations to Congress (1)
offer solutions to long- standing problems in the
current program and (2) are supported by the evaluation*s findings and
should be implemented. GAO also assessed the soundness of methodologies
used by DOD to develop cost
estimates to implement the recommendations. GAO recommends that DOD
implement the recommendations within budget constraints, quantify the risk
associated with achieving its cost estimates, monitor costs
during the implementation phase, and assess the new program to determine
if anticipated improvements were achieved at a reasonable cost. DOD agreed
with three recommendations, but did not agree with the need to quantify
the risk associated with achieving its cost estimates.
Page i GAO- 03- 367 Defense Transportation Letter 1 Results in Brief 2
Background 5 DOD*s Recommendations Offer Solutions to Long- Standing
Problems 8 Transportation Command*s Evaluation of Pilot Programs Supports
DOD*s Three Recommendations 13 Ability to Implement New Program within
Cost Estimates Reported to Congress Is Uncertain 16 Conclusions 21
Recommendations for Executive Action 22 Agency Comments and Our Evaluation
23 Appendix I Scope and Methodology 28
Appendix II Overview of Current Personal Property Program and Pilot
Programs 32
Current Program 34 Military Traffic Management Command*s Reengineered
Personal Property Program 36 The Department of Defense*s Full Service
Moving Project 39 Navy*s Service Member Arranged Move Pilot Program 42
Appendix III Comments from the Department of Defense 45
Related GAO Products 49
Tables
Table 1: Claims- Related Features of the Current Personal Property Program
and Pilot Programs 32 Table 2: Quality of Service- Related Features of the
Current
Personal Property Program and Pilot Programs 33 Table 3: Data Reliability-
Related Features of the Current Personal Property Program and Pilot
Programs 34 Contents
Page ii GAO- 03- 367 Defense Transportation This is a work of the U. S.
Government and is not subject to copyright protection in the United
States. It may be reproduced and distributed in its entirety without
further
permission from GAO. It may contain copyrighted graphics, images or other
materials. Permission from the copyright holder may be necessary should
you wish to reproduce copyrighted materials separately from GAO*s product.
Page 1 GAO- 03- 367 Defense Transportation April 18, 2003 Congressional
Committees Military personnel and their families can expect to relocate
many times
during a service member*s career. As the moving industry*s single largest
customer, the Department of Defense (DOD) spends more than $1.7 billion
annually for its personal property program, which provides household goods
transportation and storage services for military personnel and their
families when they relocate. The program manages more than 600,000
personal property shipments each year. DOD has experienced longstanding
problems with its current personal property program, including excessive
loss or damage to property, high claims costs incurred by the government,
and poor quality of service from moving companies. Moreover, the program*s
data management system does not provide reliable information on the status
of individual shipments or on the types of shipments and their costs.
In an effort to test alternative approaches and improve the quality of its
personal property program, DOD has carried out several initiatives over
the past 8 years. In 2000, the U. S. Transportation Command 1 began to
collect data from one of the three pilot programs to evaluate alternative
approaches for improving the current program. 2 The Transportation Command
compared the features of the current program with those of the three pilot
programs, and in June 2002 it submitted a report to the Deputy Under
Secretary of Defense (Logistics) outlining its evaluation strategy,
findings, and recommendations. In its November 12, 2002, report to
Congress, DOD included the three recommendations resulting from the
evaluation and the estimated additional costs required to implement
program improvements based on a subsequent assessment of pilot and current
programs* operations.
1 The Deputy Under Secretary of Defense (Logistics) tasked the U. S.
Transportation Command to evaluate alternatives and develop
recommendations to enhance the department*s current personal property
program.
2 The three pilots included in the evaluation are the Military Traffic
Management Command*s Reengineered Personal Property Program, the
Department of Defense*s Full Service Moving Project, and the Navy*s
Service Member Arranged Move Pilot Program. United States General
Accounting Office Washington, DC 20548
Page 2 GAO- 03- 367 Defense Transportation Our involvement in this issue
stems from the conference report on the 1997 Defense Appropriations Act,
which directed us to review the data collected from the pilot programs and
validate the results before DOD
expands any of the pilot programs, and a subsequent request from the
Subcommittee on Readiness of the House Committee on Armed Services. This
report supplements our previous testimony 3 on DOD*s efforts to enhance
its current program, as well as our status report 4 and status briefings
to Congress on DOD*s efforts to implement a methodologically sound
evaluation of the pilot programs. As agreed with your offices, for this
report we assessed the extent to which the recommendations in DOD*s
November 2002 report (1) offer solutions to long- standing problems in the
personal property program and (2) are supported by the evaluation*s
findings and should be implemented. In addition, we assessed the soundness
of the methodology used by DOD to develop the cost estimates for
implementing the recommendations.
To conduct our assessment of DOD*s three recommendations to Congress, we
reviewed DOD and GAO prior reports on the personal property program,
interviewed DOD officials and private- sector contractors involved in the
current and pilot personal property programs and the Transportation
Command*s evaluation, examined the methodology and findings of the
Transportation Command*s evaluation of the three pilot programs, reviewed
the methodology and data generated by each of the three pilot programs,
and assessed the methodologies used to develop the cost estimates for
implementing DOD*s recommendations. We did not make an assessment of
whether the anticipated benefits to be derived from implementing the three
recommendations would warrant the
additional costs DOD projects will be required to fund these improvements.
The scope and methodology we used in our review are described in further
detail in appendix I.
The three recommendations in DOD*s report to Congress offer solutions to
several long- standing problems in the current personal property program.
Specifically, the recommendations address previously identified problems
with the liability/ claims process (including the lengthy claims process,
3 U. S. General Accounting Office, Defense Transportation: Efforts to
Improve DOD*s Personal Property Program, GAO/ T- NSIAD- 99- 106
(Washington, D. C.: Mar. 18, 1999). 4 U. S. General Accounting Office,
Defense Transportation: Final Evaluation Plan Is Needed to Assess
Alternatives to the Current Personal Property Program,
GAO/ NSIAD- 00- 217R (Washington, D. C.: Sept. 27, 2000). Results in Brief
Page 3 GAO- 03- 367 Defense Transportation reimbursement rates that may
not fully compensate service members for loss and damage incurred during
their moves, and high claims costs to the
government), the low quality of shipping services, and the inability to
track shipments and their costs. If implemented, DOD*s first
recommendation* to reengineer the claims process* has the potential to
reduce the length of time it currently takes to resolve claims for lost,
destroyed, or damaged household property and increase the reimbursement
rates that military
personnel currently receive for their losses. The recommendation also has
the potential to reduce the high cost of claims incurred by DOD. The
second recommendation* to use performance- based service contracts* has
the potential to improve the generally low quality of service that DOD
currently receives from moving companies. The third recommendation* to put
in place new information technology with interface capabilities* has the
potential to enable the personal property program managers and other users
of the program to reliably monitor in- transit shipments and track the
number and the cost of shipments processed each year so that accurate
reporting can be provided to DOD and Congress.
The recommendations in DOD*s report to Congress are supported by the
Transportation Command*s evaluation of the pilot programs* findings and
should be implemented within budget constraints. We found that the
Transportation Command*s approach in conducting the evaluation was
methodologically sound: It developed an evaluation plan to guide its work
and adjusted the plan when necessary to address differences in the pilot
programs* approaches. While the shipments included in the evaluation do
not represent all shipment types managed annually by DOD, we believe
that the evaluation results provide sufficient information to allow DOD to
initiate actions to improve its current personal property program.
Our review showed that the soundness of the methodologies used by DOD to
develop cost estimates for implementing the three recommendations varied.
Therefore, DOD*s ability to implement changes to the existing program
within the cost estimates reported to Congress is uncertain. Our review
found that the total initial cost for implementing the information
technology improvements recommendation would more likely be $7 million
rather than DOD*s $4 million to $6 million estimate. We agreed
that the premise of two of three adjustments DOD used to develop the 13
percent cost increase to implement the claims process and performance-
based service contract recommendations was sound. However, we are less
assured about the extent to which the projected cost savings related to a
third adjustment may occur because the adjustment
was not supported by historical experience or by the same quality of data
provided for the other adjustments. While DOD believes it can incorporate
the three recommendations into a new program within its proposed
Page 4 GAO- 03- 367 Defense Transportation 13 percent increase due to the
conservative approach it took in developing this estimate, we believe that
by quantifying the risk associated with this
projection, DOD could provide the military services and Congress
information needed to develop and review future budget requests for this
program. The need for this type of information is further supported based
on the long- standing problems associated with the current program and the
large increase in costs contributing to the military services* decision to
terminate participation in one of the pilot programs. Further, without
carefully monitoring costs during the implementation process and assessing
costs and benefits from a period succeeding full implementation of the
recommendations, DOD will not have the information needed to determine if
anticipated improvements in the program are being achieved at a reasonable
cost. Currently, DOD is beginning planning efforts to
implement the recommendations. These efforts do not include monitoring and
evaluating costs and benefits during the implementation phase and post
implementation of the recommendations in a new program.
We are recommending that DOD initiate actions that will implement the
recommendations contained in its report to Congress within budget
constraints, quantify the risk associated with achieving its cost
estimates, monitor costs during the implementation phase to ensure that
the proposed changes are being achieved within an acceptable and a
predefined range, and assess the personal property program after the
recommendations have been implemented to determine whether anticipated
improvements are being achieved at a reasonable cost.
In commenting on a draft of this report, DOD concurred with three of our
four recommendations, including initiating actions to implement the
recommendations contained in its report to Congress, monitoring costs
during the implementation phase, and assessing the program after DOD*s
recommendations have been implemented. DOD partially concurred with the
remaining recommendation, i. e., to provide the military services and
Congress with additional information to quantify the risk associated with
achieving the projected 13 percent cost estimate. In its response, DOD
stated that the 5 percent reduction that it made to the pilot programs*
average costs to adjust for economies of scale/ program efficiencies was
reasonable and very conservative. Further, DOD continues to believe that
the program can be implemented within the 13 percent increase and noted
that one of the military services validated this estimate. Therefore, DOD
did not see the value added in providing the military services or Congress
a formal risk assessment. We find that these statements still do not
provide a basis for the 5 percent reduction and do not indicate the level
of risk associated with implementing the recommendation within this
estimate. We continue to believe that this information needs to be
Page 5 GAO- 03- 367 Defense Transportation developed to help the military
services prepare their budgets. The military services terminated
participation in one of the pilot programs because
actual costs were significantly greater than the projections provided to
them for budgetary purposes. Providing a measure of risk for the 13
percent projection could help prevent a repeat of what occurred during
the pilot programs. Without this risk information, the military services
will have to wait until after the moving industry submits bids to find out
if DOD*s projection was reliable.
DOD*s personal property program is managed centrally by the Military
Traffic Management Command headquarters and administered locally by about
200 military service and DOD transportation offices around the world. The
program relies on more than 1,200 domestic commercial carriers and 150
freight forwarders for international shipments to provide household goods
transportation and storage services for military
personnel and their families when they relocate. The military services pay
shipment and storage- related costs from their military personnel accounts
and loss and damage claims and personal property shipment office expenses
through their operations and maintenance accounts. The program has
remained virtually unchanged for nearly 40 years. It involves a complex
process of qualifying carriers, soliciting rates, distributing moves,
evaluating transportation providers* performance, paying invoices, and
settling claims. Among the program*s many challenges is ensuring that the
moving industry provides adequate year- round capacity, especially during
the summer peak- moving season when most service members, as well as the
general public, schedule their moves.
In prior reports, both DOD and GAO have identified problems related to the
loss and damage claims process and the low quality of service from movers.
In designing and implementing its evaluation plan, the
Transportation Command also noted that weaknesses in the current program*s
data management system precluded DOD from being able to track shipments in
transit and from being able to extract reliable data on the number and
types of shipments managed annually and their associated
costs. In response to the long- standing problems, DOD has undertaken a
number of pilot program studies to find ways to improve the process of
shipping service members* household goods. In August 1996, the Deputy
Under Secretary of Defense (Logistics) tasked the Transportation Command
with evaluating alternative approaches to the current program and
recommending changes in the program based on the results of its
evaluation. The Transportation Command identified three ongoing or
Background
Page 6 GAO- 03- 367 Defense Transportation planned pilot programs to
include in its evaluation and began to collect data for its analysis from
one of them in 2000. These three pilot programs
shared some common features, such as testing performance- based service
contracts and providing full replacement (rather than depreciated) value
for loss or damage. Each one also had some distinctive features, such as
allowing service members to participate in selecting their movers and
contracting out installation personal property shipment office functions
to private- sector move manager companies.
The three pilot programs are summarized below. Further information on each
program, as well as DOD*s current personal property program, is provided
in appendix II. The Military Traffic Management Command*s Reengineered
Personal
Property Program operated from the military services* and the Coast
Guard*s installations located in North Carolina, South Carolina, and
Florida. It used military installation personal property shipment office
personnel, as the current program does, and developed a new data
management system that tracked both the movement of individual shipments
and information on the number and cost of shipments.
The Department of Defense*s Full Service Moving Project operated from
the military services* and the Coast Guard*s installations located in the
National Capital Region (the Washington, D. C., metropolitan area),
Georgia, and North Dakota. It contracted the management of shipments to
private- sector companies and offered optional relocation services, such
as referrals for rental assistance and purchase and sale of real estate
services, to personnel participating in the pilot program.
The Navy*s Service Member Arranged Move Pilot Program operated from Navy
installations located in the states of California, Connecticut, Virginia,
and Washington. It designated current staff within the
installation personal property shipment offices as *move coordinators* to
provide assistance, allowed participants to pre- select transportation
providers, and paid for moves through government purchase cards.
In June 2002, the Transportation Command submitted a report containing its
evaluation results and proposed three recommendations to the Deputy Under
Secretary of Defense (Logistics). After reviewing the results and
receiving comments from the military services, DOD submitted its report,
dated November 12, 2002, to Congress. DOD*s report contained the same
three recommendations contained in the Transportation Command*s report.
DOD also provided cost estimates for implementing the recommendations. The
three recommendations were to
Page 7 GAO- 03- 367 Defense Transportation reengineer the liability/
claims process by adopting commercial practices of minimum valuation,
simplifying the filing of claims, and
providing direct settlement with the carrier; change the acquisition
process to implement performance- based
service contracts; and implement information technology improvements,
which could
interface functions across such areas as personnel, transportation,
financial, and claims.
In its report to Congress, DOD estimated that reengineering the liability/
claims process and changing the acquisition process to implement
performance- based service contracts would increase the current program*s
estimated $1.7 billion cost by 13 percent. Implementing the information
technology improvements to enhance its data management capabilities and to
provide training to users was estimated at an additional $4 million to $6
million.
DOD also estimated that efforts to implement the changes to the current
program would be completed by the first quarter of fiscal year 2005. DOD
has developed a plan of action and milestones for designing the new
personal property program. This initial effort identifies several teams,
which are exploring the following issues: the acquisition/ solicitation
process, quality assurance, the liability/ claims process, information
systems technology, and electronic billing and payment. Four of these
issues address the recommendations included in DOD*s November 2002 report
to Congress. The plan identifies a list of essential activities needed to
carry out the responsibilities required to build the future personal
property program. It also includes time lines and identifies a process to
monitor problems and delays. However, it does not include monitoring costs
and benefits during the implementation phase and the extent the proposed
changes are being achieved within an acceptable and a predefined range.
Further, it does not include evaluating the extent the benefits from the
pilot programs will be achieved after the new program is implemented to
determine whether the anticipated improvements were achieved at a
reasonable cost.
Page 8 GAO- 03- 367 Defense Transportation The three recommendations in
DOD*s report to Congress offer solutions to several of the current
program*s long- standing problems, such as the
liability/ claims process and the low quality of service. These problems
have been identified in DOD and GAO*s prior reports, as well as in surveys
conducted as part of the pilot program evaluation. The inability to
monitor shipments and shipping information has been long recognized and
was highlighted as an additional problem during DOD*s evaluation. If
implemented, the first recommendation (i. e., reengineering the liability/
claims process by adopting commercial practices of minimum valuation,
simplifying the filing of claims, and providing direct settlement with the
carrier) has the potential to help reduce the length of time it currently
takes to resolve claims for lost, destroyed, or damaged household goods
because the carrier recovery time would be eliminated for most moves,
increase the reimbursement rates military personnel
receive for their losses, and reduce DOD*s claims- related costs. The
second recommendation (i. e., changing the acquisition process to
implement performance- based service contracts) has the potential to help
improve the generally low quality of service that DOD currently receives
from the moving industry. The third recommendation (i. e., implementing
information technology improvements, which could interface functions
across areas such as personnel, transportation, financial, and claims) has
the potential to improve the program*s ability to reliably monitor and
collect data on the status and costs of shipments so that accurate
reporting can be provided to DOD and Congress.
As part of its evaluation, the Transportation Command cited that one of
the long- standing problems with military household goods shipments is the
liability/ claims process, including the (1) length of time it takes to
resolve claims, (2) low reimbursement rates, and (3) high cost of claims
that DOD must pay.
In a study conducted in 1999, the Military Traffic Management Command
reported that 146 days are expended between the time a claim is filed by a
service member to recovery of costs from the carrier by the government.
During this period, military personnel file their claims for lost,
destroyed, or damaged household goods with their respective military
service*s
claims offices and receive settlements (this occurs, on average, within
DOD*s Recommendations
Offer Solutions to Long- Standing Problems
Reengineered Liability/ Claims Process Has the Potential to Shorten Time,
Increase Reimbursement Rates, and Reduce Claims- Related Costs to DOD
Length of Time to Resolve Claims
Page 9 GAO- 03- 367 Defense Transportation 23 days), and then these
offices file the claims against the carriers to recover the costs (this
step is completed within the 146 day period). 5 In the
Transportation Command*s pre- evaluation survey completed in 2000,
responses from military personnel who had recently moved indicated that
one of the lowest performance ratings involved the time required to settle
a claim. Based on the Transportation Command*s evaluation of the claims
process
under the pilot programs, one of the results from implementing the pilot
programs was the 146 day average required under the current program to
settle claims and recover costs was reduced to an average of 30 days since
the service member filed directly with the carrier and the military
services did not have to recover costs. Under each pilot program, military
personnel settled claims directly with the carriers. Service members who
were not satisfied with offers made by the carriers could file their
unresolved claims directly with DOD. Military services worked these claims
with the carriers and if a claim was justified, the service member
received just settlement under the pilot programs (i. e., if the items
were lost or destroyed, the member received full replacement value, while
damaged items were repaired).
In its pre- evaluation survey, the Transportation Command found that
military personnel cited low reimbursement amounts that typically do not
cover the loss or damage of household goods as a major concern during
their moves. Under the current program, a carrier*s liability is limited
to $1.25 per pound multiplied by the shipment weight. Personnel receive
only the depreciated value of lost, destroyed, or damaged items, up to a
maximum of $40,000 per move. 6 When arranging their moves under the
current program, military personnel can buy increased insurance coverage
from their carrier, up to a full replacement value limit of $3.50 per
pound times the shipment weight, at a cost of 85 cents per $100 of the
stated value of the shipment. However, only military personnel making
moves
5 DOD reported that the 146 day recovery time is due to the fact that all
current DOD contracts for shipment and storage of household goods give the
carriers 120 days from receipt of the government*s demand to pay, deny, or
make a final written offer on the claims. While many claims are settled in
less that 120 days, mailing time and negotiations to
resolve disputes result in a slightly higher average settlement time. 6
For example, a 10,000- pound shipment would have a maximum carrier
liability for loss and damage of $12,500. Service members can make
shipments exceeding their weight allowance by paying the extra shipping
and storage costs for the overage in weight. However, the government*s
liability is limited to $40,000 per move regardless of the excess weight
shipped and stored. Low Reimbursement Rates for
Loss and Damage
Page 10 GAO- 03- 367 Defense Transportation within the continental United
States can buy this additional coverage; it is not available to those
moving to or from overseas posts. Another option that military personnel
have to increase their protection for loss and
damage is to buy additional coverage from private- sector sources. Unlike
the current program, the pilot programs provided full replacement value
for lost and destroyed goods, with maximum amounts ranging from $63,000 to
$75,000. Damaged items were repaired. Two of the three pilot
programs reported that their cost per pound times the shipment weight
rates were $3.50 for $63,000 maximum coverage and $6.00 for $75,000
maximum coverage, respectively. The remaining pilot program did not give a
cost per pound, stating only that its maximum coverage rate was $72,000.
DOD has reported that, historically, approximately 35 percent of all moves
result in loss or damage claims. A 1997 Military Traffic Management
Command survey of 3, 000 moves revealed that while 65 percent of shipments
had loss or damage, only 35 percent resulted in claims being filed. DOD
pays approximately $100 million a year in claims but recovers only 60 to
65 percent of the amounts paid to military personnel from the moving
industry. These figures understate the actual loss and damage, since all
military personnel do not file claims, apparently because the process
takes a long time and reimbursement rates do not always cover the losses.
DOD incurs these losses due to the structure of its current program. The
military claims offices assist service members by arranging to pay their
claims and then submitting the claims to the respective movers for
reimbursement. As indicated above, DOD receives only partial
reimbursement from the moving industry. If the recommendation is
implemented, DOD expects to reduce a substantial portion of the estimated
$100 million it currently pays in claims each year to service members and
eliminate much of the 35 to 40 percent in losses it incurs from settling
claims with the moving industry because service members will be settling
claims directly with their carriers. DOD
also expects additional savings because fewer demands would be placed on
military claims officials to manage the claims process. DOD believes that
these savings will help offset the higher costs of providing full
replacement value to service members for any loss and damage incurred
during the shipment and storage of their personal property. High Costs to
Government for
Claims
Page 11 GAO- 03- 367 Defense Transportation Our work has shown that
another long- standing problem with the current personal property program
is the poor quality of moving services provided
to military personnel. The high number of loss and damage claims that
military personnel file underscores this problem. According to the two
pre- evaluation surveys cited in the Transportation Command*s evaluation,
around 55 to 65 percent of respondents reported suffering some loss or
damage of household goods during a recent move. Moreover, in the pre-
evaluation survey conducted by the Transportation Command, the top four
factors identified by service members as being of greatest importance to
them in the moving process were the quality of packing, the care in
handling personal property, the condition of their property upon receipt
at the end of the move, and the receipt of fair payment for any losses or
damages they suffered. In individual comments obtained during the pre-
evaluation survey, the Transportation Command reported that some service
members also cited the lack of professionalism and quality of
customer service on behalf of moving crews as a concern. The problem stems
primarily from the current program placing greater emphasis on costs (i.
e., the lowest bids) than on the quality of service that carriers provide
when moving shipments of military household goods.
While the current program established its Total Quality Assurance Program
to measure quality, data collected to develop scores for each carrier
includes three measures (timeliness of pickup, timeliness of delivery, and
reported loss and damage), which are not collected for all household goods
shipments. The best indication of quality, customer satisfaction, is not
measured in the current program. The problem of quality is further
exacerbated by the program*s use of a 20 year- old tariff schedule that
carriers use in developing their bids. This tariff contains lower rates
than the current commercial tariff used during the pilot programs.
Unlike the current program, the pilot programs screened carriers that
wanted to participate in their programs by emphasizing the quality of
carriers* prior performance rather than the amount of their bids. For
example, the Full Service Moving Project contracted a financial services
company to conduct a financial and performance assessment of potential
movers. The pilot program emphasized best value and placed more emphasis
on performance (70 percent) than cost (30 percent) in determining which
providers were awarded shipments. The pilot programs showed that these
types of contracts could allow the government to pre- screen carriers for
financial viability and, more importantly, to institute and maintain a
quality assurance process to reduce losses and improve service. Change to
PerformanceBased
Service Contracts Could Improve Quality of Moving Services
Page 12 GAO- 03- 367 Defense Transportation In addition to prescreening
carriers for quality control purposes, the pilot programs also surveyed
military personnel who participated in the
programs and used the results to distribute future shipments to carriers
that received the best performance scores. To address concerns about the
obsolete tariff schedule, the pilot programs adopted current commercial
tariffs for carriers to use in establishing their bids.
The pilot programs also showed that the solicitation process could be
streamlined by eliminating detailed statements of work and that the pilot
programs could place responsibility for successful performance on
carriers, allowing the government to focus on outcomes, rather than
processes. Finally, the pilot programs demonstrated that using these types
of performance- based service contracts did not have an adverse effect on
small business participation, a major concern of the moving industry. On
the basis of the total dollar value of shipments, each pilot program
exceeded the Small Business Administration*s goal of 23 percent
participation for the industry. Specifically, 48 percent of the Military
Traffic Management Command*s Reengineered Personal Property Program*s
revenues, 74 percent of the Full Service Moving Project*s revenues, and
100 percent of the Navy*s Service Member Arranged Move*s revenues went to
small businesses.
Another ongoing problem with the current personal property program is its
inability to provide reliable data on the status of in- transit shipments
or on the number and associated costs of shipments managed by DOD each
year. Because of the lack of reliable data on shipments and costs, program
managers have no way of knowing the actual costs of moving military
personnel*s household goods. In addition, they have no access to real-
time tracking data that they could use to manage transportation and
storage costs and to help cut down on the need for temporary storage by
reducing the number of failed deliveries.
Two of the pilot programs included features to address the problems
associated with the current program*s stand- alone data management system.
The pilot programs each developed a Web- based data management system to
enhance the visibility of individual shipments and provide more reliable
data on shipments and costs. For example, the Military Traffic Management
Command*s Reengineered Personal Property Program*s data management system
provided in- transit visibility. This made it possible to
track the status of individual shipments and gave real- time access to
those sections of the shipment records that various parties involved in
the relocation process needed for data entry or status review. The pilot
program*s data management system provided a complete picture of the
Implementing Information
Technology Improvements to Address Data Reliability Problems
Page 13 GAO- 03- 367 Defense Transportation service member*s move from
start (the move application process) to finish (the claims submission and
resolution process). In addition, the data
management system demonstrated the potential to provide information to
personnel in various functional areas involved in the service members*
relocation process (such as personnel, transportation, financial, and
claims). Finally, the data management system demonstrated the potential to
provide data for planning and budgeting purposes on the types of
shipments made annually across DOD and their costs. The Full Service
Moving Project*s data management system was developed but not fully
implemented because the military services terminated their participation
in the pilot program due to its high costs. While the Navy*s pilot program
developed a database near the end of the Transportation Command*s
evaluation, the database was not fully implemented nor assessed as part of
the evaluation.
Our analysis indicated that DOD*s three recommendations are supported by
the results of the Transportation Command*s evaluation of the three pilot
programs. The Transportation Command adopted a sound methodology to
conduct its evaluation, and it adjusted this methodology when
circumstances warranted. The results of the Transportation Command*s
evaluation are based on data collected from a limited number of
geographical areas. While the shipments included in the evaluation do not
represent all shipment types managed annually by DOD, we believe
that the evaluation results provide sufficient information to allow DOD to
initiate actions to improve its current personal property program within
budget constraints.
We found that the Transportation Command used a methodologically sound
approach to evaluate the results of the three pilot programs and make its
recommendations. Before it started the evaluation process, the
Transportation Command considered some lessons learned that had emerged
from our review of the Hunter Pilot Program in 1999, 7 and it followed
through with several of them. For example, it obtained assistance from a
contractor to design an evaluation plan that met professional standards.
The Transportation Command identified four aspects, or factors, of the
property program that served as the focus of its evaluation (i. e.,
quality of life, total costs, small business participation, and process
7 U. S. General Accounting Office, Defense Transportation: The Army*s
Hunter Pilot Project Is Inconclusive but Provides Lessons Learned, GAO/
NSIAD- 99- 129 (Washington, D. C.: June 23, 1999). Transportation
Command*s Evaluation of Pilot Programs Supports DOD*s Three
Recommendations
Transportation Command Implemented a Methodologically Sound Evaluation
Page 14 GAO- 03- 367 Defense Transportation improvements). The evaluation
plan also prescribed that only one quality of life survey be administered
to each participating service member in order to avoid survey *fatigue*
that can result from subjecting a person to multiple surveys, and thus
avoid the resulting potential for questionable
results. In designing the evaluation plan, the Transportation Command
incorporated a number of important evaluation features. These features
included assessing the four factors consistently across all three pilot
programs, ensuring that the evaluation received data from the pilot
programs during the same time period to avoid the need to make adjustments
due to potential changes in carrier operations and costs, conducting a
survey of service members using the current program to establish a
baseline from which to measure the pilot program results, and developing a
method to provide estimates of what DOD would have paid for comparable
shipments under the current program for those shipments completed under
the pilot programs. The Transportation Command made appropriate
adjustments to the
evaluation plan when it learned that the three pilot programs would not be
underway at the same time and that they would not provide all of the
information originally outlined in the plan. For example, the Full Service
Moving Project began later and terminated earlier than was expected, the
Navy*s Service Member Arranged Move Pilot Program failed to conduct a
quality of life survey and collect cost data as outlined in the evaluation
plan, and none of the three pilot programs provided costs associated with
individual process improvements. The Transportation Command included
qualitative analytical techniques so that it could include as much
information on each pilot program as possible in its evaluation while also
dealing appropriately with data limitations. The Transportation Command
also shifted the evaluation focus from the individual pilot programs to
specific features from the three programs, such as full replacement value
for loss and damage and the screening process for carrier participation.
Our work indicated that the Transportation Command*s analysis of data
collected from the three pilot programs supports the three recommendations
that DOD included in its report to Congress. The Transportation Command*s
analysis of household goods shipment data from the pilot programs showed
that the average amount of time that service members and DOD spend to
settle claims and recover costs from carriers fell dramatically in all
three pilot programs. In comparison with the current program*s 146- day
average, it took only 30 days, on average, to settle a claim under the
Reengineered Personal Property Program and the Full Service Moving Project
and fewer than 14 days under the Navy*s program. Survey results indicated
that full replacement (rather than Survey and Analytical Data
Support Command*s Recommendations
Page 15 GAO- 03- 367 Defense Transportation depreciated) value, direct
claims settlements, and anticipated improvements in the claims process
accounted for the highest increases in
satisfaction. Based on experiences during the pilot programs, DOD believes
that direct claims settlement between service members and carriers should
reduce claims costs DOD currently incurs. Under the current program, DOD
must collect from the carriers after it has paid the service members*
claims. DOD expects that this step will be eliminated in most instances
because it is anticipated that service members will be resolving most of
their claims directly with their carriers.
The Transportation Command*s analysis of process improvement data,
interviews and observations during site visits, and survey results from
the pilot programs supported DOD*s recommendation to use performancebased
service contracts to improve the quality of services that the moving
industry provides to the military. The process of prescreening carriers
desiring to participate in the pilot programs on the basis of their
financial viability and past performance helped to eliminate poor
performers.
Furthermore, the pilot programs* use of post- move surveys allowed them to
get immediate and continuous feedback on the carriers* performance and to
use this information to distribute future work to those carriers with the
highest performance ratings and best value. In addition, two of the pilot
programs reduced the amount of paperwork associated with soliciting
proposals and approving carriers.
Finally, the Transportation Command*s review and observations of two of
the pilot programs* Web- based data management systems supported DOD*s
recommendation to overhaul the current personal property program*s
computer system (the Transportation Operational Personal Property Standard
System). The Transportation Command found that the Reengineered Personal
Property Program*s data management system significantly improved
communications between the various DOD offices
and the moving industry. The system gave real- time access to shipment
records to DOD*s personal property shipment offices, certifying officers,
prepayment auditors, military service headquarters, and military service
claims offices and finance centers, as well as moving industry
participants. Similar results occurred with the Full Service Moving
Project*s Best Value
Distribution Database system, but the military services terminated their
participation in this pilot program before the system*s full potential
could be demonstrated.
While the shipments included in the evaluation do not represent all the
shipment types managed annually by DOD, we believe that the evaluation
results provide sufficient information to allow DOD to initiate actions to
improve its current personal property program.
Page 16 GAO- 03- 367 Defense Transportation Our review found that the
estimates DOD reported to Congress might understate the total initial cost
for implementing the information technology improvements recommendation
and contain a questionable
adjustment for costs associated with the claims and contracting process
recommendations. Also, DOD did not quantify the risk associated with
implementing these latter recommendations within its projected 13 percent
increase over the current program*s cost. Therefore, the ability
to implement changes to the existing program within the cost estimates
reported to Congress is uncertain.
Based on our discussions with Military Traffic Management Command
officials and review of available documents, we concluded that the total
initial cost to implement the information technology improvements
recommendation will more likely be $7 million rather than the $4 million
to $6 million estimate that DOD previously reported to Congress. In its
response to a draft of this report, DOD maintained that the costs to
implement a new Web- based data management system would fall within its
initial cost estimate of $4 million to $6 million. DOD*s projected cost
estimate includes $5 million for development and implementation of the new
system and $500,000 each for user training and system verification and
validation testing. At a minimum, based on these projected cost estimates,
the initial cost to implement the information technology improvements
recommendation would more likely be $6 million.
While we concur with the premise of two of the three adjustments used to
develop the 13 percent cost increase to implement the remaining
recommendations, we are less assured in the extent to which the projected
savings related to the third adjustment may occur. We found that the first
two adjustments were based on historical data. However, we question the
rationale DOD used to develop the third adjustment, as the savings
associated with this adjustment are based on assumed cost reductions
resulting from changes in program operations. Also, these reductions lack
the same quality of evidentiary support as DOD provided for the other two
adjustments.
DOD believes it took a conservative approach in developing the savings in
each of the three adjustments; therefore, it assumes that the proposed
changes to claims and the contracting process can be achieved within the
13 percent increase over the current program*s costs. Due to the
longstanding problems with this program and the high pilot program costs
that contributed to the military services* early termination of
participating in
one of the pilot programs, we believe that by quantifying the risk
associated with this projection, DOD could provide the military services
and Congress information needed to develop and review future budget
Ability to Implement New Program within
Cost Estimates Reported to Congress Is Uncertain
Page 17 GAO- 03- 367 Defense Transportation requests for this program.
Further, without carefully monitoring costs during the implementation
process and assessing costs and benefits from a
period succeeding full implementation of the recommendations, DOD will not
have the information needed to determine if anticipated improvements in
the program are being achieved at a reasonable cost. Currently, DOD is
beginning planning efforts to implement the recommendations. These efforts
do not include monitoring and evaluating costs and benefits during the
implementation phase and post implementation of the recommendations in a
new program.
The information DOD has provided on costs to implement the information
technology improvements recommendation varies. Information provided during
our review indicated that the total initial cost to improve the current
data management system would be higher than the $4 million to $6 million
DOD included in its report to Congress. DOD worked with the contractor who
developed the Reengineered Personal Property Program*s Web- based data
management system to develop an estimate of the cost to expand the
capabilities tested during the pilot program. Also included in this
estimate were funds to provide training for users of the new system.
Based on our discussions with officials from the Military Traffic
Management Command and our review of available documents, we concluded
that these costs would more likely be $6 million, as the data management
system development cost was projected to be $5 million with an additional
$1 million for user training. The need for this training as part of a new
personal property program was identified during DOD*s
evaluation of the pilot programs. We increased our overall projections for
the cost of the new system to $7 million when we learned that DOD planned
to continue spending at least another $1 million annually for independent
verification and validation testing and contractor support. This latter
expense was identified to us during discussions following DOD*s submission
of its report to Congress.
In its response to a draft of this report, DOD maintained that the costs
to implement a new Web- based data management system would fall within its
initial cost estimate of $4 million to $6 million. It projected a cost of
$5 million for system development and implementation and $500,000 each for
user training and initial system validation. At a minimum, based on these
projected cost estimates, the initial cost to implement the information
technology improvements recommendation would more likely be $6 million.
Because we did not assess the sufficiency of DOD*s original estimates of
$1 million each for training and validation testing, we are unable to
assess the impact of the reduction on the improvements in information
technology across DOD. Costs to Implement Information Technology
Improvements Vary
Page 18 GAO- 03- 367 Defense Transportation Based on our discussion with
DOD officials, we learned that the plan is to implement this
recommendation regardless of the status of the other two
recommendations because managers and users of the program need more
reliable information to manage the program*s shipments and their costs.
Funds to implement this recommendation would come from the military
services* operations and maintenance accounts.
The soundness of the three adjustments the Military Traffic Management
Command used to develop its estimated 13 percent increase over the current
program costs to implement the remaining recommendations* the claims
process and performance- based service contracts* varies. We found that
two of these adjustments are based on reasonable assumptions and are
supported by historical experience and by data. The savings associated
with the third adjustment are based on assumed cost reductions resulting
from changes in program operations and lack the same quality of
evidentiary support as DOD provided for the other two adjustments.
Therefore, we are less assured in the extent to which the savings
associated with this adjustment may occur. Finally, we found that in its
report to Congress, DOD did not quantify the risk of achieving these
recommendations within the projected 13 percent increase. This information
is important to the military services as they develop their military
personnel and operations and maintenance budget requests and to Congress
as it assesses the reasonableness of these requests.
In developing the 13 percent estimate, the Military Traffic Management
Command determined that three adjustments to the average costs for the
pilot programs were required to develop the cost for the full rollout of a
new personal property program. The first two adjustments (i. e., reducing
the average weight of shipments and reducing costs to adjust for a mix of
small and large businesses) were made to offset differences between the
pilot programs* shipments and those more typically managed across DOD. The
third adjustment was made to reduce the pilot programs* costs to
reflect anticipated savings based on economies of scale. 8 In developing
these adjustments, the Military Traffic Management Command worked with a
contractor and consulted with officials from the military services and
moving industry associations.
8 In later discussions, department officials stated that *economies of
scale* should be changed to *program efficiencies* to reflect a more
efficient program with timely and accurate management data; member
counseling; and reduced loss, storage, and indirect costs. Soundness of
Estimates to Develop Cost for Changing
Claims and Contracting Processes Varies
Estimate Includes a Questionable Cost Adjustment
Page 19 GAO- 03- 367 Defense Transportation While we believe that the
shipment weight and small business mix adjustments are reasonable, we
question the extent to which the
economies of scale or program efficiencies adjustment may be achieved. For
the weight adjustment, the Military Traffic Management Command determined
that the average weights of moves in the two pilot program areas were
higher than those experienced in typical departmentwide
moves. As a result, the Military Traffic Management Command reduced the
pilot programs* average weights to reflect the lower, more typical weights
to be used in calculating a total cost for a departmentwide program. This
adjustment resulted in a 12 percent drop in average costs. We found the
approach of using historical data to more accurately reflect the typical
shipment weights to be reasonable. Next, the Military Traffic Management
Command further lowered the pilot programs* average costs because the
pilot programs had higher small
business participation rates than the departmentwide average, and small
businesses are typically more expensive than large businesses. Small
businesses accounted for 48 percent of the cost of all moves under the
Reengineered Personal Property Program and 73 percent under the Full
Service Moving Project. In addition, small businesses were 14 percent more
expensive per shipment in the Reengineered Personal Property Program and
74 percent more expensive in the Full Service Moving Project than what
each pilot program paid to large businesses. In developing its
departmentwide estimate, the Military Traffic Management Command used a
small business participation target rate of 30 percent. This 30 percent
target rate is higher than the Small Business Administration*s
23 percent goal for government agencies conducting business with this
industry. On the basis of this lower participation rate, the Military
Traffic Management Command reduced the pilot programs* average costs
further by 8 percent. We agree that this adjustment in costs based on
differences in the pilot programs* small business participation rate and
the new 30 percent goal is a reasonable way to reflect the differences
between the pilot programs* costs and the departmentwide- projected costs.
We found that the third adjustment that the Military Traffic Management
Command made* to reduce the cost of departmentwide shipments because of
economies of scale or program efficiencies* was not adequately supported
based on either historical experience or data that DOD later provided. The
Military Traffic Management Command reduced
the pilot programs* average costs by 5 percent on the assumption that
the pilot programs* shipments involved only a limited number of
providers;
Page 20 GAO- 03- 367 Defense Transportation the pilot programs only
included a limited number of shipments while the current program manages
over 600,000 shipments annually;
more accurate and timely management data that includes service member
counseling, reduced losses, and storage and indirect costs will result in
a more efficient program; and
overhead and operating costs will be spread due to a larger volume of
shipments.
While recognizing that some changes may result from these anticipated
program efficiencies, the effect of these changes on potential cost
savings is uncertain at this time. The Military Traffic Management Command
did not provide the same level of evidentiary support that it provided on
the other two adjustments. Further, we believe that only time will
determine if DOD*s assumption for this adjustment, in particular, proves
to be correct.
We found that DOD has not provided a level of assurance to the military
services and Congress that its projected 13 percent increase over the
current program*s cost can be achieved. Quantifying the risk associated
with this projection could provide the military services assurance of the
viability of the projected 13 percent increase as they prepare budgets to
support the increased cost for this program. Congress could also use this
information as it reviews DOD*s requests for additional funds to implement
changes in this program. The need for this type of information is further
supported based on the long- standing problems associated with the current
program and the fact that shipment and storage costs under the pilot
programs were significantly higher than those that DOD estimated it would
have paid under its current program in the same geographical areas. These
costs ranged from 31 to 32 percent higher under the Reengineered Personal
Property Program and from 51 to 54 percent higher under the Full Service
Moving Project. 9 These higher- thananticipated costs contributed to the
military services* decision to terminate their participation in the Full
Service Moving Project before its test period ended.
While DOD did not quantify the risk, per se, it believes a conservative
approach was taken in developing the savings in each of the three
adjustments. As a result, DOD assumes that the proposed changes to the
claims and contracting processes can be achieved with its projected
9 We believe that the methodology and data used to develop these cost
estimates are sound. DOD Did Not Quantify the Risk
for Its Cost Estimate
Page 21 GAO- 03- 367 Defense Transportation increase of 13 percent over
the current program*s budget. We still believe that the Military Traffic
Management Command could have quantified the
risk and provided this additional information to the military services and
Congress as additional assurance of the likelihood of implementing the two
recommendations within its projected 13 percent increase. The need for
this information is further supported based on the long- standing problems
DOD has experienced in this program, the fact that the military services
terminated participation in one of the pilot programs due to the high cost
increases, and the need to determine whether the proposed additional funds
from military personnel and operations and maintenance accounts will be
sufficient to implement the recommendations.
In addition to the information that could be gained from quantifying the
risk of its cost projection, we believe that only by careful monitoring
during the implementation phase will DOD be able to ensure that the
proposed changes are being achieved within an acceptable and a predefined
range. Further, while we believe that the evaluation results support
implementing plans to enhance the current program, it should be noted that
the pilot programs* shipments included in the evaluation were not typical
of all types of shipments managed annually. Therefore, DOD was precluded
from projecting the extent to which the recommended improvements can be
achieved DOD- wide. Unless a subsequent evaluation is undertaken after the
recommendations have been implemented, DOD
will not be able to assess the extent to which the projected benefits are
being achieved for military personnel, their families, and DOD, and
whether the benefits are being achieved at a reasonable cost. Selecting an
evaluation period to include the peak- moving season would also provide
DOD with the information its needs to determine if the proposed changes
can be achieved during the summer, when the demand for moving services by
DOD and the private sector is at its highest.
The three recommendations DOD developed from its evaluation of the current
and pilot programs, if implemented successfully, could enhance the quality
of life for relocating service members and their families; reduce claims-
related costs to DOD; and resolve problems related to the reliability of
management information on the status of shipments and on the quantity,
types, and costs of shipments that DOD and the military services manage
annually. Delaying implementation of the recommendations only prolongs
problems military personnel, their families, and DOD experience
under the current program. DOD has not quantified the risk associated with
achieving its projected 13 percent increase over the current program*s
costs to implement the Conclusions
Page 22 GAO- 03- 367 Defense Transportation claims process and
performance- based service contract recommendations. Without quantifying
the risk, the military services and Congress cannot be
assured that these recommendations can be achieved within this estimate or
whether additional funding or trade- offs may be needed. Further, without
careful monitoring during the implementation phase, DOD will not be able
to ensure that the proposed changes are being achieved within an
acceptable and a predefined range. Because the pilot programs* shipments
included in the evaluation were not
typical of all types of shipments managed annually, it was not possible
for DOD to project the extent to which the recommended improvements can be
achieved departmentwide. Without evaluating the program following
implementation of the recommendations, DOD will be unable to assess the
extent to which the projected benefits for military personnel, their
families, and DOD are being achieved and, if so, whether they are being
achieved within a reasonable cost. Also, if DOD does not select an
evaluation period that includes the peak- moving season, it will not have
the information needed to determine if the proposed changes can be
achieved in the summer, when the demand for moving services is at its
highest.
To improve the personal property program for military personnel, their
families, and program administrators, we recommend that the Secretary of
Defense direct the Commander, U. S. Transportation Command, to
initiate actions to implement the three recommendations contained in
DOD*s report to Congress within budget constraints,
provide the military services and Congress additional information to
quantify the risk associated with achieving the projected 13 percent cost
estimate before the claims process and performance- based service
contracts recommendations are implemented to provide the military services
with information needed for budgeting purposes,
monitor costs for all recommendations during the implementation phase to
ensure that the proposed changes are being achieved within an acceptable
and a predefined range, and
assess the effects of the three recommendations on the personal property
program (to be carried out after the summertime peak- moving season once
the recommendations have been implemented) to determine whether the
anticipated improvements in the program are being achieved at a reasonable
cost. Recommendations for
Executive Action
Page 23 GAO- 03- 367 Defense Transportation In commenting on a draft of
this report, DOD concurred with three of our four recommendations. For the
first of these recommendations, DOD
stated that it is developing a plan to implement those recommendations it
reported to Congress and anticipates its recommendations will be
implemented by the end of the first quarter of fiscal year 2006, assuming
the military services receive the additional funds needed to fund program
enhancements. In response to our recommendation to monitor costs during
the implementation phase, DOD stated that rolling out the new program will
require monitoring of costs to determine if the moving industry partners
are submitting bids that will allow DOD to enhance this program within the
projected 13 percent cost increase. Further, DOD plans to include a
process to conduct a rate reasonableness analysis upon receipt of the
rates. For rates found to be outside the range of reasonableness, carriers
will be given one opportunity to resubmit their
rates. DOD plans to only use those rates determined to be reasonable in
the new program. DOD also plans to include metrics, target/ benchmark
performance indicators, and a methodology for data collection in an
updated program of action and milestone plan. For our recommendation, i.
e., assess the effects of the three DOD recommendations on the personal
property program to determine whether the anticipated improvements in the
program are being achieved at a reasonable cost, DOD plans to collect data
needed to determine if anticipated improvements have been achieved
on a continuing basis. DOD plans to use customer satisfaction surveys in
developing carrier performance ratings, which will be established
quarterly, with the exception of the peak season, when performance ratings
will be established monthly. If properly implemented, we believe the
proposed DOD actions will sufficiently address these recommendations.
DOD partially concurred with the remaining recommendation, i. e., provide
the military services and Congress with additional information to quantify
the risk associated with achieving the projected 13 percent cost estimate
to provide the military services with information needed for budgeting
purposes. DOD continues to believe that the 5 percent reduction it made to
pilot programs* average costs to adjust for economies of scale/ program
efficiencies was reasonable and very conservative and that the program can
be implemented within the projected 13 percent increase over current
program costs. DOD also reported that one of the military services
validated the 13 percent cost increase following our audit. Further, DOD
stated that it did not see value added in providing the military services
or Congress a formal risk assessment but will continue to work with the
military services as execution progresses to make sure they have all
information required for budget purposes. Additionally, while not part of
this recommendation, DOD also said it did not concur with our finding
Agency Comments
and Our Evaluation
Page 24 GAO- 03- 367 Defense Transportation that the cost estimate for
implementing its information technology improvements recommendation would
be $7 million.
In reviewing the response, we found that DOD still did not provide any
data to support its assumption of a 5 percent cost savings from economies
of scale/ program efficiencies. DOD stated that the new program will be
about 200 times larger than the pilot programs and that the resulting
increase in volume will lower the cost per unit, a standard and accepted
law of economics. While we agree that the cost may decrease, it may also
increase or remain unchanged. Moreover, the cost may decrease by less
than 5 percent. Without specific data showing the per move costs will
decrease as the scale of operations increase, we continue to question the
basis for DOD*s assumption of a 5 percent reduction.
We believe that the validation effort completed by one of the military
services, along with the calculations and assumptions DOD used in
developing the 13 percent cost estimate, does not provide the military
services and Congress with information needed to reliably develop and
review budget requests to fund enhancements to the current program. We
continue to believe that DOD needs to qualify this estimate with a measure
of the risk associated with implementing its recommendations. Without
providing the range of possible cost increases and the risk regarding the
likelihood of achieving this 13 percent projection within that range, DOD
may encounter a repetition of its experience with one of the pilot
programs, which had to be terminated because actual costs exceeded
projected costs. Absent this risk information, the military services will
have to wait until after the transportation providers submit their bids in
order to learn whether the recommendations can be implemented within the
13 percent projection. Should the bids result in costs that exceed this
estimate, DOD and the military services will need to make adjustments to
ensure that the recommendations are implemented within funding limits.
Therefore, we continue to believe that our recommendation has merit. Our
finding that the implementation of the information technology improvements
recommendation would likely cost $7 million rather than the $4 million to
$6 million that DOD projected was based on information we received from
DOD during the audit. Specifically, we calculated that the costs to
develop and implement the new system would be about $5 million and that
training for users of the enhanced system would cost
an additional $1 million. DOD had identified the need for this training
during its evaluation of the pilot programs. After DOD submitted its
report to Congress, it identified another potential cost* an additional $1
million for independent verification and validation testing of the system.
Our $7 million estimated included all three of these cost elements. In its
Page 25 GAO- 03- 367 Defense Transportation response to a draft of this
report, DOD maintained that its costs estimate would fall within its
initial cost estimate of $4 million to $6 million,
including $5 million for system development and implementation and an
additional $500,000 each for user training and system verification and
validation testing. At a minimum, based on these projected cost estimates,
the initial cost to implement the information technology improvements
recommendation would more likely be $6 million. However, since we did not
originally assess the sufficiency of the $1 million estimates for training
and testing, we are unable to assess what impact DOD*s reduction for these
costs to $500,000 would have on the implementation of the system across
DOD. We have reflected DOD*s changes in the body of our report.
DOD*s comments are reprinted in appendix III. DOD also provided technical
comments, and we revised our report to reflect them where appropriate. We
performed our review from April 2002 through February 2003 in
accordance with generally accepted government auditing standards. Appendix
I contains the scope and methodology for this report. DOD*s comments are
reprinted in their entirety in appendix III.
We are sending copies of this report to the appropriate congressional
committees; the Secretary of Defense; the Commander, U. S. Transportation
Command; and the Director, Office of Management and Budget. We will also
make copies available to others upon request. In addition, the report will
be made available at no charge on the GAO Web site at http:// www. gao.
gov.
Page 26 GAO- 03- 367 Defense Transportation Please contact me at (202)
512- 8365 or Lawson Gist, Jr., at (202) 512- 4478 if you or your staff
have any questions concerning this report. Key contributors to this
assignment were Robert L. Self, Jacqueline S. McColl,
Arthur L. James, Jr., Charles W. Perdue, and Nancy L. Benco. William M.
Solis Director Defense Capabilities and Management
Page 27 GAO- 03- 367 Defense Transportation List of Congressional
Committees The Honorable John Warner Chairman
The Honorable Carl Levin Ranking Minority Member Committee on Armed
Services United States Senate
The Honorable Ted Stevens Chairman The Honorable Daniel K. Inouye Ranking
Minority Member Subcommittee on Defense Committee on Appropriations United
States Senate
The Honorable Duncan Hunter Chairman The Honorable Ike Skelton Ranking
Minority Member Committee on Armed Services House of Representatives
The Honorable Joel Hefley Chairman The Honorable Solomon P. Ortiz Ranking
Minority Member Subcommittee on Readiness Committee on Armed Services
House of Representatives
The Honorable Jerry Lewis Chairman The Honorable John P. Murtha Ranking
Minority Member Subcommittee on Defense Committee on Appropriations House
of Representatives
Appendix I: Scope and Methodology Page 28 GAO- 03- 367 Defense
Transportation To assess the extent to which the recommendations in the
Department of Defense*s (DOD) November 2002 report to Congress addressed
major
problems in the personal property program, we took the following steps:
To identify the major problems facing the current personal property
program, we reviewed DOD and GAO reports addressing this program. These
reports identified problems associated with quality of service and claims.
We also conducted interviews with personal property program officials and
their contractors to gain an understanding of the current data management
system*s limitations and the long- standing problems involving the lack of
reliable information on shipments and their costs.
To determine whether the proposed recommendations in DOD*s report to
Congress addressed the major problems of the current program, we tracked
the recommendations back to the U. S. Transportation Command*s report on
its evaluation results and assessed the extent to which the
recommendations are linked to and have the potential to address problems.
To assess whether the recommendations in DOD*s report to Congress were
supported by DOD*s evaluation findings and should be implemented, we took
the following steps:
To determine if the Transportation Command developed a methodologically
sound evaluation plan, we assessed the command*s efforts against the
findings and recommendations contained in our report 1 on the Army*s
Hunter Pilot Program results and against professional standards 2 we would
use if conducting a comparable evaluation. These sources addressed issues
such as (1) seeking advice in designing a methodologically sound
evaluation plan, (2) developing the evaluation plan prior to testing, (3)
identifying factors to be assessed and the data required for analyses to
develop findings and recommendations, (4) limiting quality of life surveys
to only one for each participant to preclude survey *fatigue,* and (5)
conducting simultaneous testing of the pilot and current programs.
1 GAO/ NSIAD- 99- 129. 2 U. S. General Accounting Office, Developing and
Using Questionnaires, GAO/ PEMD10.1.7 (Washington, D. C.: Oct. 1993) and
U. S. General Accounting Office, Using Structured Interviewing Techniques,
GAO/ PEMD- 10.1.5 (Washington, D. C.: June 1991). Appendix I: Scope and
Methodology
Appendix I: Scope and Methodology Page 29 GAO- 03- 367 Defense
Transportation To determine if the Transportation Command implemented an
effective
evaluation strategy during the data collection phase of its evaluation, we
reviewed the pilot programs* efforts to collect data for the four factors
as prescribed in the Transportation Command*s evaluation plan. We also
assessed the adjustments the Transportation Command made in its evaluation
strategy to address issues that could affect the soundness of the results.
An example of the issues addressed included developing a constructed cost
methodology to provide better estimates of what DOD would have paid under
the current program for shipments made by the pilot programs.
To assess the Transportation Command*s development of findings and
recommendations to improve the current personal property program, we
reviewed the evaluation techniques (quantitative and qualitative analyses)
used to analyze data collected for the four factors. Further, we assessed
the extent to which the Transportation Command adjusted the evaluation
techniques to make up for differences in the way that the pilot programs
provided data for the evaluation.
To assess the methodology that DOD used to develop cost estimates for
implementing the recommendations, we took the following steps:
To determine the reliability of the cost estimates for the pilot
programs and for the proposed recommendations, we reviewed the cost
projection methodologies used by the Transportation Command and by the
Military Traffic Management Command.
To determine the reliability of pilot program shipment- related costs
used in the report, we reviewed the data collection efforts used by each
pilot program for the transportation and storage of household goods
included in the Transportation Command*s evaluation. Further, we reviewed
the constructed cost methodology used to develop the estimates of what DOD
would have paid to make comparable shipments under the current program in
the pilot programs* test areas.
To determine the reasonableness of the assumptions and sources of data
used to develop cost estimates for implementing recommendations for the
personal property program, we met with officials from the Military Traffic
Management Command and their contractor to discuss the methodology. We
also reviewed the contents of their briefing on the cost estimate work for
implementing changes to the claims process and performance- based service
contracts and
Appendix I: Scope and Methodology Page 30 GAO- 03- 367 Defense
Transportation additional information the Military Traffic Management
Command
provided on the costs to implement information technology improvements.
We did not make an assessment of whether the anticipated benefits to be
derived from implementing the three recommendations would warrant the
additional costs DOD projects will be required to fund these improvements.
Furthermore, we did not independently test the reliability of data DOD
extracted from its data system to develop costs. We found that the
department placed proper caveats on their use of such data, and in the
case of comparing pilot programs* shipment costs to current program costs,
developed a constructed cost methodology to address current program data
management system weaknesses.
During this and prior 3 reviews of DOD*s evaluation efforts, we met with
officials and obtained documents from the Office of the Assistant Deputy
Under Secretary of Defense (Transportation Policy), Washington, D. C.; the
U. S. Transportation Command, Scott Air Force Base, Illinois; the Military
Traffic Management Command, Alexandria, Virginia; the Department of
Defense Inspector General, Full Service Moving Project, and Hay Group
(Transportation Command Contractor), Arlington, Virginia; American
Management Systems (Transportation Command contractor),
PricewaterhouseCoopers (Military Traffic Management Command contractor),
and Systems Research and Applications (Military Traffic Management Command
contractor), Fairfax, Virginia; Logistics Management Institute (Military
Traffic Management Command
contractor), McLean, Virginia; the Navy*s Service Member Arranged Move
Pilot Program, Mechanicsburg, Pennsylvania; The Gallup Organization (Full
Service Moving Project contractor), Omaha and Lincoln, Nebraska; and
Parsifal Corporation (Military Management Traffic Command contractor),
Palm Bay, Florida. In addition to these agency meetings and documents, we
drew upon information contained in a testimony
3 Our prior reviews focused on the department*s efforts (1) to develop a
methodologically sound evaluation plan and (2) to collect data according
to the plan for future analysis and development of recommendations for an
improved departmentwide program.
Appendix I: Scope and Methodology Page 31 GAO- 03- 367 Defense
Transportation statement, in reports, and in status briefings resulting
from our prior reviews of this program.
Our work for this review was performed from April 2002 through February
2003 in accordance with generally accepted government auditing standards.
Appendix II: Overview of Current Personal Property Program and Pilot
Programs Page 32 GAO- 03- 367 Defense Transportation The Transportation
Command evaluated three pilot programs to assess alternative approaches
that might address long- standing problems with its
current personal property program. The following tables provide features
of the current program and the three pilot programs. As the tables show,
the pilot programs had several features that provided enhancements to
military personnel and their families and to DOD that are not offered by
the current program.
Table 1 compares claims- related features. Specifically, the pilot
programs provided full replacement value rather than depreciated value for
loss and damage and guaranteed claims settlement with 45 to 60 days of
filing the claims.
Table 1: Claims- Related Features of the Current Personal Property Program
and Pilot Programs Program features Current personal
property program Military Traffic
Management Command*s Reengineered Personal Property Program
Department of Defense*s Full Service Moving Project
Navy*s Service Member Arranged Move Pilot Program Loss and damage claims
Reimbursements: *Basis for valuing property Depreciated value Full
replacement value Full replacement value Full replacement
value *Maximum dollar value per move $40,000 $63,000 $75,000 $72,000
Guaranteed claims settlement Not specified Within 60 days Within 45 days
Within 60 days Sources: DOD (data); GAO (analysis).
Appendix II: Overview of Current Personal Property Program and Pilot
Programs
Appendix II: Overview of Current Personal Property Program and Pilot
Programs Page 33 GAO- 03- 367 Defense Transportation Table 2 compares the
quality of service- related features. Some of the comparable features
included emphasizing performance over cost in
selecting transportation providers and prescreening of transportation
providers.
Table 2: Quality of Service- Related Features of the Current Personal
Property Program and Pilot Programs Program features Current personal
property program Military Traffic
Management Command*s Reengineered Personal Property Program
Department of Defense*s Full Service Moving Project
Navy*s Service Member Arranged Move Pilot Program Counseling and arranging
shipment services
Single relocation coordinator No No Yes Yes Program management and
counseling services provider
Installation personal property shipping office Installation personal
property shipping office Private- sector move managers Installation
personal
property shipping office
Screening process for transportation providers
Program emphasis in selecting transportation providers
Lowest cost Performance Performance Performance Prescreening of
transportation providers Very limited Yes Yes Yes Customer satisfaction
surveys conducted and method
Surveys are not conducted Survey company calls member Survey company calls
member Service member returns survey via mail
Visibility of shipments during relocation process
Methods used to increase service members* visibility of shipments during
move process
None Toll- free number to transportation provider Toll- free number to
move
manager and transportation provider
Toll- free number to move coordinator and transportation provider,
and member has pager Sources: DOD (data); GAO (analysis).
Appendix II: Overview of Current Personal Property Program and Pilot
Programs Page 34 GAO- 03- 367 Defense Transportation Table 3 compares data
reliability- related features. As noted, only one of the pilot programs
had a data management system that provided reliable information to track
individual shipments in transit and provide overall
data on shipments and their associated costs.
Table 3: Data Reliability- Related Features of the Current Personal
Property Program and Pilot Programs Program features Current personal
property program Military Traffic
Management Command*s Reengineered Personal Property Program
Department of Defense*s Full Service Moving Project
Navy*s Service Member Arranged Move Pilot Program Availability and
reliability of data on household goods shipments
Reliable data management system to track individual shipments in transit
and
to provide overall data on shipments and associated costs
No (current system is not designed to track shipments nor provide reliable
shipment and cost data)
Yes Not determined (developed but not fully operational* data management
system needed additional refinement)
Not determined (developed a database by end of pilot program but was not
fully implemented or evaluated)
Sources: DOD (data); GAO (analysis).
Additional information on the current program and on each pilot program
and its unique features follows.
The current DOD personal property program, valued at over $1.7 billion
annually, moves more than 600,000 shipments each year for military
personnel and their families from the military services, Defense agencies,
and the Coast Guard. DOD is the moving industry*s single largest customer.
Managed centrally by the headquarters office of the Military Traffic
Management Command and administered locally by about 200 military and DOD
transportation offices around the world, this program relies on over 1,200
domestic commercial carriers and more than 150 forwarders for
international traffic to provide moving and storage services.
When loss and damage occur, military personnel can submit claims to their
respective military service claims office. Based on depreciated values,
the reimbursement rate is $1.25 per pound multiplied by the shipment
weight, with a maximum amount of $40,000 per move. Military personnel have
up to 2 years after receiving their shipments to file claims but must
submit notice of loss and damage within 70 days of delivery. The Current
Program
Loss and Damage Claims
Appendix II: Overview of Current Personal Property Program and Pilot
Programs Page 35 GAO- 03- 367 Defense Transportation current program does
not have a specified time period in which the claims are to be settled.
The current program provides counseling services and arranges the
shipment and storage of household goods and unaccompanied baggage through
government representatives, who are available to assist military personnel
and their families at the origin and destination points of their moves.
The current program does not have a real- time tracking system for
shipments nor does it provide a single point of contact to manage the
entire moving process; therefore, military personnel may interact with
several people at the origin and destination offices during their
relocation. The current system is not designed to select transportation
providers on
the basis of quality service; rather, transportation providers offering a
minimally acceptable level of quality are generally selected based on the
lowest rates. The program uses the Total Quality Assurance Program to
develop quality scores for each transportation provider. Each local
military installation distributes its traffic using a traffic distribution
roster. Transportation providers are placed on the rosters for each
channel (origin and destination areas) by order of rate level and quality
score. Transportation providers who participate in the domestic part of
the current program submit their rates as a percentage of the government
tariff, which is nearly 20 years old. The providers who participate in the
international part of the current program submit single factor or fixed
rates per hundredweight of the shipments.
The current program does not use customer satisfaction surveys as a means
to evaluate transportation provider performance. To remain in the program,
a provider must maintain a minimally acceptable level of quality* a 90
percent score. Three factors are measured: on- time pickup, on- time
delivery, and reported loss and damage to determine if points should be
deducted from transportation providers and allocation of shipments should
be reduced or terminated.
The current program does not provide service members with real- time
visibility of shipments during the relocation process. Counseling and
Arranging
Shipment Services Screening and Shipment Distribution Process for
Transportation Providers
Visibility of Shipments during the Relocation Process
Appendix II: Overview of Current Personal Property Program and Pilot
Programs Page 36 GAO- 03- 367 Defense Transportation Personnel at origin
and destination personal property shipping offices enter information on
shipments to their respective Transportation
Operational Personal Property Standard Systems. However, data in these
individual systems does not include all shipments that occur during the
year, and the systems are not accessible to all parties involved in the
relocation process. Destination personal property shipment offices are
forwarded information on shipments via the current system; however,
payment data on these shipments is maintained in a separate system.
In addition to not providing information on all aspects of individual
shipments, the current program*s data management system does not provide
DOD and the military services information about the types of shipments and
related costs for planning and budgeting purposes. The following are
examples of the current system*s limitations:
the format of the system makes compiling data from multiple sites
difficult; not all data is captured promptly; and not all data and
costs are captured/ updated in the system.
Under the current program, the military services reimburse carriers and
forwarders for shipment- related costs from military personnel accounts.
Personal property shipment office expenses and claims filed with the
government are funded from the military services* operations and
maintenance accounts.
Sponsored by the Military Traffic Management Command, the Reengineered
Personal Property Program included outbound shipments for military and
Coast Guard personnel departing from installations located in North
Carolina, South Carolina, and Florida (excluding Tyndall Air Force Base).
The pilot program ran concurrently with the existing
program at these installations. The pilot program*s goal was to include 50
percent of eligible moves from the above installations to continental
United States and European locations. The remaining shipments were to be
moved under the existing program. The Reengineered Personal Property
Program was initiated in January 1999 and operated for 12 months before
data was submitted to the Transportation Command for evaluation.
Reimbursement for loss and damage claims was increased from depreciated
value to full replacement value, and the dollar amounts per Availability
and Reliability
of Data on Household Goods Shipments
Funding Sources Military Traffic Management Command*s Reengineered
Personal Property Program
Loss and Damage Claims
Appendix II: Overview of Current Personal Property Program and Pilot
Programs Page 37 GAO- 03- 367 Defense Transportation move increased from
$40,000 under the current program to $63,000 under the Reengineered
Personal Property Program. Additionally, the pilot
program provided direct claims settlement between military personnel and
their transportation providers and a requirement that transportation
providers settle claims within 60 days of receiving claims forms from
military personnel.
Like the current program, the Reengineered Personal Property Program
relied on personnel in the personal property shipping offices to provide
counseling services and arrange for shipment and storage of household
goods and unaccompanied baggage. A central contact point in these offices
was not designated to manage the entire moving process; therefore,
military personnel may have interacted with several people at the origin
and destination offices during the relocation process. However, to improve
customer service, the program*s Pilot Transportation Operational Personal
Property Standard System provided real- time worldwide tracing capability.
Greater emphasis was placed on performance in awarding shipments to
transportation providers. Evaluation of financial status, elimination of
high- risk companies, and consideration of providers* past performance,
rather than lowest bid, played the dominant role in selecting initial
transportation providers to participate in this pilot program.
Transportation providers who participated in the pilot program submitted
their bids for various origin and destination routes as a discount from
the commercial tariff. Prices were fixed for a year, with no provision for
rate increases during the contract period. Awards were made only to
transportation providers whose offers conformed to the solicitation and
represented the best overall value to the government.
The Military Traffic Management Command evaluated company performance
quarterly and compliance with terms and conditions of the contracts
annually. Subsequent performance reviews were conducted based on customer
satisfaction surveys and claims data. After transportation providers
received their minimum guarantee of business for the year ($ 25, 000),
future awards were offered to the best performers. Feedback was provided
monthly to transportation providers, and those that became poor performers
were no longer offered household goods and unaccompanied baggage
shipments. Counseling and Arranging
Shipment Services Screening and Shipment Distribution Process for
Transportation Providers
Appendix II: Overview of Current Personal Property Program and Pilot
Programs Page 38 GAO- 03- 367 Defense Transportation The Reengineered
Personal Property Program provided the transportation provider*s toll-
free number to military personnel to enhance visibility over their
shipments throughout the relocation process.
The Reengineered Personal Property Program implemented its central, Web-
based Pilot Transportation Operational Personal Property Standard System
in part to address problems associated with visibility of and
availability of information on shipments during the relocation process.
The pilot program*s data management system provided real- time access to
both shipment and payment records. Access to the various modules of the
system was granted to personal property shipment office personal at origin
and destination locations, transportation providers, invoice certifying
officers, prepayment auditors, military service headquarters, and military
service claims offices and finance centers, based on each party*s need for
the information.
The system*s design allowed for entry of current address and telephone
numbers of military personnel to improve the process of delivering
household goods to a new residence. Data reliability was enhanced under
the Reengineered Personal Property Program, but one problem noted
during the evaluation was the need for military personnel to ensure that
their contact information (phone number and address) was current during
the relocation process. This had an effect on deliveries of household
goods and the quality of life survey contractor*s ability to reach
military personnel to ascertain their opinions about their relocation
experience.
In addition to providing information on all aspects of individual
shipments, the Reengineered Personal Property Program*s data management
system demonstrated the potential to provide DOD and the military services
with information about the types of shipments and related costs managed
annually for planning and budgeting purposes.
The Reengineered Personal Property Program achieved stronger
transportation provider commitment with long- term contracts, and it used
contractor support to conduct quality of life surveys with military
personnel moving under the pilot program and to perform audits of each
invoice submitted by transportation providers.
Like the current program, the military services reimbursed carriers and
forwarders for shipment- related costs from their military personnel
accounts. Transportation office expenses and any claims filed with the
Visibility of Shipments
during the Relocation Process
Availability and Reliability of Data on Household Goods Shipments
Other Reengineered Personal Property Program Features Funding Sources
Appendix II: Overview of Current Personal Property Program and Pilot
Programs Page 39 GAO- 03- 367 Defense Transportation government were
funded from the services* operations and maintenance accounts.
Sponsored by the Office of the Assistant Deputy Under Secretary of Defense
(Transportation Policy), the Full Service Moving Project included outbound
shipments for military and Coast Guard personnel and DOD
civilian departing from locations in the National Capital Region, Georgia
(excluding Robins Air Force Base), and Minot Air Force Base, North Dakota.
The pilot program*s goal was to include 90 percent of the moves from these
locations to continental United States and to European and Asian- Pacific
locations. The remaining shipments were to be moved under the current
program. The Full Service Moving Project began in January 2001 and
continued until its early termination in September 2001. Due to continuing
delays in implementing this pilot program and DOD*s decision to terminate
the pilot program in September 2001, the Full Service Moving Project had
limited operational experience before submitting data to the
Transportation Command.
Reimbursement for loss and damage claims was increased from depreciated
value to full replacement value, with the dollar amounts increasing from
$40,000 per move under the current program to $75, 000 per move under the
Full Service Moving Project. Additionally, the pilot program provided for
direct claims settlement between military personnel and their
transportation providers and a requirement that the responsible party
(transportation providers or move managers) settle claims within 45 days
of receiving claim forms from military personnel.
Unlike the current program and other pilot programs, the Full Service
Moving Project tested the use of private- sector relocation companies
(move managers) for outsourcing traditional transportation services
(counseling and arranging for the shipment and storage of household goods
and unaccompanied baggage) performed by origin and destination personal
property shipping offices. The pilot program*s goal was to provide a
single point of contact (move manager) for military personnel and
transportation providers to contact throughout the relocation process.
The Full Service Moving Project made major changes to the existing
transportation provider approval, rate solicitation, and traffic
distribution processes. The pilot program emphasized best value and placed
more weight on performance (70 percent) than cost (30 percent) in
determining The Department of
Defense*s Full Service Moving Project
Loss and Damage Claims Counseling and Arranging Shipment Services
Screening and Shipment Distribution Process for Transportation Providers
Appendix II: Overview of Current Personal Property Program and Pilot
Programs Page 40 GAO- 03- 367 Defense Transportation which transportation
providers would be awarded shipments. The pilot program contracted with a
financial services company to conduct financial
and performance assessments of transportation providers and move manager
companies that wanted to participate in the pilot program. For approved
transportation providers, rates were established for a 1- year cycle. The
providers submitted their rates as a discount from the commercial tariff
for domestic shipments and negotiated single rate factors for the overseas
locations. Approved move managers were awarded 2- year contracts with 1-
year options. The move management companies competitively bid their fees
as flat rates, depending on whether they were responsible for claims
settlement or the transportation provider carried this liability. Also,
different fees were established for domestic and international shipments.
The Full Service Moving Project used survey data from all personnel
participating in the pilot program to determine future percentages of
shipments that would be allocated to the transportation providers. The
pilot program also planned to use survey data on move manager performance
to determine future participation in the pilot program and incentive
payments. The Full Service Moving Project*s Web- based Best Value
Distribution Database maintained the transportation providers* quality and
cost scores based on survey information and costs associated with prior
shipments. Move managers used this data to assign future shipments.
However, in some instances (i. e., for group moves, when meeting small
business requirements, when there was a lack of transportation provider
capacity to handle shipments offered, for multiple shipments to a single
transportation provider, and for international shipments to areas without
an established rate), move managers were told to deviate from the
information provided by the data management system.
One of the goals of incorporating move managers into the relocation
process was to provide real- time information to military personnel and to
transportation providers regarding the status of household goods
shipments. The move managers, unlike the current program*s personal
property shipping office personnel, were responsible for the entire
relocation process from the point of origin in establishing entitlements,
arranging for transportation providers, and handling other
personnelrelated issues, to the destination in overseeing deliveries,
approving storage, and either settling claims or assisting military
personnel with issues involving settling claims with the transportation
providers if the liability fell with the providers. Working with both
military personnel and transportation providers, the move managers used
contact information to keep military personnel informed of their
shipments* status and to Visibility of Shipments
during the Relocation Process
Appendix II: Overview of Current Personal Property Program and Pilot
Programs Page 41 GAO- 03- 367 Defense Transportation coordinate the
delivery of the shipments at the destination. Additionally, as part of the
pilot program, all participants were provided a toll- free
number to maintain visibility over their shipments throughout the process.
In addition to move managers, the Full Service Moving Project*s Webbased
Best Value Distribution Database was implemented to address problems
associated with visibility of shipments during the relocation process. The
pilot program*s data management system had access to both shipment and
payment records via interface with US Bank*s PowerTrack and the move
managers* systems. Access to the pilot program*s data management system
was granted to move managers, invoice certifying officers, military
service headquarters, and military service claims offices and finance
centers, based on each party*s requirements.
Move managers were responsible for keeping the status of the shipments
current in the pilot program*s data management system. However, the move
managers did not always update this information in the system.
Further, the ability of the move manager to contact the service member was
directly affected by the information provided by the member.
The Full Service Moving Project*s Web- based Best Value Distribution
Database was anticipated to provide DOD and the military services
information on the types of shipments and related costs managed annually
for planning and budgeting purposes. Unlike the Reengineered Personal
Property Program where various parties in the relocation process entered
data into that pilot program*s data management system, the majority of the
data in the Full Service Moving Project*s data management system was
predicated on the move managers gathering and entering the information.
The Full Service Moving Project achieved stronger transportation provider
commitment with long- term contracts and faster payment of invoices; it
offered binding cost estimates for shipments; it used contractor support
to conduct quality of life surveys with military personnel moving under
the pilot program and to perform audits of each invoice submitted by the
transportation providers; and it offered optional relocation referral
assistance for activities such as the purchase and sale of service
members* residences.
Move managers were required to perform prepayment audits and business
rules were established for an automatic payment method. Payment
methodology was predicated on the move manager entering the expected
invoice into PowerTrack and the transportation provider submitting a
Availability and Reliability
of Data on Household Goods Shipments
Other Full Service Moving Project Features
Appendix II: Overview of Current Personal Property Program and Pilot
Programs Page 42 GAO- 03- 367 Defense Transportation notice of delivery
and invoice. Payment timeliness was also driven by the timeliness of
documentation submitted by the transportation providers.
On some invoices, the contracting representative had to review and certify
payment in PowerTrack. This occurred when the match showed a difference of
more than $1.00.
For this pilot program, the military services reimbursed carriers and
forwarders for shipment- related costs from military personnel accounts.
These accounts were also used to fund move manager expenses. Any claims
that might have been filed with the government would have been funded from
the military services* operations and maintenance accounts.
Sponsored by the Navy, the Service Member Arranged Move Pilot Program
included only domestic outbound intrastate and interstate shipments for
Navy personnel moving from its installations located at Puget Sound,
Washington; San Diego, California; Norfolk, Virginia; New London,
Connecticut; and Whidbey Island, Washington. One of this program*s
objectives was to offer Navy military personnel a set of moving choices to
meet their specific needs. This pilot program was one of three choices
offered. Military personnel moving from the above locations could choose
to move under the current personal property program, move their own
household goods, or participate in the pilot program. The pilot program
was initiated in April 1997 and began operations in January 1998. Because
the Navy decided not to scope the Service Member Arranged Move Pilot
Program comparable to other pilot programs (i. e., operational at multiple
military services) and the pilot program did not provide data as outlined
by the Transportation Command*s evaluation plan, its inclusion in the
Transportation Command*s evaluation was limited to a qualitative
assessment.
Reimbursement for loss and damage claims was increased from depreciated
value to full replacement value, with the dollar amounts per move
increasing from $40,000 under the current program to $72,000 under the
Service Member Arranged Move Pilot Program. Additionally, the pilot
program provided direct claims settlement between military personnel and
their transportation providers and a requirement that transportation
providers settle claims within 60 days of receiving claims forms from
military personnel. Funding Sources
Navy*s Service Member Arranged Move Pilot Program
Loss and Damage Claims
Appendix II: Overview of Current Personal Property Program and Pilot
Programs Page 43 GAO- 03- 367 Defense Transportation Like the current
program, the Service Member Arranged Move Pilot Program also relied on
personnel in the personal property shipping offices
to provide counseling services and arrange shipment and storage of
household goods and unaccompanied baggage. However, the shipping office
personnel at the origin installations participating in this pilot program
served as the single point of contact coordinating the service members*
moves and remained available throughout the move to handle all issues,
including claims. Unlike those participating in the current program and
other pilot programs, service members participating in this pilot program
identified the transportation provider they desired to handle their
household goods shipments after they completed their reviews of
participating providers* vendor quality books (containing provider
information and marketing materials) and of surveys completed by previous
pilot program participants. The personal property office coordinator
assigned to the service member had to concur with the member*s request,
and the coordinator made actual arrangements with the carrier.
Staff in the program management office and personal property shipping
offices participating in the Service Member Arranged Move Pilot Program
initially screened transportation providers that wished to participate in
the
pilot program based on providers* performance rather than low cost.
Letters of agreement were adopted to streamline the contracting process
and improve the quality of the move for Navy personnel. According to pilot
program officials, these letters of agreement provided commercial best
practices and enabled lessons learned from prior pilot program efforts and
industry to be incorporated into the Navy pilot program. Actual contract
awards were made on a case- by- case basis based on the best value
decision for each move. Transportation providers used commercial tariffs
in developing their bids for each move.
Transportation providers approved to participate in the pilot program
submitted their bids for various origin and destination channels using
commercial tariffs. Bids were rejected if they did not fall within
acceptable percentage discounts. Feedback was provided monthly to
transportation providers, and those that became poor performers were no
longer offered household goods and unaccompanied baggage shipments.
Subsequently, service members who were planning their upcoming moves
relied on
information contained in a nine- question survey that other service
members had completed after their moves and claims process ended. Service
members who volunteered to participate in this pilot program had to
manually review carrier books, which included documents provided by the
carriers and prior surveys completed by service members who had Counseling
and Arranging
Shipment Services Screening and Shipment Distribution Process for
Transportation Providers
Appendix II: Overview of Current Personal Property Program and Pilot
Programs Page 44 GAO- 03- 367 Defense Transportation been moved by the
carriers. According to pilot program officials, six carriers were
terminated or canceled from the pilot program* four for
providing poor service and two for price gouging. The Service Member
Arranged Move Pilot Program relied upon the shipping office personnel, who
served as the single point of contact coordinating the service members*
moves, to maintain visibility of shipments during the relocation process.
In addition, the pilot program provided both the personal property
shipping office*s and transportation provider*s toll- free numbers, as
well as a pager to service members to enhance the members* visibility of
their shipments during the relocation process.
The Service Member Arranged Move Pilot Program did not initially develop
an alternative data management system to capture data on shipments and
payment records. By the end of the pilot program, the Navy had developed a
database to capture shipment data; however, the system was not fully
implemented or evaluated.
Unlike the other pilot programs, the Navy pilot program used local
personal property program personnel rather than third parties to review
all invoices for payment. Navy personnel who participated in the pilot
program completed their own surveys, mailing the paper forms to their
respective personal property program offices.
The Service Member Arranged Move Pilot Program was designed to offer all
shipments to small businesses, to provide direct claims settlements
between Navy personnel and the transportation providers, to make faster
payments to transportation providers through government purchase cards,
and to establish a stronger commitment from transportation providers by
offering long- term contracts.
Like the current program, the Navy reimbursed carriers and forwarders for
shipment- related costs from its military personnel account. Personal
property shipment office expenses were funded from the Navy*s operations
and maintenance account. While information on claims filed with the
government was not provided, under this pilot program such expenses would
also be funded from the operations and maintenance account. Visibility of
Shipments
during Relocation Process Availability and Reliability of Data on
Household Goods Shipments
Other Service Member Arranged Move Pilot Program Features Funding Sources
Appendix III: Comments from the Department of Defense
Page 45 GAO- 03- 367 Defense Transportation Appendix III: Comments from
the Department of Defense
Appendix III: Comments from the Department of Defense Page 46 GAO- 03- 367
Defense Transportation
Appendix III: Comments from the Department of Defense Page 47 GAO- 03- 367
Defense Transportation
Appendix III: Comments from the Department of Defense Page 48 GAO- 03- 367
Defense Transportation
Related GAO Products Page 49 GAO- 03- 367 Defense Transportation Defense
Transportation: Final Evaluation Plan Is Needed to Assess Alternatives to
the Current Personal Property Program. GAO/ NSIAD- 00- 217R. Washington,
D. C.:
September 27, 2000.
Defense Transportation: The Army*s Hunter Pilot Project Is Inconclusive
but Provides Lessons Learned. GAO/ NSIAD- 99- 129. Washington, D. C.: June
23, 1999.
Defense Transportation: Plan Needed for Evaluating the Navy Personal
Property Pilot. GAO/ NSIAD- 99- 138. Washington, D. C.: June 23, 1999.
Defense Transportation: Efforts to Improve DOD*s Personal Property
Program. GAO/ T- NSIAD- 99- 106. Washington, D. C.: March 18, 1999.
Defense Transportation: The Army*s Hunter Pilot Project to Outsource
Relocation Services. GAO/ NSIAD- 98- 149. Washington, D. C.: June 10,
1998.
Defense Transportation: Reengineering the DOD Personal Property Program.
GAO/ NSIAD- 97- 49. Washington, D. C.: November 27, 1996. Related GAO
Products
(350161)
The General Accounting Office, the audit, evaluation and investigative arm
of Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO*s commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO*s Web site (www. gao. gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as *Today*s Reports,* on its
Web site daily. The list contains links to the full- text document files.
To have GAO e- mail
this list to you every afternoon, go to www. gao. gov and select
*Subscribe to daily E- mail alert for newly released products* under the
GAO Reports heading.
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to: U. S. General Accounting Office 441 G Street NW, Room LM
Washington, D. C. 20548 To order by Phone: Voice: (202) 512- 6000
TDD: (202) 512- 2537 Fax: (202) 512- 6061
Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm E- mail:
fraudnet@ gao. gov Automated answering system: (800) 424- 5454 or (202)
512- 7470 Jeff Nelligan, managing director, NelliganJ@ gao. gov (202) 512-
4800
U. S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.
C. 20548 GAO*s Mission Obtaining Copies of
GAO Reports and Testimony
Order by Mail or Phone To Report Fraud, Waste, and Abuse in Federal
Programs Public Affairs
*** End of document. ***