Defense Transportation: Efforts to Improve DOD's Personal Property
Program (Testimony, 03/18/99, GAO/T-NSIAD-99-106).
The Defense Department (DOD) has since 1994 been trying to reengineer
its program for relocating military personnel and their families. DOD's
goal has been to simplify processes, control program costs, ensure
quality service by adopting commercial business practices, and relieve
carriers of terms and conditions unique to the military. The U.S.
Transportation Command is responsible for evaluating the pilot programs
to determine which of them could provide better results for DOD. This
testimony discusses the Army's Hunter pilot and DOD's plans to evaluate
other ongoing and planned pilot programs. In assessing the Army's
evaluation of the Hunter pilot, GAO focuses on the evaluation
methodology and the Army Audit Agency's validation of the test and
baseline data.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: T-NSIAD-99-106
TITLE: Defense Transportation: Efforts to Improve DOD's Personal
Property Program
DATE: 03/18/99
SUBJECT: Department of Defense contractors
Household goods
Program evaluation
Military personnel
Personal property
Transportation costs
Evaluation methods
Customer service
Employee transfers
Surveys
IDENTIFIER: Army Hunter Pilot Program
DOD Personal Property Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. This text was extracted from a PDF file. **
** Delineations within the text indicating chapter titles, **
** headings, and bullets have not been preserved, and in some **
** cases heading text has been incorrectly merged into **
** body text in the adjacent column. Graphic images have **
** not been reproduced, but figure captions are included. **
** Tables are included, but column deliniations have not been **
** preserved. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
cover.book GAO United States General Accounting Office
Testimony Before the Subcommittee on Military Readiness, Committee
on Armed Services, House of Representatives
For Release Expected at 11: 00 a. m., EST Thursday, March 18, 1999
DEFENSE TRANSPORTATION
Efforts to Improve DOD's Personal Property Program
Statement by David R. Warren, Director, Defense Management Issues,
National Security and International Affairs Division
GAO/T-NSIAD-99-106
GAO/T-NSIAD-99-106
Page 1 GAO/T-NSIAD-99-106
Mr. Chairman and Members of the Subcommittee: We are pleased to be
here today to discuss the Department of Defense's (DOD) efforts to
improve its program for relocating military personnel and their
families. This has been a long, complicated process. Since 1994,
DOD has been engaged in initiatives to reengineer the personal
property program to simplify current processes, control program
costs, ensure quality of service by adopting commercial business
processes characteristic of world- class businesses, and relieve
carriers of
DOD- unique terms and conditions. The U. S. Transportation Command
(USTRANSCOM) is tasked with evaluating the pilots to determine
which pilots, or portions thereof, could provide better long- term
results for DOD. 1 The statement of managers in the conference
report on the DOD
Appropriations Act for 1997 directed us to validate the results
and savings achieved from any personal property program before DOD
proposes further expansion of such programs. 2 My testimony today
will focus on our ongoing work related to the Army's Hunter pilot
and DOD's plans to evaluate other ongoing and planned pilot
programs. In assessing the Army's evaluation of the Hunter Pilot,
we focused on (1) the evaluation
methodology and (2) the Army Audit Agency's (AAA) validation of
the test and baseline data. Results in Brief We agree that DOD
needs to reengineer its personal property moving
process to improve the poor quality of service military members
are receiving. We support the use of pilots as a means to test new
concepts. Although this process has been ongoing since the early
1990s, DOD is not yet in a position to determine what changes are
needed.
We were unable to validate the reported results of the Army's
evaluation of the Hunter pilot because of weaknesses in the
evaluation methodology and the data. However, the lessons learned
from the pilot do provide information that should be useful to DOD
as it assesses and conducts its pilot efforts. Further, the Hunter
pilot provides military personnel with
1 The mission of USTRANSCOM, which is DOD's single manager of all
defense transportation services, is to provide global air, land,
and sea transportation to meet national security needs, both in
time of peace and time of war. USTRANSCOM executes its mission
through three component commands: Military Traffic Management
Command (MTMC) for land transportation and port operations, the
Military Sealift Command for sea transport, and the Air Mobility
Command for air transport.
2 House Report 104- 863, Sept. 28, 1996.
Page 2 GAO/T-NSIAD-99-106
services that were not previously available during the moving
process, including personal move counseling and coordination, full
replacement value for lost or damaged household goods, and
visibility of the shipment throughout the move. Key pilot results
that we were unable to validate included the level of customer
satisfaction and costs. Although customer surveys were conducted,
the results were inconclusive because of the methods used. While
the Army indicated that the estimated pilot cost was higher than
the baseline cost, we could not validate the extent to which the
pilot cost exceeded the baseline cost. Further, we could not
calculate the baseline and pilot costs due to weaknesses in the
Army's methodology and data reliability. As for the participation
of small businesses, we confirmed that 33 percent of the pilot
shipments were awarded by Cendant Mobility to small business
carriers and agents.
DOD has three pilot programs underway to improve its current
personal property program and is proposing a fourth pilot. The
general plan is to evaluate the results of these pilots and use
that information to develop a redesigned Department- wide
relocation program. To achieve this objective, it is imperative
that DOD develop a well thought out strategy with clear time lines
for testing each of the approaches, and more importantly, an
evaluation methodology that will produce credible and accurate
information to be used in making a final decision. However, plans
for accomplishing these tasks and milestones for implementing a
new
process have not yet been finalized. Background DOD has long been
concerned about the quality service members receive
from its nearly $3 billion annual program to transport, store, and
manage the household goods and unaccompanied baggage of its
personnel. Past problems include poor service from its movers,
excessive incidence of loss or damage to service members'
property, and high claims costs to the government. All of these
problems contribute to a poor quality of service for persons using
the system.
MTMC's Reengineering Efforts Began in 1994 DOD first proposed
reengineering its personal property program 5 years
ago. On June 21, 1994, the Deputy Commander in Chief, USTRANSCOM,
directed MTMC, the Army component of USTRANSCOM and program
manager for DOD's Personal Property Shipment and Storage Program,
to reengineer the personal property program. On June 15, 1995, the
House
Page 3 GAO/T-NSIAD-99-106
Committee on National Security, concurring that DOD must pursue a
higher level of service, directed that DOD undertake a pilot
program to implement commercial business practices and standards
of service. 3 MTMC planned to award contracts for the new program
pursuant to the
competitive acquisition system provisions (chapter 137 of title 10
of the U. S. Code and the primary implementing regulation
contained in the Federal Acquisition Regulation). Expressing
congressional concerns about
the impact that the competition system and any nonstandard
commercial business practice requirements might have on small
businesses, the statement of managers accompanying the 1997 DOD
Authorization Act directed that DOD report on the impact of the
pilot program on small business. After reviewing the defense
reports on small business impact, the House Committee on National
Security was still concerned that MTMC's pilot program did not
satisfactorily address issues raised by the small moving companies
and directed that the Secretary of Defense establish a working
group of military and industry representatives to develop an
alternative pilot program.
MTMC Pilot Commenced in 1999
Although the working group reached a consensus on many issues,
including a set of program goals, it could not reach agreement on
the approach to take for the pilot test. Consequently, the two
sides presented separate proposals. In November 1996, we reported
that in our assessment MTMC's proposal met the goals of
reengineering the personal property
program to a greater extent than the industry plan. 4 The MTMC
pilot program implementation, delayed by numerous bid protests,
finally commenced in January 1999. 5 The pilot reengineers the
existing program for 50 percent of the moves originating in three
states: North Carolina, South Carolina, and Florida. The pilot
will run concurrently with the existing MTMC- managed program at
installations in the three states and will involve approximately
18, 500 annual moves. The
3 The House Committee on National Security is now called the House
Armed Services Committee. 4 See Defense Transportation:
Reengineering the DOD Personal Property Program (GAO/NSIAD-97-49,
Nov. 27, 1996) for more detailed information on the MTMC pilot
effort.
5 Solicitation disputes in the following decisions: Aalco
Forwarding, Inc., et al. (B-277241. 8, B-277241.9, Oct. 21, 1997);
Aalco Forwarding, Inc. et al. (B-277241.12, B-277241.13, Dec.
29,1997); Aalco Forwarding, Inc. et al. (B-277241. 15, Mar. 11,
1998); Aalco Forwarding, Inc. et al. (B-277241. 16, Mar. 11,
1998); Aalco Forwarding, Inc., et al. (B-277241, 277241.14, June
8, 1998); and Aalco Forwarding, Inc., et al. (B-277241, B-
277241.20, B-277241. 21, July 1, 1998).
Page 4 GAO/T-NSIAD-99-106
key features of the pilot program include selecting carriers based
on service member satisfaction and past performance rather than
simply price; achieving stronger carrier commitment with long-
term contracts; and offering full replacement value protection and
direct claims settlement to users. MTMC's pilot will run for a 3-
year test period (1 year with two 1- year
option periods), which will end in September 2002. Hunter Pilot
Effort Began in 1996
Separately from the MTMC pilot program, the Army decided to
determine whether the commercial business practice of outsourcing
could alleviate known problem areas. The pilot was initiated in
February 1996 as a quality- of- life effort to improve the
relocation process and to test commercial business practices in a
military environment at Hunter Army
Airfield, Savannah, Georgia. On January 31, 1997, the Army
selected PHH Relocation, a relocation/ move management company
(today named Cendant Mobility, located in Bethesda, Maryland), as
the contractor for the pilot. In July 1997, Cendant Mobility began
relocation operations at Hunter Army Airfield, consisting of
point- to- point move management, personal move counseling and
coordination, assistance in buying/ selling a residence, 24- hour
in- transit visibility of the shipment, direct claims settlement
with
the service member, full replacement value, and a single point of
contact for the member, that were not previously available under
the existing system. The contract will end on September 30, 1999.
Navy Pilot Effort Began in 1997 The Navy initiated a separate
pilot effort in 1997 to test the option of allowing service
members to select their carrier, giving them more control over the
relocation process to meet their specific needs. The pilot, which
commenced in January 1998, is not intended to replace the existing
MTMC- managed program and affects a very small number of military
personnel. Presently, the option is offered for shipments
originating in the Puget Sound, Washington; San Diego, California;
Norfolk, Virginia; and Groton, Connecticut areas. Participation is
limited to Navy military members with permanent change of station
orders from the participating
sites. Eligible participants must limit shipments to at least
3,000 pounds and to costs between $2, 500 and $25,000 (and it
excludes shipments from nontemporary storage or warehouse). The
service member can select a mover from a list of carriers that are
self- certified as small businesses and approved by MTMC. As of
March 1999, 169 Navy service members had selected this option.
This pilot does not have a specific end date.
Page 5 GAO/T-NSIAD-99-106
DOD Pilot Effort Proposed in 1999
On February 12, 1999, DOD stated that it intended to begin a
fourth test, called the Full Service Moving Project, with certain
modifications based on lessons learned from the Army Hunter test.
The test will involve a larger volume of moves and will include
the National Capital Region (17,000
annual moves), Georgia (26,000 annual moves), and North Dakota
(2,000 annual moves). Like the Army pilot, it is intended to
replace the existing MTMC- managed program by using a contractor
or contractors to provide both transportation and move management
services. Presently, there is no
official start date for this project. USTRANSCOM to Evaluate All
Pilot Programs
To oversee all personal property pilots, the Deputy Under
Secretary of Defense (Logistics) has tasked USTRANSCOM to ensure
consistency in evaluation criteria and assessment. USTRANSCOM
plans to evaluate the pilots to determine which pilots or tests,
or portions thereof, could provide better long- term results. It
will then recommend the follow- on course of action and time lines
for implementation throughout DOD. Presently, USTRANSCOM is
working with the services to develop a common set of
data measures to evaluate the pilot projects' results. Results of
Army Hunter Pilot Are Inconclusive
Because of the weaknesses in the Army's evaluation methodology and
data, we were unable to validate all reported results of the
Hunter pilot program. Moreover, the Army's evaluation plan
methodology was not an effective tool for collecting and analyzing
the pilot results. However, the lessons learned from the pilot
program do provide useful information to DOD as it conducts and
assesses its pilot efforts. Also, the pilot program is providing
services and benefits that were not previously available,
including point- to- point move management, personal move
counseling and coordination, direct claims settlement, assistance
in buying/ selling a residence, and visibility of the shipment
throughout the move.
The Army's Assessment and Evaluation Approach
Through the first 12 months of operations, the Army reported that
the Hunter pilot was successful in demonstrating that commercial
practices can be applied to the military relocation process. That
is, Army officials stated that they were able to contract with a
relocation company to provide services similar to those available
in the private sector. To evaluate the pilot, in January 1997, the
Army began developing a test evaluation plan that defined roles
and responsibilities, test factors, and the process for capturing
data related to each test factor, and stated how test data would
Page 6 GAO/T-NSIAD-99-106
be compared to that from the existing program. The plan stated
that AAA would be responsible for validating the baseline and test
data and for providing the Army with a report on the results of
the test. AAA conducted a review and its findings are incorporated
in our evaluation. We reported in June 1998 that the Army needed
to further define the various factors and
measurements. 6 The Army provided more clarity and a revised
methodology in the evaluation plan dated August 12, 1998. This
represented the third such revision, which was completed more than
a year after the pilot had begun. The modified plan indicated that
the pilot results would be reviewed on the basis of three factors:
quality of life, cost, and impact on small business and would use
a 5- point scoring system. Quality of life was the key element the
Army used in its evaluation, consisting of 1 point each for its
three sub- factors customer satisfaction, average claims
settlement time, and percentage of direct deliveries along
with 1 point each for the other two factors total cost and impact
on small business, for a total of 5 points. In its October 1998
evaluation report and February and March 1999
supplements, the Army reported to us that the pilot project earned
a passing score on each of the test factors that it measured over
the 12- month period ending June 30, 1998. The Army determined
that the Hunter pilot was a success because the factor points
exceeded the minimum threshold of 3.0 based on a 5. 0- point
scale. The Hunter pilot test earned 3. 75 points, as follows:
customer satisfaction improved by over 11 percent (0.25 point),
the initial offer to settle claims averaged 9 days (1 point),
100 percent of eligible shipments were delivered directly to the
service
member without storage in transit (1 point), Pilot project costs
exceeded baseline costs by 18.6 percent (0. 50 point),
and 33 percent of the shipments were awarded to small businesses
(1 point).
The Army stated that it also considered other factors, generally
related to transportation process improvements, and did not rely
on test scores alone to determine the pilot a success. These
benefits included simplifying the price structure and claims
process, providing pre- audit and certification services, and
minimizing government- unique requirements. Each service
6 Defense Transportation: The Army's Hunter Pilot Project to
Outsource Relocation Services (GAO/NSIAD-98-149, June 10, 1998).
Page 7 GAO/T-NSIAD-99-106
member worked with one personal move coordinator who was
accountable for direct delivery at destination, timely service,
and prompt resolution of any problems. This is in contrast to the
current process where a service member may need to contact four to
five different offices over the course of a single move. Neither
AAA nor we reviewed these benefits cited by the Army because they
were not included as measurable factors in the evaluation plan.
Evaluation Weaknesses Led to Inconclusive Results We identified a
number of shortfalls with the Army's evaluation
methodology and data. In several instances, the data provided to
support the measurements did not demonstrate achievement of the
reported results. Most importantly, we could not confirm that
customer satisfaction improved by 11. 5 percent. Further, we could
not confirm that all eligible shipments were delivered directly
without incurring in- transit storage
costs, that offer of claims settlement time averaged 9 days, and
that the pilot program cost 18.6 percent more than the baseline
cost. We reviewed the methodology and data for each factor quality
of life (consisting of three sub- factors: customer satisfaction,
claims settlement time, and direct deliveries); costs; and impact
on small business. Inconclusive Results for
Customer Satisfaction Because service members were surveyed
multiple times and the survey
methodology was flawed, the Army's reported results on customer
satisfaction were compromised. Therefore, we could not validate
that customer satisfaction improved by 11.5 percent. However, data
gathered in these surveys did provide indicators of customer
satisfaction as well as provide lessons learned for DOD,
USTRANSCOM, and the Army that may
be useful for conducting their pilot efforts. To measure customer
satisfaction, the Army said it would use its contractor
(Battelle's Pacific Northwest National Laboratory) to survey
members at Hunter and those making similar moves at Fort Stewart,
Georgia.
Responses from service members at both locations would be
compared. The Army told us that Battelle would conduct the survey,
but instead, the Army used USTRANSCOM's survey, conducted by
Electronic Data Systems (EDS), to evaluate and score the pilot
project. Initially, the Army informed
Page 8 GAO/T-NSIAD-99-106
us that it did not use Battelle's results because the draft report
was difficult to interpret and inconclusive. It subsequently
expressed concern over a disclaimer on the survey results. 7
Nonetheless, service members were surveyed multiple times,
possibly by
four different entities. EDS and Battelle (the survey
contractors), Cendant Mobility (which was required by its contract
to survey a sample of customers), and the carrier (for its own
quality control) each may have contacted the same service members
to query them on their move experience. As a result, the customer
satisfaction results were compromised and produced inconclusive
results. EDS reported that some service members refused to respond
to the survey and that it does not know how many of those who did
respond only gave perfunctory answers. In a caveat in its report,
Battelle stated that it was aware that some individuals were
interviewed multiple times. In addition, the contractors' survey
results varied significantly. While EDS
reported that 11.5 percent of respondents were more satisfied with
their current relocation experience (the pilot), Battelle reported
that only 3 percent of its respondents were more satisfied. 8 We
are uncertain of the specific reasons for these differences.
However, we do know that the
question asked by the contractors to determine customer
satisfaction was not identical, and that the number of respondents
in the two surveys varied significantly. This may have contributed
to the differences. EDS had a much lower response rate, 53
percent, than Battelle, which achieved an 89- percent response
from service members on which their results were calculated.
Further, EDS used a 75- percent significance level, while Battelle
calculated responses using a 95- percent significance level. The
latter is a standard used in social science research. 9 7 The Army
said it subsequently learned that the Department of Energy
requires such a disclaimer in all studies performed for government
entities and that the disclaimer had no relevance to the results
reported by Battelle. 8 Battelle also reported that the 3- percent
figure was statistically insignificant, meaning that the variance
in responses from the two groups (personnel at Hunter and at Fort
Stewart) was not great enough to conclude that their move
experiences differed significantly.
9 Significance level is the likelihood that a true population
value may be rejected by a statistical test. In the case of the
two surveys, for example, Battelle used a 95- percent and EDS used
a 75- percent probability that the differences in relocation
satisfaction levels were due to something other than random
occurrence. Battelle found no significant difference in
satisfaction levels, with a 5- percent chance of reaching an
incorrect conclusion. EDS did find a significant difference but
with a 25- percent chance of reaching an incorrect conclusion.
Page 9 GAO/T-NSIAD-99-106
Other data obtained by the surveys, but not measured by the Army,
provide indicators of customer satisfaction and highlights areas
that could be addressed in future evaluations. For example, the
Battelle survey found that 61 percent of the Hunter respondents
were more satisfied with their most recent (pilot) moving
experience than with their prior move. Eighty- seven percent of
these respondents were satisfied or very satisfied with the
carrier's responsiveness. Both surveys also indicated that service
members relocating within the continental United States were
generally more satisfied with the pilot process than personnel
moving overseas were. The Army cited several examples of problems
with overseas moves that indicated greater dissatisfaction by
those service members, and stated that it has taken action to
correct some of the problems. The surveys also showed that service
members liked some unique features the pilot offered, such as one-
on- one counseling and the simplified claims
process. However, few service members took advantage of many of
the additional services that Cendant Mobility offered,
particularly assistance in house hunting. Moreover, officials
representing The Military Coalition,
which represents five million active duty, retired, reserve, and
National Guard personnel and their families, told us that more
could be done to tailor the range of services offered by the
relocation company to meet unique military needs. Such services,
they said, might include more
assistance in finding rental property because military personnel
tend to rent rather than purchase, and providing assistance for
spouses seeking employment. Another lesson learned related to
access to survey data by outside reviewers. AAA and we were unable
to verify the survey data because the independent survey
contractors, citing privacy act restrictions, refused to grant
access to the original survey documentation. The Army stated that,
in future pilot projects, it would ensure that service members are
not
subjected to multiple surveys and would establish a better survey
method for determining customer satisfaction.
Limited Data on Claims Settlement Time
Due to weaknesses in the Army's data collection and analysis, we
could not validate that the initial offer to settle claims
occurred on average in 9 days after a completed claim form was
received by Cendant Mobility. While the claims settlement time is
one approach to measuring the claims
process, it does not capture the entire claims process and does
not measure the extent the process represented an improvement over
the existing system. The pilot process, however, does offer
benefits to the service member in terms of full replacement value
protection. For
Page 10 GAO/T-NSIAD-99-106
example, household effects are insured for their replacement value
(up to $75,000) rather than their depreciated value (which had
been capped at $40,000). The pilot process also requires less work
for the service member since the contractor provides estimates on
lost and damaged household goods, and the contractor pays the
service member directly.
The claims settlement time measurement is based on the contract
requirement that the contractor offer settlement to a member
within 30 days after receiving a claim for damaged or lost
property. Consequently, the Army developed a threshold for claims
settlement of 30 days. Under the current process, we were told
that it takes an average 30 days for the U. S. Army Claims Service
to review a completed claim form and offer settlement, and it
takes the Defense Finance and Accounting Service an additional day
to process the payment. Service members may appeal to the
contractor for reconsideration by providing additional information
or file an appeal with the Army if resolution with the contractor
could not be achieved a process that can take more than 2 years to
complete. While we found that some claims remained unpaid and
unsettled because the claims were put on hold by the service
member, the Army reported that no one under the Hunter pilot filed
an appeal as of January 1999.
The Army reported that Cendant Mobility took an average 9 days to
offer settlement for lost or damaged property, but AAA and we
could not confirm this because the Army had not collected the
necessary data. AAA
determined, and we concur, that Cendant Mobility may have taken as
long as an average 28 days to offer settlement. This figure is
based on measuring the time between the date the claim was signed
and the date the member was offered settlement. The difference
between the two estimates is primarily due to an average 19- day
period between the date the claim was
signed and the date the contractor reported that it entered the
claim into its database. Although Cendant Mobility officials
stated that its standard company practice is to enter a claim into
the database the same day a claim is received, delays could be
caused by (1) the form being mailed some time after it is dated,
(2) mail delays, and (3) contractor delays in entering the
claim into the database. Army officials stated that, due to the
difficulties in measuring claims settlement time, they would
change their data collection and analysis of this sub- factor in
future pilot programs. Direct Delivery Results May Be Overstated
We could not confirm that all eligible direct deliveries were made
without
in- transit storage because the necessary data was not collected.
Direct delivery is defined by the Army as delivery of a shipment
to a service member's residence without storage in transit. A
shipment is eligible for
Page 11 GAO/T-NSIAD-99-106
direct delivery only if it has a destination address before the
shipment is offered for delivery. Direct delivery is a contract
performance measurement, and the contractor is paid an incentive
for maintaining a direct delivery rate of over 60 percent.
Consequently, the Army threshold for this measurement is 60
percent. Measuring direct delivery in this manner does not provide
the Army sufficient information to determine
whether the results represent an improvement over the current
system. Neither AAA nor we could confirm that the shipments that
were directly delivered to service members were the only ones
eligible for direct delivery because the necessary information on
all shipments was not collected. That is, other shipments may have
been candidates (eligible) under the Army criteria; however, AAA
and we could not determine which shipments had addresses prior to
being offered for delivery (and were thus eligible) but ended up
requiring in- transit storage. The Army acknowledged the
difficulty in validating eligible direct deliveries and stated
that in the future, it would consider using in- transit storage
costs as a test factor. In the February 5, 1999, supplemental
information provided to us, the Army stated that 74 percent of
Army- wide shipments required in- transit storage, compared to 66
percent requiring temporary storage during the Army pilot.
Service members benefit from having household goods delivered
directly to their homes because the practice limits additional
handling of their property, reducing opportunities for loss and
damage. Also, the government avoids temporary, in- transit
storage, which is costly and hard for the Army to control.
Historically, household goods shipments
frequently require temporary storage because service members often
do not know at the time of shipment their new address and/ or
service members will not immediately move to the new duty station
due to vacations or military requirements.
Cost Data Were Inaccurate and Incomplete
We could not validate that 18. 6 percent represented the
additional cost of the pilot project over the baseline cost due to
weaknesses in the Army's methodology and the reliability of
overhead cost data. As we reported in June 1998, developing
overhead costs historically has been difficult in the government,
including DOD, because such data are often unreliable and
unavailable (see app. I for a detailed breakdown of pilot and
baseline costs).
Neither AAA nor we could validate some overhead costs.
Specifically, we could not confirm the accuracy of the overhead
costs attributed to MTMC because the Army used data from fiscal
year 1994. MTMC has changed
Page 12 GAO/T-NSIAD-99-106
significantly since that time due to downsizing, but AAA could not
obtain updated costs from MTMC representing the current
organization to determine the reasonableness of its overhead cost.
AAA and we also agree that the costs associated with reducing the
cost of processing documentation (claims, invoices, and inbound)
would not result in a cost reduction to the government unless the
activities that perform these functions, such as the Defense
Finance and Accounting Service, take corresponding action to
reduce their costs (for example, eliminating personnel to reflect
the drop in the workload).
AAA and we also identified other costs that were not included in
the Army's analysis, such as the costs related to awarding the
Hunter contract and the contractor's use of foreign- flag vessels.
The Army could not provide an estimated dollar value associated
with the cost of developing the solicitation, reviewing offers,
and resolving bid protests. While we recognize that some of these
estimated costs may be infrequent or one- time costs, they should
be considered. Also, the Army did not factor the contractor's use
of foreign- flag vessels into the pilot project's transportation
costs. 10 While individually, costs such as these are probably of
low dollar value in relation to other costs in the analysis;
collectively, they could have an impact on the difference between
the baseline and pilot
costs. According to the Army, some of the reasons for the higher
estimated cost of the pilot project can be attributed to (1) the
difficulty in calculating accurate baseline costs, which the Army
believes are understated; (2) the
low volume of moves 1,400 which did not provide enough leverage to
negotiate better rates and discounts; (3) higher- than- expected
cost of overseas shipments; (4) relatively high unaccompanied
baggage shipment rates, which could have been lower using
negotiated rates; (5) the packing allowance for do- it- yourself
moves was incorrectly calculated using
commercial rates during part of the 12- month test period, which
resulted in a higher rate; and (6) the fact that, generally,
quality moving services cost more. In regard to the high cost of
overseas shipments, the Army provided us with additional details
of pilot project costs that showed a 44- percent increase for
overseas household goods shipments and a 2- percent increase for
domestic shipments over the baseline cost.
10 Foreign- flag vessels are those ships registered in foreign
countries.
Page 13 GAO/T-NSIAD-99-106
Impact on Small Business Both AAA and we validated that Cendant
Mobility awarded 33 percent of the shipments to small businesses
10 percent to small business carriers and 23 percent to small
business agents. The Army measured the impact of the pilot on two
types of small business providers (carriers and agents) and
established a threshold of 23 percent to demonstrate successful
participation by small businesses. 11 The Army based this factor
on section 15( g) of the Small Business Act, which establishes a
governmentwide goal for participation by small business concerns
at not less than 23 percent of the value of all prime contracts.
Of the 790 billed Hunter shipments during
the 12- month pilot, the Army reported that 261 (33 percent) were
awarded by Cendant Mobility to small businesses.
Status of Plans to Evaluate the Pilot Programs
USTRANSCOM is still in the process of finalizing the evaluation
plan for other ongoing and planned relocation pilot programs. It
is proposing to evaluate the pilots on the same three factors that
the Army used in its
evaluation, except the factors will be defined differently and
will use a more expansive point scale. Unresolved issues, as of
February 1999, include the development of a method that recognizes
the unique characteristics and/ or process improvements of each
pilot program, and
the validation of the baseline indirect costs that will be used
for each of the pilot programs. The evaluation plan does not
currently include an evaluation of the Hunter pilot, but has used
lessons learned in the Hunter pilot to help develop the plan. We
provided comments to USTRANSCOM in this process, and among other
things, encouraged them to seek expert methodological advice
before finalizing the evaluation plan to enhance the
quality of USTRANSCOM's assessment. Summary Improving DOD's
personal property program has been a slow, complex
process. DOD and the services have spent a large amount of time
and effort to dramatically change the quality of service their
military customers receive. We support these efforts. However,
before any final conclusions can be reached, DOD must have
accurate and credible data to determine the type and extent of
changes that should be made. To facilitate a timely completion of
this process, DOD needs to (1) develop a comprehensive strategy
for testing a finite number of approaches, (2) specify time lines
for
11 In the motor freight and transportation industry, firms with
annual gross revenues of $18.5 million or less are classified as
small businesses. See Federal Acquisition Regulation 19. 102.
Page 14 GAO/T-NSIAD-99-106
implementation and completion, and (3) ensure that it has a
methodologically sound evaluation plan to assess each pilot's
attributes in a comparable manner. As it has been in the past, it
is important for DOD to continue taking into consideration the
views of the moving and relocation
companies that will be affected by changes to the current program.
Mr. Chairman, this concludes my prepared statement. I would be
pleased to answer any questions you or members of the Subcommittee
might have at this time.
Page 15 GAO/T-NSIAD-99-106
Appendix I Pilot and Baseline Costs Calculated by the Department
of the Army (Office of the Deputy Chief of Staff for Logistics)
and Validated by Army Audit Agency Appendi x I
a Military Traffic Management Command (MTMC) Source: Office of the
Deputy Chief of Staff for Logistics, Department of the Army
Cost elements Estimated pilot costs Estimated
baseline costs Transportation costs
1. Transportation $3, 005, 229 $2, 102, 384 2. Accessorials 25,
767 30, 655 3. Storage 210, 706 211, 655
Total transportation costs $3, 241, 702 $2, 344, 694
Claims paid by Army
4. Claims paid by Army 0 $66, 268
Overhead costs
5. Personnel $79,802 $221, 721 6. Management price 132, 700 0 7.
MTMC headquarters a 0 63, 510 8. Automation 0 52, 612 9. Voucher
processing 4, 614 90,658 10. Inbound processing 0 46, 834 11.
Claims processing 0 11, 510 12. Pay for performance 14, 756 0 13.
Building overhead 28, 321 43, 286 14. Telephone and copier 1,281
10, 106 15. Consumables 3, 760 5,189
Total overhead costs $265,234 $545, 426 Total cost $3, 506, 936
$2, 956, 388
Pilot costs Baseline costs Difference Percent of
difference Percent pilot costs exceeded baseline costs $3,506, 936
$2, 956, 388 $550,548 18. 6%
Page 16 GAO/T-NSIAD-99-106
Related GAO Products Defense Transportation: The Army's Hunter
Pilot Project to Outsource Relocation Services (GAO/NSIAD-98-149,
June 10, 1998). Defense Outsourcing: Better Data Needed to Support
Overhead Rates for A- 76 Studies (GAO/NSIAD-98-62, Feb. 27, 1998).
Defense Transportation: Reengineering the DOD Personal Property
Program (GAO/NSIAD-97-49, Nov. 27, 1996).
Household Goods: Administrative Changes Would Improve DOD's Do-
It- Yourself Moving Program (GAO/NSIAD-94-226, Sept. 27, 1994).
DOD Commercial Transportation: Savings Possible Through Better
Audit and Negotiation of Rates (GAO/NSIAD-92-61, Dec. 27, 1991).
Household Goods: Competition Among Commercial Movers Serving DOD
Can Be Improved (GAO/NSIAD-90-50, Feb. 12, 1990).
(709408) Lett er
Ordering Information The first copy of each GAO report and
testimony is free. Additional copies are $2 each. Orders should be
sent to the following address, accompanied by a check or money
order made out to the Superintendent of Documents, when necessary,
VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address are
discounted 25 percent.
Orders by mail: U. S. General Accounting Office P. O. Box 37050
Washington, DC 20013
or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW)
U. S. General Accounting Office Washington, DC
Orders may also be placed by calling (202) 512- 6000 or by using
fax number (202) 512- 6061, or TDD (202) 512- 2537.
Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512- 6000 using a
touchtone phone. A recorded menu will provide information on how
to obtain these lists.
For information on how to access GAO reports on the INTERNET, send
an e- mail message with info in the body to: info@ www. gao. gov
or visit GAO's World Wide Web Home Page at: http:// www. gao. gov
United States General Accounting Office Washington, D. C. 20548-
0001
Official Business Penalty for Private Use $300
Address Correction Requested Bulk Rate
Postage & Fees Paid GAO Permit No. GI00
*** End of document. ***