Contract Management: No Reliable Data to Measure Benefits of the 
Simplified Acquisition Test Program (30-SEP-03, GAO-03-1068).	 
                                                                 
In recent years, the federal government has introduced new ways  
to streamline the acquisition process. One of those vehicles is  
the simplified acquisition procedures test program, which removes
some of the procedural requirements for buying commercial goods  
and services. Using the test program, federal procurement	 
officials can make purchases faster than they have in the past	 
for procurements not exceeding $5 million. Congress mandated that
GAO determine the extent to which federal executive agencies--at 
a minimum, the Department of Defense (DOD)--have taken advantage 
of the test program and any benefits realized. One way to measure
use is to examine test program data from the Federal Procurement 
Data System (FPDS). It is the central repository of contracting  
information. In addition to examining FPDS data, GAO looked at	 
data from DOD's data system.					 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-03-1068					        
    ACCNO:   A08617						        
  TITLE:     Contract Management: No Reliable Data to Measure Benefits
of the Simplified Acquisition Test Program			 
     DATE:   09/30/2003 
  SUBJECT:   Federal procurement				 
	     Federal procurement policy 			 
	     Procurement planning				 
	     Procurement records				 
	     Program evaluation 				 
	     Reporting requirements				 
	     Data integrity					 
	     Data collection					 
	     Defense Contract Action Data System		 
	     Federal Procurement Data System			 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-03-1068

United States General Accounting Office

GAO

                       Report to Congressional Committees

September 2003

CONTRACT MANAGEMENT

No Reliable Data to Measure Benefits of the Simplified Acquisition Test Program

GAO-03-1068

Highlights of GAO-03-1068, a report to the Senate and House Committees on
Armed Services

In recent years, the federal government has introduced new ways to
streamline the acquisition process. One of those vehicles is the
simplified acquisition procedures test program, which removes some of the
procedural requirements for buying commercial goods and services. Using
the test program, federal procurement officials can make purchases faster
than they have in the past for procurements not exceeding $5 million.

Congress mandated that GAO determine the extent to which federal executive
agencies-at a minimum, the Department of Defense (DOD)-have taken
advantage of the test program and any benefits realized. One way to
measure use is to examine test program data from the Federal Procurement
Data System (FPDS). It is the central repository of contracting
information. In addition to examining FPDS data, GAO looked at data from
DOD's data system.

Before Congress decides whether to make the test program a permanent
contracting vehicle, GAO recommends that DOD and other selected federal
executive agencies ensure that reliable data are available to make program
assessments. DOD agreed with GAO's recommendation, while the other
selected federal agencies had no comments on the recommendation.

www.gao.gov/cgi-bin/getrpt?GAO-GAO-031068.

To view the full product, including the scope and methodology, click on
the link above. For more information, contact David Cooper at (202)
512-4125 or [email protected].

September 2003

CONTRACT MANAGEMENT

No Reliable Data to Measure Benefits of the Simplified Acquisition Test Program

Because the Federal Procurement Data System contains unreliable data about
the simplified acquisition test program, GAO was unable to determine the
extent to which federal executive agencies-including DOD- have used the
test program and have realized any benefits. Specifically, the database
indicated that the Departments of Treasury, Defense, and Justice were the
three largest dollar-value users of the test program in fiscal year 2001
(the latest year with complete data available). But GAO found that FPDS
either overstated or understated use of the test program by millions of
dollars. The table below shows examples of discrepancies at different
buying organizations within these three departments.

Examples of Discrepancies with FPDS's Data

Value of test program

Department's contracts, according to What procurement officials said about

buying organization FPDS (fiscal year 2001) FPDS's data

Department of the $242 million U.S. Mint said it did not use the test
Treasury's U.S. Mint program at all
DOD's Defense $4 million Defense Logistics Agency said it obligated
Logistics Agency $146 million in test program contracts
Department of $118 million After reviewing portions of FPDS data,
Justice's Federal about $31 million in contract actions,
Prison Industries Federal Prison Industries said none of

those items were purchased under the test

program

Sources: FPDS (data); GAO (analysis).

GAO also found data reliability problems with contract data in DOD's own
data system-the Defense Contract Action Data System (DCADS)-which feeds
into FPDS on an ongoing basis. For example, for fiscal year 2002, DCADS
showed about $146 million in test program contract actions for two buying
organizations for the Naval Air Systems Command and the Defense
Intelligence Agency. After reviewing contract actions that had the highest
dollar value, procurement officials at these two DOD buying organizations
said that none of the entries were awarded through the test program. There
were also reliability problems at other buying commands.

The federal buying organizations we visited have not collected any other
data that would allow us to assess whether the test program is helping to
increase efficiency, improve contract prices, reduce administrative costs,
or improve the delivery of goods and services. Anecdotal evidence
indicates that the test program is getting favorable reviews. For example,
nearly all procurement officials with whom GAO spoke at the buying
organizations GAO visited indicated that the program's primary benefit is
the ability to process a contract more efficiently.

Contents

       Letter                                                               1 
                                    Results in Brief                        2 
                                       Background                           2 
                    No Reliable Data Available to Assess Test Program       4 
                                       Conclusion                           6 
                          Recommendations for Executive Action              7 
                                     Agency Comments                        7 
     Appendix I                   Scope and Methodology                 

Appendix II 	Comments from Department of Justice's Federal Bureau of
Prisons

  Table

Table 1: Examples of Discrepancies with FPDS's Data

Abbreviations

DCADS Defense Contract Action Data System
DOD Department of Defense
FAR Federal Acquisition Regulation

FPDS Federal Procurement Data System
GAO General Accounting Office
OFPP Office of Federal Procurement Policy

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

United States General Accounting Office Washington, DC 20548

September 30, 2003

The Honorable John W. Warner
Chairman
The Honorable Carl Levin
Ranking Minority Member
Committee on Armed Services
United States Senate

The Honorable Duncan Hunter
Chairman
The Honorable Ike Skelton
Ranking Minority Member
Committee on Armed Services
House of Representatives

Acquisition reform has fundamentally changed the way the federal
government procures billions of dollars worth of goods and services each
year. The procurement process is more streamlined than ever before.
Government buyers can make their purchases with less turnaround time,
they have less paperwork, and they can rely on a variety of tools to help
them expedite the process. One tool is the simplified acquisition
procedures test program, which reduces the procedural requirements for
buying commercial goods and services not exceeding $5 million.

Congress mandated that we report on the test program and address the (1)
extent to which federal executive agencies-at a minimum, the
Department of Defense (DOD)-have used the test program, (2) benefits
realized through its use, and (3) impact that the program has had on
contract competition.1

To satisfy these objectives, we obtained and analyzed test program data
from the Federal Procurement Data System (FPDS) and DOD's Defense
Contract Action Data System (DCADS). We used these data to identify
buying organizations that were among the largest total-dollar-value users
of the test program and to review contract files at selected buying

1 Our reporting mandate is found in the Bob Stump National Defense
Authorization Act for Fiscal Year 2003, Pub. L. No. 107-314, sec. 812,
Dec. 2, 2002.

  Results in Brief

Background

organizations to determine the accuracy of the data reported. We
interviewed procurement officials to determine whether use of the test
program has resulted in noticeable benefits or affected competition.

Because the FPDS and DCADS databases contain unreliable test program data,
we were unable to determine the extent to which federal executive
agencies, including DOD, have taken advantage of the program. Neither were
we able to determine the test program's benefits and impact on
competition.

The federal buying organizations we visited have not collected any other
data that would allow us to assess whether the test program is helping to
increase efficiency, improve contract prices, reduce administrative costs,
or improve the delivery of goods and services. Two years ago, we reported
that data were not being collected to provide a basis for measuring
whether the test program produced the desired results of maximizing
efficiency and economy and minimizing administrative burden and cost. We
recommended that data be collected to demonstrate the benefits of the test
program. In response to our recommendation, DOD said it planned to convene
an integrated process team to determine ways to measure the benefits of
the test program. However, DOD has not taken action to measure the
program's benefits.

This report recommends that DOD and the Office of Federal Procurement
Policy (OFPP) develop evaluation mechanisms for measuring test program
benefits. It also recommends that the Departments of Treasury, Justice,
and Defense improve the reliability of test program data to make program
assessments. DOD partially concurred with the first recommendation and
agreed with the second. The other two federal executive agencies had no
comments on our recommendations.

To streamline the federal procurement process, Congress in 1994 authorized
the use of simplified acquisition procedures for purchases not exceeding
$100,000.2 Simplified procedures allow agency officials to expedite the
evaluation and selection processes and keep documentation to a minimum.

2 Federal Acquisition Streamlining Act of 1994, Pub. L. No. 103-355, Oct.
13, 1994.

In 1996, Congress expanded the use of simplified acquisition procedures3
by authorizing a test program that allows government buyers to procure
commercial items not exceeding $5 million4 in order to maximize efficiency
and economy and minimize burden and administrative costs for both the
government and industry.5 For example, government buyers

o  	may issue a combined synopsis and solicitation and may require
proposal submission in fewer than 45 days, as would otherwise be required;

o  	need not establish a formal evaluation plan or competitive range,
conduct discussions with vendors, or score quotations or offers; and

o  	can minimize the documentation required to justify contract award
decisions.

Under simplified acquisition procedures, the Federal Acquisition
Regulation (FAR) requires competition to the maximum extent practicable.6

The test program, which expires on January 1, 2004, is only one of a
number of streamlined contracting vehicles that federal agencies use to
procure goods and services. Other options include purchase cards, multiple
award Federal Supply Schedule contracts, governmentwide acquisition
contracts, and multiple award task and delivery order contracts. In one
way or another, these options allow government buyers to simplify and
expedite the procurement process.

In 2001, to find out whether the test program was achieving desired
results, we evaluated DOD's use of the program for commercial

3 Clinger-Cohen Act of 1996, Pub. L. No. 104-106, Div. D, Feb. 10, 1996
(short title changed from Federal Acquisition Reform Act of 1996 to
Clinger-Cohen Act of 1996, Pub. L. No. 104-208, sec. 808, Sept. 30, 1996).

4 Federal Acquisition Regulation, Subpart 13.5.

5 Since then, there has been another effort to simplify procurement in
special situations. The Homeland Security Act of 2002 temporarily expands
the use of simplified acquisition procedures (Pub. L. No. 107-296, sec.
855(b), Nov. 25, 2002). The act authorizes executive agencies to use
simplified procedures in any procurement of property or services acquired
as part of the fight against terrorism and related threats.

6 FAR sec. 13.104.

purchases.7 We reported that data had not been collected to provide a
basis for measuring whether the test program was maximizing efficiency and
economy and minimizing administrative burden and cost. However, our report
summarized a 1999 OFPP survey showing that procurement executives believed
the program improved the federal procurement process and that the test
program should be made permanent. We asked Congress to consider requiring
the OFPP to develop a method for demonstrating that the test program was
producing desired results.

FPDS is the central repository of federal contracting information, and it
contains detailed data on contract actions exceeding $25,000. Although
federal agencies collect contract data using their own data systems, their
data must be transmitted to and consolidated in FPDS on an ongoing basis.
FPDS can assist procurement managers in making such decisions as
understanding the consequences of their purchasing decisions, projecting
future needs, or leveraging overall buying power.

FPDS was designed to provide basic contracting information, such as
whether the simplified acquisition test program was used. FPDS also was
designed to provide insight on small business participation and
competition, among other things. Federal officials can use the data to
perform oversight responsibilities. The General Services Administration,
through the Federal Procurement Data Center, operates and maintains FPDS.
DOD accumulates similar data on contract actions of over $25,000 in the
DCADS database, and, like other federal agencies, transmits contract
information to FPDS.

We found significant data-reporting errors related to the test program in
both FPDS's and DCADS's databases. Because of unreliable data, we were
unable to determine the extent to which federal executive agencies have
used the simplified acquisition test program and what benefits they may
have realized from its use. We also could not determine the impact that
the test program has had on contract competition.

To verify FPDS's data, we visited the Departments of Treasury, Defense,
and Justice-the three largest dollar-value users of the simplified

  No Reliable Data Available to Assess Test Program

7 U.S. General Accounting Office, Contract Management: Benefits of
Simplified Acquisition Test Procedures Not Clearly Demonstrated,
GAO-01-517 (Washington, D.C.: Apr. 20, 2001).

acquisition test program in fiscal year 2001, as reported in FPDS. FPDS's
data showed that the Departments of Treasury, Defense, and Justice had
test program contract actions worth about $303 million, $209 million, and
$157 million, respectively, in fiscal year 2001, the latest year with the
most complete data available. We found that these figures were either
overstated or understated by millions of dollars.

For example, as shown in table 1, FPDS's data showed that the Department
of the Treasury's U.S. Mint had about $242 million in test program
contract actions, making it the largest user of the test program in fiscal
year 2001. However, the U.S. Mint officials told us that they did not use
the program at all. We also found reporting errors with FPDS's data for
DOD. An official at one of DOD's buying commands-the Defense Logistics
Agency-said it had about $146 million in test program contract actions,
but FPDS's data showed only $4 million. In the case of the Department of
Justice, FPDS's data showed that the Federal Prison Industries, which is a
part of the Federal Bureau of Prisons, had about $118 million in test
program contract actions. Federal Prison Industries officials told us they
used the test program extensively, but when we asked them to review a
portion of FPDS's data, about $31 million worth of contract actions, they
said that none of the contract actions listed were part of the test
program.

              Table 1: Examples of Discrepancies with FPDS's Data

Department's buying organization

Value of test program contracts, according to FPDS (fiscal year 2001) What
procurement officials said about FPDS's data

Department of the $242 million U.S. Mint said it did not use the Treasury's U.S.
                            Mint test program at all

DOD's Defense $4 million Defense Logistics Agency said

Logistics Agency 	it obligated $146 million in test program contracts

                Department of   $118 million      After reviewing portions of 
            Justice's Federal                  FPDS's data, about $31 million 
            Prison Industries                    in contract actions, Federal 
                                             Prison Industries said none of   
                                             those items were purchased       
                                                       under the test program 

Sources: FPDS (data); GAO (analysis).

As with other federal agencies, DOD has its own database system to collect
contract data, and it transmits those data to FPDS on an ongoing basis. We
decided to take a closer look at DOD's database-DCADS-for

fiscal year 2002, the latest year with the most complete data. As we found
with FPDS, there were reporting errors in DCADS. According to DCADS, DOD
had a total of $1.9 billion in test program contract actions. For
verification, we reviewed selected test program contract actions for DOD's
buying organizations that were major dollar-value users of the test
program, according to DCADS. While we did find that one Air Force buying
organization correctly reported its test program contract actions, other
buying commands reported them incorrectly in DCADS. For example, DCADS's
data showed that an organization within the Naval Air Systems Command had
about $122 million in test program contract actions and that the Defense
Intelligence Agency's Virginia Contracting Activity had $24 million. We
asked procurement officials at DOD's buying organizations to review
DCADS's data for contract actions that had the highest dollar value. They
said that none of those listed were awarded under the test program. Three
other DOD buying organizations also had reporting errors.

In addition to the data reliability problems we found with FPDS's and
DCADS's test program data, we also found that federal buying organizations
had not collected any other data to document whether the test program is
helping to increase efficiency, improve contract prices, reduce
administrative costs, or improve the delivery of goods and services.
However, indications are that the test program is well received. Nearly
all procurement officials with whom we spoke at selected buying
organizations view the test program favorably. They cite as the program's
primary benefit the ability to process a contract more efficiently, and
the majority advocates making the test authority permanent.

In commenting on our 2001 report, DOD stated its intention to convene an
integrated process team to consider ways for measuring the benefits of the
test program. However, DOD has not acted on this initiative.

In discussing the results of this effort, DOD officials stated that they
are willing to assess the benefits of the test program. Justice and
Treasury Department officials stated that they would work to improve the
reliability of test program's data.

Conclusion 	The simplified acquisition test program, which streamlines the
process for buying commercial items that do not exceed $5 million, expires
on January 1, 2004. Most procurement officials with whom we spoke at
selected buying organizations said that they would like the test program
to be made permanent and that there are benefits associated with buying

commercial items using simplified procedures. However, anecdotal
information is not enough to determine this program's overall impact and
benefits. Inherent in any test program is the expectation that federal
agencies establish evaluation mechanisms for assessing program results,
which includes ensuring that reliable data are collected and used for the
assessments. Our observations are that there is no reliable information
for measuring the test program's benefits.

  Recommendations for Executive Action

Agency Comments

Before Congress provides permanent authority for using simplified
procedures to acquire commercial items costing up to $5 million, we
recommend that DOD work with the Administrator of OFPP to develop
evaluation mechanisms for measuring test program benefits. In addition,
the Departments of Treasury, Justice, and Defense should independently
take appropriate actions to ensure that reliable FPDS test program data
are available to make program assessments.

We asked OFPP and the Departments of Treasury, Justice, and Defense for
written comments on the draft report. OMB, which oversees OFPP, and DOD
provided oral comments. OMB did not comment on the specifics of the report
but made a general observation that the test program provides a benefit if
used correctly. It said the test program enables agencies to gain timely
access to the marketplace while still reaping the benefits of open market
competition. OMB further noted that the test program may be especially
beneficial as an alternative to the Federal Supply Schedule and multiple
or single award task and delivery order contracts, when prenegotiated
terms and conditions of these vehicles are not suitable to meet an
agency's needs.

DOD partially concurred with our recommendation that it work with OFPP to
develop evaluation mechanisms for measuring test program benefits. DOD
stated that it would develop a methodology to evaluate the benefits of the
test program, on the basis of such metrics as procurement lead-time, and
share the results with OFPP. Using this methodology, the benefits of the
test program would be measured by sampling test program contracts from the
contract reporting system. DOD concurred with our recommendation that it
take appropriate actions to ensure that reliable FPDS test program data
are available to make program assessments. DOD is planning to issue a
memorandum to the military departments and defense agencies emphasizing
the need for all test program data to be entered into the
contract-reporting system accurately, so reliable data are available to
demonstrate the continuing need for this program.

The Department of Justice's Federal Bureau of Prisons concurred with the
information reflected in the report; the bureau's comments appear in
appendix II. The Department of the Treasury's U.S. Mint agreed with our
finding that it did not use the test program.

We are sending copies of this report to other interested congressional
committees and the Secretary of Defense; Secretary of the Air Force;
Secretary of the Army; Secretary of the Navy; Director, Defense Logistics
Agency; Director, Office of Management and Budget; Administrator, Office
of Federal Procurement Policy; Administrator of General Services;
Assistant Attorney General for Administration, Department of Justice;
Secretary of the Treasury; and Director, U.S. Mint. We will also make
copies available to others on request. In addition, this report will be
available at no cost on the GAO Web site at http://www.gao.gov.

If you have questions about this report, please contact me at (617) 788-
0500 or Ralph Dawn at (202) 512-4544. Key contributors to this assignment
were Jeffrey Rose, Ralph Roffo, Marie Ahearn, Lily Chin, and Julia
Kennon.

Sincerely yours,

David E. Cooper
Director, Acquisition and Sourcing Management

                       Appendix I: Scope and Methodology

Our objectives were to determine the (1) extent to which federal executive
agencies used the simplified acquisition test authority, (2) benefits they
realized through its use, and (3) impact that the program has had on
contract competition. To satisfy these objectives, we obtained and
analyzed test program data from the Federal Procurement Data System (FPDS)
and the Department of Defense's (DOD) Defense Contract Action Data System
(DCADS). Using the data, we identified buying organizations that were
among the largest dollar-value users of the test program, and we reviewed
contract files at selected buying organizations to determine the accuracy
of the data reported. We interviewed procurement officials from the Office
of Federal Procurement Policy and from the Departments of Treasury,
Defense, and Justice to obtain their views of the benefits from using the
test authority. In addition, we reviewed federal regulations and available
test program guidance.

We initially identified, using FPDS's data, the major dollar-value users
of the test program during fiscal year 2001, the year of the most recent
and complete data we had available at the time of our review. According to
the data, the Departments of Treasury, Defense, and Justice were the
largest dollar-value users of the test program. We then used the data to
identify the buying organizations within each of these departments that
were the largest dollar-value users. We met with procurement officials at
selected buying organizations to verify the reliability of FPDS's test
program data and to discuss the benefits realized.

We also reviewed the DCADS database to respond to our congressional
mandate's minimum requirements. Using DCADS's complete fiscal year 2002
data, we selected the buying organizations within DOD commands that were
major dollar-value users of the test authority. The selected organizations
were the (1) Air Force Air Mobility Command, (2) Air Force 21st
Contracting Squadron, (3) Army Communications-Electronics Command, (4)
Naval Air Systems Command's Naval Air Warfare Center, (5) Defense
Intelligence Agency's Virginia Contracting Activity and (6) Defense
Logistics Agency's Defense Supply Center Philadelphia. We discussed with
procurement officials at each of these buying organizations the
reliability of DCADS's test program data and the program benefits
realized.

We conducted our work from March through August 2003 in accordance with
generally accepted government auditing standards.

Appendix II: Comments from Department of Justice's Federal Bureau of Prisons

  GAO's Mission

Obtaining Copies of GAO Reports and Testimony

The General Accounting Office, the audit, evaluation and investigative arm
of Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO's Web site (www.gao.gov) contains abstracts
and full-text files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document files.
To have GAO e-mail this list to you every afternoon, go to www.gao.gov and
select "Subscribe to e-mail alerts" under the "Order GAO Products"
heading.

Order by Mail or Phone 	The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:

U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C.
20548

To order by Phone: 	Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061

Contact:

To Report Fraud, Web site: www.gao.gov/fraudnet/fraudnet.htm

  Waste, and Abuse in E-mail: [email protected]

Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470

Jeff Nelligan, Managing Director, [email protected] (202) 512-4800

Public Affairs 	U.S. General Accounting Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***