Department of Homeland Security: Better Planning and Oversight	 
Needed to Improve Complex Service Acquisition Outcomes		 
(08-MAY-08, GAO-08-765T).					 
                                                                 
The Department of Homeland Security (DHS) has relied on service  
acquisitions to meet its expansive mission. In fiscal year 2006, 
DHS spent $12.7 billion to procure services. To improve service  
acquisition outcomes, federal procurement policy establishes a	 
preference for a performance-based approach, which focuses on	 
developing measurable outcomes rather than prescribing how	 
contractors should perform services. This testimony focuses on	 
how contract outcomes are influenced by how well DHS components  
have defined and developed contract requirements and performance 
standards, as well as the need for improved assessment and	 
oversight to ensure better acquisition outcomes. GAO's statement 
is based on its report being released today, which reviewed	 
judgmentally selected contracts for eight major investments at	 
three DHS components--the Coast Guard, Customs and Border	 
Protection (CBP), and the Transportation Security Administration 
(TSA)-- totaling $1.53 billion in fiscal years 2005 and 2006;	 
prior GAO and DHS Inspector General reviews; management documents
and plans; and related data, including 138 additional contracts, 
primarily for basic services from the Coast Guard, CBP, TSA, and 
Immigration and Customs Enforcement.				 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-08-765T					        
    ACCNO:   A82027						        
  TITLE:     Department of Homeland Security: Better Planning and     
Oversight Needed to Improve Complex Service Acquisition Outcomes 
     DATE:   05/08/2008 
  SUBJECT:   Agency missions					 
	     Contract administration				 
	     Contract costs					 
	     Contract oversight 				 
	     Contract performance				 
	     Cost overruns					 
	     Data integrity					 
	     Federal procurement				 
	     Homeland security					 
	     Internal controls					 
	     Performance measures				 
	     Procurement					 
	     Procurement law					 
	     Procurement planning				 
	     Procurement policy 				 
	     Procurement practices				 
	     Quality assurance					 
	     Requirements definition				 
	     Schedule slippages 				 
	     Service contracts					 
	     Standards						 
	     Performance-based contracting			 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-08-765T

   

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Testimony: 

Before the Committee on Homeland Security, House of Representatives: 

United States Government Accountability Office: 
GAO: 

For Release on Delivery: 
Expected at 10:00 a.m. EDT: 
Thursday, May 8, 2008: 

Department Of Homeland Security: 

Better Planning and Oversight Needed to Improve Complex Service 
Acquisition Outcomes: 

Statement of John P. Hutton, Director: Acquisition and Sourcing 
Management: 

GAO-08-765T: 

GAO Highlights: 

Highlights of GAO-08-765T, a testimony before the Committee on Homeland 
Security, House of Representatives. 

Why GAO Did This Study: 

The Department of Homeland Security (DHS) has relied on service 
acquisitions to meet its expansive mission. In fiscal year 2006, DHS 
spent $12.7 billion to procure services. To improve service acquisition 
outcomes, federal procurement policy establishes a preference for a 
performance-based approach, which focuses on developing measurable 
outcomes rather than prescribing how contractors should perform 
services. 

This testimony focuses on how contract outcomes are influenced by how 
well DHS components have defined and developed contract requirements 
and performance standards, as well as the need for improved assessment 
and oversight to ensure better acquisition outcomes. 

GAOï¿½s statement is based on its report being released today, which 
reviewed judgmentally selected contracts for eight major investments at 
three DHS componentsï¿½the Coast Guard, Customs and Border Protection 
(CBP), and the Transportation Security Administration (TSA)ï¿½totaling 
$1.53 billion in fiscal years 2005 and 2006; prior GAO and DHS 
Inspector General reviews; management documents and plans; and related 
data, including 138 additional contracts, primarily for basic services 
from the Coast Guard, CBP, TSA, and Immigration and Customs 
Enforcement. 

What GAO Found: 

Over the past several years, GAO has found that appropriate planning, 
structuring, and monitoring of agency service acquisitions, including 
those that are performance-based, can help minimize the risk of cost 
overruns, delayed delivery, and unacceptably quality. Several prior GAO 
and DHS Inspector General reviews of major DHS investments using a 
performance-based approach point to such shortcomings. While all of the 
contracts GAO reviewed at the Coast Guard, CBP, and TSA had outcome-
oriented requirements, contracts for four of the eight investments did 
not have well-defined requirements, or a complete set of measurable 
performance standards, or both at the time of contract award or start 
of work. These service contracts experienced cost overruns, schedule 
delays, or did not otherwise meet performance expectations. In 
contrast, contracts for the other four investments had well-defined 
requirements linked to measurable performance standards and met the 
standards for contracts that had begun work. 

In managing its service acquisitions, including those that are 
performance-based, DHS has faced oversight challenges that have limited 
its visibility over service acquisitions and its ability to make 
informed acquisition management decisions. Notably, the department 
lacks reliable data on performance-based service acquisitions. About 
half of the 138 contracts identified by DHS as performance-based had 
none of the elements DHS requires for such contracts: a performance 
work statement, measurable performance standards, or a quality 
assurance surveillance plan. Such inaccurate data limit DHSï¿½s ability 
to perform management assessments of these acquisitions. In addition, 
the Chief Procurement Officer, who is responsible for departmentwide 
procurement oversight, has not conducted management assessments of 
performance-based service acquisitions. 

To help DHS improve outcomes for its service acquisitions, including 
those that are performance-based, GAO recommended that DHS routinely 
assess requirements for complex investments to ensure that they are 
well-defined, and develop consistently measurable performance standards 
linked to those requirements. GAO also recommended that DHS 
systematically evaluate the outcomes of major investments and relevant 
contracting methods and improve the quality of data to facilitate 
identifying and assessing the use of various contracting methods. DHS 
generally concurred with GAOï¿½s recommendations, noting some 
departmental initiatives to improve acquisition management. 

To view the full product, including the scope and methodology, click on 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-765T]. For more 
information, contact John Hutton at (202) 512-4841 or [email protected]. 

[End of section] 

Mr. Chairman and Members of the Committee: 

Thank you for inviting me here today to discuss the Department of 
Homeland Security's (DHS) planning and assessment of its complex 
service acquisitions. To meet its expansive homeland security mission, 
DHS spends billions of dollars on service acquisitions for critical 
trade, transportation, and border security investments. In fact, more 
than 80 percent of DHS's total procurement dollars are spent on 
services. Prior GAO work has found that appropriate planning, 
structuring, and monitoring of acquisitions is critical to ensuring 
that the services provided meet the government's needs.[Footnote 1] To 
help improve service acquisition outcomes, federal procurement policy 
calls for agencies to use a performance-based approach to the maximum 
extent practicable. This approach includes a performance work statement 
that describes outcome-oriented requirements, measurable performance 
standards, and quality assurance surveillance. If properly implemented, 
these characteristics can help ensure that contracted services meet 
cost, schedule, and performance requirements. Other factors, such as 
pressure to get programs up and running, additional external 
requirements, and technological challenges also impact the ability to 
achieve good acquisition outcomes. 

While a performance-based approach has been widely accepted, we have 
found that agencies face certain challenges in implementing the 
approach--especially for complex or major investments. My testimony 
today will focus on the particular challenges DHS has confronted. 
Specifically, I will discuss how contract outcomes are influenced by 
how well DHS components have defined and developed contract 
requirements and performance standards. I will also discuss the need 
for improved assessment and oversight to ensure better outcomes. 

My statement is based on our report that is being released today. 
[Footnote 2] This report focused on contracts for major investments 
with complex service acquisitions at the Coast Guard, Customs and 
Border Protection (CBP), and the Transportation Security Administration 
(TSA)--three of the DHS components reporting among the highest 
obligations for performance-based service acquisitions in fiscal years 
2005 and 2006. We conducted this performance audit in accordance with 
generally accepted government auditing standards. Those standards 
require that we plan and perform the audit to obtain sufficient, 
appropriate evidence to provide a reasonable basis for our findings and 
conclusions based on our audit objectives. We believe that the evidence 
obtained provides a reasonable basis for our findings and conclusions 
based on our audit objectives. 

Summary: 

Over the past several years, we have found that if agency service 
acquisitions, including those that are performance-based, are not 
appropriately planned, structured, and monitored, there is an increased 
risk that the government may receive products or services that are over 
budget, delivered late, and of unacceptable quality. Several prior GAO 
and DHS Inspector General reviews of major DHS investments using a 
performance-based approach point to such shortcomings. For the report 
we are releasing today, we reviewed contracts for eight major 
investments at the Coast Guard, CBP, and TSA and found that all had 
outcome-oriented requirements--as required in acquisition regulations 
and policy. However, contracts for four of these investments did not 
have well-defined requirements, or a complete set of measurable 
performance standards, or both at the time of contract award or start 
of work. These service contracts experienced cost overruns, schedule 
delays, or did not otherwise meet performance expectations. In 
contrast, service contracts for the other four investments had well- 
defined requirements linked to measurable performance standards and 
performed within budget meeting the standards in all cases where 
contractors had begun work. DHS components conducted quality assurance 
surveillance, and for the contracts that had negative outcomes, 
surveillance helped to identify contractor performance weaknesses and 
corrective action was taken. 

In managing its service acquisitions, including those that are 
performance-based, DHS has faced oversight challenges, including a lack 
of reliable data and systematic management reviews. Although 
contracting and program staff at DHS components told us that they used 
a performance-based approach to the maximum extent practicable, the 
department does not have reliable data to facilitate required reporting 
or perform management assessments of these acquisitions. Our review of 
an additional 138 contracts, which were primarily for basic services, 
found that about half of the contracts coded by DHS as performance- 
based had none of the three elements DHS requires: a performance work 
statement, measurable performance standards, or a quality assurance 
surveillance plan. Inaccurate data limit DHS's visibility over service 
acquisitions and the department's ability to make informed acquisition 
management decisions. The Chief Procurement Officer (CPO), who has 
responsibility for departmentwide procurement oversight, has begun some 
initial review of performance-based service acquisitions, but has not 
conducted management assessments of this acquisition method. 

Background: 

Over the last decade, the use of federal service contracting has 
increased and now accounts for over 60 percent of federal procurement 
dollars spent annually. A performance-based approach to federal service 
contracting was introduced during the 1990s, representing a shift from 
specifying the way in which contractors should perform work to 
specifying acquisition outcomes. Regardless of the contracting method, 
focusing on outcomes and collaboration among multiple stakeholders in 
the contracting process has been acknowledged as sound contract 
management. In 2000, federal procurement law established a performance- 
based approach as the preferred acquisition method for services. 
[Footnote 3] The Federal Acquisition Regulation requires all 
performance-based service acquisitions to include: 

* a performance work statement that describes outcome-oriented 
requirements in terms of results required rather than the methods of 
performance of the work; 

* measurable performance standards describing how to measure contractor 
performance in terms of quality, timeliness, and quantity; and: 

* the method of assessing contract performance against performance 
standards, commonly accomplished through the use of a quality assurance 
surveillance plan.[Footnote 4] 

A 1998 Office of Federal Procurement Policy (OFPP) study on performance-
based contracts--based largely on contracts for basic services, such as 
janitorial or maintenance services--showed that a number of anticipated 
benefits had been achieved, including reduced acquisition costs, 
increased competition for contracts, and improved contractor 
performance.[Footnote 5] However, implementing a performance-based 
approach is often more difficult for complex acquisitions, such as 
information technology, than it is for basic services, because agencies 
begin with requirements that are less stable, making it difficult to 
establish measurable outcomes. Such complex acquisitions may need to 
have requirements and performance standards continually refined 
throughout the life-cycle of the acquisition for a contractor to 
deliver a valuable service over an extended period of time. OFPP also 
has noted in policy that certain types of services, such as research 
and development, may not lend themselves to outcome-oriented 
requirements. 

To encourage agencies to apply a performance-based approach to service 
acquisitions, the Office of Management and Budget (OMB) established 
governmentwide performance targets, which increased to 50 percent of 
eligible service contract dollars for the current fiscal year. In 
January 2007, the congressionally mandated Acquisition Advisory Panel 
reported that performance-based acquisition has not been fully 
implemented in the federal government, despite OMB encouragement, and 
recommended that OMB adjust the governmentwide target to reflect 
individual agency assessments and plans.[Footnote 6] In May 2007, OMB's 
OFPP issued a memo providing that agencies, at a minimum, were expected 
to meet targets established and report on them in their management 
plans. In response, DHS's CPO established a performance-based target of 
25 percent for fiscal year 2007, increasing to 40 percent by fiscal 
year 2010, that was included in DHS's Performance-Based Management 
Plan. The Acquisition Advisory Panel also recommended that OFPP issue 
more explicit implementation guidance and create an "Opportunity 
Assessment" tool to help agencies identify when they should consider 
using this acquisition method. 

Requirements and Standards Definition Influence Program Outcomes: 

Our work has found that performance-based acquisitions must be 
appropriately planned and structured to minimize the risk of the 
government receiving services that are over cost estimates, delivered 
late, and of unacceptable quality.[Footnote 7] Specifically, we have 
emphasized the importance of clearly defined requirements to achieving 
desired results and measurable performance standards to ensuring 
control and accountability. Prior GAO and DHS Inspector General reviews 
of complex DHS investments using a performance-based approach point to 
a number of shortcomings. For example, in June 2007, we reported that a 
performance-based contract for a DHS financial management system, 
eMerge2, lacked clear and complete requirements, which led to schedule 
delays and unacceptable contractor performance.[Footnote 8] Ultimately, 
the program was terminated after a $52 million investment. In March 
2007, we similarly reported that the Coast Guard's performance-based 
contract for replacing or modernizing its fleet of vessels and 
aircraft, Deepwater, had requirements that were set at unrealistic 
levels and were frequently changed.[Footnote 9] This resulted in cost 
escalation, schedule delays, and reduced contractor accountability. The 
DHS Inspector General has also indicated numerous opportunities for DHS 
to make better use of sound practices, such as well-defined 
requirements.[Footnote 10] 

Consistent with our prior work, definition of requirements and 
performance standards influenced outcomes for the eight complex 
investments we reviewed. In using a performance-based approach, sound 
contracting practices dictate that required contract outcomes or 
requirements be well-defined, providing clear descriptions of results 
to be achieved. While all eight contracts for these investments had 
outcome-oriented requirements, the requirements were not always well- 
defined.[Footnote 11] Further, contracts for half of the investments 
did not have a complete set of measurable performance standards. 
Appendix I provides a summary of our analysis of the requirements, 
performance standards, and outcomes for the eight performance-based 
contracts for major investments we reviewed. 

Complex investments with contracts that did not have well-defined 
requirements or complete measurable performance standards at the time 
of contract award or start of work experienced either cost overruns, 
schedule delays, or did not otherwise meet performance expectations. 
For example, contracts for systems development for two CBP major 
investments lacked both well-defined requirements and measurable 
performance standards prior to the start of work and both experienced 
poor outcomes. The first, for DHS's Automated Commercial Environment 
(ACE) Task Order 23 project--a trade software modernization effort--was 
originally estimated to cost $52.7 million over a period of 
approximately 17 months.[Footnote 12] However, the program lacked 
stable requirements at contract award and, therefore, could not 
establish measurable performance standards and valid cost or schedule 
baselines for assessing contractor performance. Software requirements 
were added after contract award, contributing to a project cost 
increase of approximately $21.1 million, or 40 percent, over the 
original estimate. Because some portions of the work were delayed to 
better define requirements, the project is not expected to be completed 
until June 2009--about 26 months later than planned. 

The second, Project 28 for systems development for CBP's Secure Border 
Initiative (SBInet)--a project to help secure a section of the United 
States-Mexico border using a surveillance system--did not meet expected 
outcomes due to a lack of both well-defined requirements and measurable 
performance standards. CBP awarded the Project 28 contract planned as 
SBInet's proof of concept and the first increment of the fielded SBInet 
system before the overall SBInet operational requirements and system 
specifications were finalized. More than 3 months after Project 28 was 
awarded, DHS's Inspector General reported that CBP had not properly 
defined SBInet's operational requirements and needed to do so quickly 
to avoid rework of the contractor's systems engineering. We found that 
several performance standards were not clearly defined to isolate the 
contractor's performance from that of CBP employees, making it 
difficult to determine whether any problems were due to the 
contractor's system design, CBP employees, or both. As a result, it was 
not clear how CBP intended to measure compliance with the Project 28 
standard for probability of detecting persons attempting to illegally 
cross the border. Although it did not fully meet user needs and its 
design will not be used as a basis for future SBInet development, DHS 
fully accepted the project after an 8-month delay.[Footnote 13] In 
addition, DHS officials have stated that much of the Project 28 system 
will be replaced by new equipment and software. 

Conversely, we found that contracts with well-defined requirements 
linked to measurable performance standards delivered results within 
budget and provided quality service. For example, contracted security 
services at the San Francisco International Airport for TSA's Screening 
Partnership Program had well-defined requirements, and all measurable 
performance standards corresponded to contract requirements--an 
improvement from our prior reviews of the program.[Footnote 14] The 
requirements for gate, checkpoint, and baggage screening services 
clearly stated that the contractor should use technology and staff to 
prevent prohibited items from entering sterile areas of the airport and 
should work to minimize customer complaints while addressing in a 
timely manner any complaints received. The performance standards 
assessed how often screeners could successfully detect test images of 
prohibited items in checked baggage; the percentage of audited records 
and inspected equipment, property, and materials that were well-kept, 
operational, and recorded on maintenance logs; and whether all new 
hires received the required training before assuming their screening 
responsibilities. In terms of expected outcomes, the contractor 
achieved a 2.2 percent cost underrun during the first 5 months of the 
contract and exceeded most requirements. 

Unreliable Data and Lack of Management Review Constrain Oversight: 

In managing its service acquisitions, including those that are 
performance-based, DHS has faced oversight challenges, including a lack 
of reliable data and systematic management reviews. DHS contracting and 
program representatives told us that they use a performance-based 
approach to the maximum extent practicable. However, DHS does not have 
reliable data--either from the Federal Procurement Data System-Next 
Generation (FPDS-NG), the governmentwide database for procurement 
spending, or at a departmentwide level--to systematically monitor or 
evaluate or report on service acquisitions, including those that are 
performance-based. Reliable data are essential to overseeing and 
assessing the implementation of contracting approaches, acquisition 
outcomes, and making informed management decisions. Moreover, the Chief 
Procurement Officer (CPO), who has responsibility for departmentwide 
procurement oversight, has begun some initial review of performance- 
based service acquisitions, but has not conducted systematic management 
assessments of this acquisition method. 

Our analysis of information provided by contracting representatives at 
the Coast Guard, CBP, Immigration and Customs Enforcement (ICE), and 
TSA showed that about 51 percent of the 138 contracts we identified in 
FPDS-NG as performance-based had none of the required performance-based 
elements: a performance work statement, measurable performance 
standards, and a method of assessing contractor performance against 
performance standards. Only 42 of the 138 contracts, or 30 percent, had 
all of the elements, and about 18 percent had some but not all of the 
required performance-based acquisition elements (see table 1). 

Table 1: Review of Performance-Based Elements on Selected Contracts: 

Performance-based elements: All elements; 
Coast Guard: 18; Customs and Border Protection: 3; 
Immigration and Customs Enforcement: 0; 
Transportation Security Administration: 21; 
Total contracts: 42; 
Percentage of total contracts: 30.4. 

Performance-based elements: Some elements; 
Coast Guard: 16; Customs and Border Protection: 0; 
Immigration and Customs Enforcement: 5; 
Transportation Security Administration: 4; 
Total contracts: 25; 
Percentage of total contracts: 18.1. 

Performance-based elements: No elements; 
Coast Guard: 20; 
Customs and Border Protection: 5; 
Immigration and Customs Enforcement: 34; 
Transportation Security Administration: 12; 
Total contracts: 71; 
Percentage of total contracts: 51.5. 

Performance-based elements: Total; 
Coast Guard: 54; 
Customs and Border Protection: 8; 
Immigration and Customs Enforcement: 39; 
Transportation Security Administration: 37; 
Total contracts: 138; 
Percentage of total contracts: 100.0. 

Source: GAO analysis of DHS review of 138 contracts coded as 
performance-based in FPDS-NG. 

[End of table] 

Lacking reliable FPDS-NG data, reports on the use of performance-based 
contracts for eligible service obligations are likely inaccurate. Data 
reported on the use of performance-based contracts by service types-- 
ranging from basic, such as janitorial and landscaping, to complex, 
such as information technology or systems development--requested by 
OFPP in July 2006--are also likely misleading. The Acquisition Advisory 
Panel and DHS's CPO also have raised concerns regarding the accuracy of 
the performance-based designation in FPDS-NG. The Acquisition Advisory 
Panel's 2007 report noted from its review at 10 federal agencies that 
42 percent of the performance-based contracts the panel reviewed had 
been incorrectly coded. 

Inaccurate federal procurement data is a long-standing governmentwide 
concern. Our prior work and the work of the General Services 
Administration's Inspector General have noted issues with the accuracy 
and completeness of FPDS and FPDS-NG data.[Footnote 15] OMB has 
stressed the importance of submitting timely and accurate procurement 
data to FPDS-NG and issued memos on this topic in August 2004 and March 
2007. Accurate FPDS-NG data could facilitate the CPO's departmentwide 
oversight of service acquisitions, including those that are performance-
based. 

At a departmentwide level, CPO representatives responsible for 
procurement oversight indicated that they have not conducted systematic 
assessments including costs, benefits, and other outcomes of a 
performance-based approach. To improve the implementation of 
performance-based acquisitions, CPO representatives established a work 
group in May 2006 to leverage knowledge among DHS components. They also 
noted that they are working with OFPP to develop a best practices guide 
on measurable performance standards and to gather good examples of 
performance-based contracts. In addition, the CPO has implemented a 
departmentwide acquisition oversight program, which was designed with 
the flexibility to address specific procurement issues, such as 
performance-based service acquisitions, and is based on a series of 
component-level reviews.[Footnote 16] Some initial review of 
performance-based acquisitions has begun under this program, but 
management assessment or evaluation of the outcomes of this acquisition 
method has not been conducted. 

Conclusion and Recommendations: 

Consistent with federal procurement policy, DHS has emphasized a 
performance-based approach to improve service acquisition outcomes. 
However, in keeping with our prior findings, DHS's designation of a 
service acquisition as performance-based was not as relevant as the 
underlying contract conditions. Sound acquisition practices, such as 
clearly defining requirements and establishing complementary measurable 
performance standards, are hallmarks of successful service 
acquisitions. In the cases we reviewed as well as in prior findings 
where these key elements were lacking, DHS did not always achieve 
successful acquisition outcomes. The report we are releasing today 
recommends that the Secretary of Homeland Security take several actions 
to increase DHS's ability to achieve improved outcomes for its service 
acquisitions, including those that are performance-based. These actions 
include routinely assessing requirements for complex investments to 
ensure that they are well-defined and developing consistently 
measurable standards linked to those requirements; systematically 
evaluating outcomes of major investments and relevant contracting 
methods; and improving the quality of FPDS-NG data to facilitate 
identifying and assessing the use of various contracting methods. DHS 
generally concurred with our recommendations, noting some departmental 
initiatives under way to improve acquisition management. However, the 
department's response did not address how the CPO's process and 
organizational changes at the departmental level will impact component- 
level management and assessment of complex acquisitions to improve 
outcomes. Improving acquisition management has been an ongoing 
challenge since the department was established and requires sustained 
management attention. 

Mr. Chairman, this concludes my prepared statement. I would be pleased 
to respond to any questions that you or other members of the committee 
may have at this time. 

Contacts and Acknowledgments: 

For further information about this statement, please contact John P. 
Hutton at (202) 512-4841 or [email protected]. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this statement. Staff making key contributions to this 
statement were Amelia Shachoy, Assistant Director; Jeffrey Hartnett; 
Sean Seales; Karen Sloan; and Don Springman. 

[End of section] 

Appendix I: Key Characteristics of Eight Performance-Based Service 
Contracts: 

Table: Key Characteristics of Eight Performance-Based Service 
Contracts: 

Major investment by component: Coast Guard: Response Boat Medium; 
Service: Research, analysis, and financial and information management; 
Well-defined requirements[A]: contract met or mostly met the criteria; 
Measurable performance standards[B]: contract met or mostly met the 
criteria; 
Outcomes: Contractor submitted all required documentation on time; met 
project management quality standards; and maintained electronic 
archiving and restoration standards. 

Major investment by component: Customs and Border Protection: Automated 
Commercial Environment; 
Service: Trade systems software development (task order 23); 
Well-defined requirements[A]: contract did not meet the criteria; 
Measurable performance standards[B]: contract partially met the 
criteria; 
Outcomes: Costs increased by 40 percent ($21.1 million). More than a 
year behind schedule; unplanned software redesign. 

Major investment by component: Customs and Border Protection: National 
Prime Integration; 
Service: Maintenance of equipment used at border crossings, airports, 
and seaports; 
Well-defined requirements[A]: contract met or mostly met the criteria; 
Measurable performance standards[B]: contract partially met the 
criteria; 
Outcomes: Costs increased by 53 percent ($24 million). Maintenance wait 
times were longer than planned. 

Major investment by component: Customs and Border Protection: SBInet; 
Service: Project 28 border surveillance systems development and 
fielding; 
Well-defined requirements[A]: contract did not meet the criteria; 
Measurable performance standards[B]: contract partially met the 
criteria; 
Outcomes: DHS rejected initial acceptance of Project 28. The project 
was delayed 8 months with final acceptance in February 2008. DHS noted 
that the contractor met the requirements, but the project did not fully 
meet DHS's needs and the technology will not be replicated in future 
SBInet development. 

Major investment by component: Transportation Security Administration: 
Electronic Baggage Screening Program; 
Service: Maintenance for explosive trace detection machines; 
Well-defined requirements[A]: contract met or mostly met the criteria; 
Measurable performance standards[B]: contract met or mostly met the 
criteria; 
Outcomes: Contractor exceeded the performance standard for machine 
downtime with a score 1 hour less than required and operated at cost 
through the second quarter of fiscal year 2007. 

Major investment by component: Transportation Security Administration: 
Screening Partnership Program; 
Service: Passenger screening services at one airport; 
Well-defined requirements[A]: contract met or mostly met the criteria; 
Measurable performance standards[B]: contract met or mostly met the 
criteria; 
Outcomes: Contractor exceeded most performance standards; for example: 
threat detection performance and false alarm rates exceeded the quality 
standards. Contractor had cost underrun of 2.2 percent ($677,000). 

Major investment by component: Transportation Security Administration: 
Secure Flight; 
Service: Maintaining database used to screen airline passenger data; 
Well-defined requirements[A]: contract met or mostly met the criteria; 
Measurable performance standards[B]: contract partially met the 
criteria; 
Outcomes: Initial contractor planning reports were inadequate; system 
experienced operational downtime; surveillance reports identified poor 
contractor performance. Contractor generally met time frames and 
delivered within budget. 

Major investment by component: Transportation Security Administration: 
Transportation Worker Identification Credential; 
Service: Issuing identification credentials to maritime workers; 
Well-defined requirements[A]: contract met or mostly met the criteria; 
Measurable performance standards[B]: contract met or mostly met the 
criteria; 
Outcomes: Outcomes not available at the time of our review. 

Source: GAO analysis. 

[A] Well-defined requirements should provide clear descriptions of 
results to be achieved at the time of the award or start of work and 
primary requirements should not change substantially following contract 
award. 

[B] The set of measurable performance standards for a contract enables 
the government to assess all aspects of the contractor's work in terms 
of quality, timeliness, and quantity. The contract's performance 
standards are also linked to the requirements. 

[End of table] 

[End of section] 

Footnotes: 

[1] For example, GAO, Defense Acquisitions: Tailored Approach Needed to 
Improve Service Acquisition Outcomes, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-07-20] (Washington, D.C.: Nov. 
9, 2006). 

[2] GAO, Department of Homeland Security: Better Planning and 
Assessment Needed to Improve Outcomes for Complex Service Acquisitions, 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-263] 
(Washington, D.C.: Apr. 22, 2008). 

[3] Floyd D. Spence National Defense Authorization Act for Fiscal Year 
2001, Pub. L. No. 106-398 ï¿½ 821(a) (2000) required that the Federal 
Acquisition Regulation (FAR) be revised to establish a preference for 
the use of a performance-based approach in the acquisition of services, 
which was done in FAR 37.102(a), providing that performance-based 
acquisition is the preferred method for acquiring services and 
generally is to be used to the maximum extent practicable. In addition, 
this act established a preference for using firm fixed-price contracts 
or task orders--where a specified price is paid regardless of the 
contractor's incurred costs--when using a performance-based approach 
for service acquisitions. 

[4] FAR 37.601; FAR 37.602(b); FAR 37.604. A fourth element, 
performance incentives, is required where appropriate. 

[5] Office of Management and Budget, Office of Federal Procurement 
Policy, A Report on the Performance-Based Service Contracting Pilot 
Project, May 1998. 

[6] Report of the Acquisition Advisory Panel to the Office of Federal 
Procurement Policy and the United States Congress, January 2007. 

[7] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-20]. 

[8] GAO, Homeland Security: Departmentwide Integrated Financial 
Management Systems Remain a Challenge, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-07-536] (Washington, D.C.: 
June 21, 2007). 

[9] GAO, Coast Guard: Status of Efforts to Improve Deepwater Program 
Management and Address Operational Challenges, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-07-575T] (Washington 
D.C.: Mar. 8, 2007). 

[10] See for example, Department of Homeland Security Inspector 
General, Major Management Challenges Facing the Department of Homeland 
Security, OIG-08-11 (Jan. 2008), and Department of Homeland Security 
Inspector General, Transportation Security Administration's Information 
Technology Managed Services Contract, OIG-06-23 (Feb. 2006). 

[11] FAR 2.101 specifically provides that a performance work statement 
for performance-based acquisitions describe the required results in 
clear, specific, and objective terms with measurable outcomes. 

[12] Begun in 2001, ACE is intended to replace and supplement existing 
cargo processing technology and will be developed and deployed in a 
series of increments. The goals of ACE include (1) supporting border 
security by enhancing analysis and information sharing with other 
government agencies and providing CBP with the means to decide before a 
shipment reaches the border if it should be targeted or expedited and 
(2) streamlining time-consuming and labor-intensive tasks for CBP 
personnel and the trade community through a national trade account and 
single Web-based interface. Task Order 23 was the sole focus of our 
review. 

[13] GAO, Secure Border Initiative: Observations on the Importance of 
Applying Lessons Learned to Future Projects, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-08-508T] (Washington, D.C.: 
Feb. 27, 2008). 

[14] GAO, Aviation Security: Preliminary Observations on TSA's Progress 
to Allow Airports to Use Private Passenger and Baggage Screening 
Services, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-126] 
(Washington, D.C.: Nov. 19, 2004), and Aviation Security: Progress 
Made to Set Up Program Using Private-Sector Airport Screeners, but 
More Work Remains, GAO-06-166 (Washington, D.C.: Mar. 31, 2006). 

[15] For example, GAO, Reliability of Federal Procurement Data, 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-295R] 
(Washington, D.C.: Dec. 30, 2003); GAO, Improvements Needed to the 
Federal Procurement Data System-Next Generation, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-05-960R] (Washington, D.C.: 
Sept. 27, 2005); and General Services Administration Inspector 
General, Review of the Federal Procurement Data System-Next 
Generation (FPDS-NG), Report Number A040127/O/T/F06016 (March 2006). 

[16] GAO, Department of Homeland Security: Progress and Challenges in 
Implementing the Department's Acquisition Oversight Plan, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-07-900[ (Washington, D.C.: June 
2007). 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office: 
441 G Street NW, Room LM: 
Washington, D.C. 20548: 

To order by Phone: 
Voice: (202) 512-6000: 
TDD: (202) 512-2537: 
Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: [email protected]: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, [email protected]: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, [email protected]: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: 

*** End of document. ***