    Federal Bureau of Investigation: Weak Controls over Trilogy
    Project Led to Payment of Questionable Contractor Costs and
    Missing Assets (28-FEB-06, GAO-06-306).

    The Trilogy project--initiated in 2001--is the Federal Bureau of
    Investigation's (FBI) largest information technology (IT) upgrade
    to date. While ultimately successful in providing updated IT
    infrastructure and systems, Trilogy was not a success with regard
    to upgrading FBI's investigative applications. Further, the
    project was plagued with missed milestones and escalating costs,
    which eventually totaled nearly $537 million. In light of these
    events, Congress asked GAO to determine whether (1) internal
    controls provided reasonable assurance that improper payment of
    unallowable contractor costs would not be made or would be
    detected in the normal course of business, (2) payments to
    contractors were properly supported as a valid use of government
    funds, and (3) FBI maintained proper accountability for assets
    purchased with Trilogy project funds.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-06-306
        ACCNO:   A49337
      TITLE:     Federal Bureau of Investigation: Weak Controls over
    Trilogy Project Led to Payment of Questionable Contractor Costs
    and Missing Assets
         DATE:   02/28/2006
      SUBJECT:   Accountability
    	     Computer equipment contracts
    	     Contract oversight
    	     Contractor payments
    	     Equipment upgrades
    	     Federal funds
    	     Government contracts
    	     Information technology
    	     Internal controls
    	     Questionable payments
    	     Technology modernization programs
    	     Cost analysis
    	     Financial management
    	     FBI Trilogy Project

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-06-306

         * Report to Congressional Requesters
              * February 2006
         * FEDERAL BUREAU OF INVESTIGATION
              * Weak Controls over Trilogy Project Led to Payment of Questionable
                Contractor Costs and Missing Assets
         * Contents
              * Results in Brief
              * Background
                   * FBI's Asset Accountability Procedures
                   * Internal Control
              * Insufficient Invoice Review and Approval Process Increased FBI's
                Vulnerability to Payment of Unallowable Contractor Costs
                   * Invoices Were Approved for Payment without Validation that
                     Goods and Services Were Received
                   * Invoice Review Responsibilities Were Not Clearly Defined
                   * Invoices Did Not Provide Adequate Support for All Charges
              * Some Payments Made to Contractors Were for Questionable Costs
                   * First-Class Travel and Other Excessive Airfare Costs
                   * Excess Overtime Charges
                   * Questionable Labor Rates
                   * Other Questionable Costs
                        * Example 1-Subcontractor Labor Costs
                        * Example 2-Other Direct Costs/Training
                        * Example 3-Other Direct Costs/Equipment Disposal
                        * Example 4-Subcontractor Labor Invoice-Duplicate Payment
              * Major Lapses in Accountability Resulted in Millions of Dollars of
                Missing Trilogy Equipment
                   * FBI's Overreliance on Contractors Diminished Its Ability to
                     Properly Account for Trilogy Assets
                   * FBI Lacked Adequate Physical Control over Trilogy Assets
                   * FBI Is Unable to Locate Millions of Dollars of Trilogy
                     Assets
              * Conclusions
              * Recommendations for Executive Action
              * Agency Comments and Our Evaluation
         * Key Trilogy Milestones
         * Scope and Methodology
              * Validity of Payments
                   * Payments to SAIC
                   * Payments to CSC
                   * Payments to Mitretek
              * FBI's Asset Accountability
         * Comments from the Federal Bureau of Investigation
         * Comments from the General Services Administration
         * GAO Contacts and Staff Acknowledgments

    Report to Congressional Requesters

    February 2006

    FEDERAL BUREAU OF INVESTIGATION

    Weak Controls over Trilogy Project Led to Payment of Questionable
    Contractor Costs and Missing Assets

    Contents

    Tables

    Figures

    February 28, 2006Letter

    The Honorable Arlen Specter Chairman The Honorable Patrick J. Leahy
    Ranking Minority Member Committee on the Judiciary United States Senate

    The Honorable Richard J. Durbin United States Senate

    The Honorable Charles E. Grassley United States Senate

    The Honorable Orrin G. Hatch United States Senate

    For several years Congress recognized that the Federal Bureau of
    Investigation's (FBI) information technology (IT) systems were archaic and
    inadequate for efficiently and effectively investigating criminal cases.
    FBI recognized the need to modernize its IT systems before the September
    11, 2001, terrorist attacks, but those events underscored FBI's need to
    improve its ability to effectively retrieve, analyze, and share
    investigative information necessary to carry out its mission. Initiated in
    mid-2001, Trilogy-FBI's largest IT upgrade to date-was intended to
    modernize FBI's IT infrastructure and systems and provide needed
    applications to help FBI agents, analysts, and others do their jobs.

    The Trilogy project consisted of two primary efforts: an IT infrastructure
    update and an upgrade of FBI's investigative applications. While
    ultimately successful in providing the infrastructure update, the project
    was not a success with regard to upgrading the investigative applications.
    Further, the project experienced numerous schedule delays and cost
    increases.1 Project costs, which were originally estimated at
    approximately $380 million, eventually escalated to approximately $537
    million. Although the scheduled completion date for the overall Trilogy
    project was June 2004, after September 11, 2001, FBI required an
    accelerated deployment plan and moved up the expected completion dates.
    The completion date for the portion of Trilogy related to upgrading FBI's
    IT infrastructure was accelerated from May 2004 to July 2002. After
    several delays, the upgrade was completed in April 2004, a month before
    the original due date. While the overall scheduled completion date for the
    investigative application upgrades, which became known as the Virtual Case
    File (VCF), was originally June 2004, the due date for the first VCF
    deliverable was accelerated to December 2003. However, in July 2004 the
    VCF portion of the Trilogy project was scaled back after the completion of
    the project was determined to be infeasible and cost prohibitive as
    originally envisioned.

    The Department of Justice (DOJ) Office of Inspector General (OIG) has
    reported numerous issues that contributed to the cost increases and
    delays, including poorly defined and slowly evolving design requirements,
    contracting weaknesses, unrealistic task scheduling, and lack of
    management continuity and oversight for tracking and overseeing costs
    effectively.2 GAO also reported on weaknesses in FBI's IT systems
    development and management capabilities, including contractor oversight.3

    Because of these issues, you asked us to audit the costs of the Trilogy
    project, the majority of which represented the purchase of goods and
    services from contractors. Our objectives were to determine whether (1)
    FBI's internal controls provided reasonable assurance that payment of
    unallowable contractor costs would not be made or would be detected in the
    normal course of business,4 (2) FBI's payments to contractors were
    properly supported as a valid use of government funds, and (3) FBI
    maintained proper accountability for assets purchased with Trilogy project
    funds.

    To address these objectives, we used various internal control standards
    and guidance5 as a basis to assess FBI's internal controls over the
    payments made with Trilogy funds. We also reviewed FBI policy and
    procedure manuals; applicable federal regulations, including the Federal
    Acquisition Regulation (FAR),6 Federal Travel Regulation,7 and Joint
    Travel Regulations (JTR);8 prior GAO and DOJ OIG reports on Trilogy
    issues; Trilogy contract documents and interagency agreements; contractor
    invoices; and other documentation supporting goods provided and services
    rendered. We performed data mining and forensic auditing techniques to
    select transactions to determine whether payments to contractors were
    properly supported as a valid use of government funds. We tested
    accountable property to determine whether assets were entered in FBI's
    property system and conducted a physical observation of selected assets to
    validate their existence. In addition, we conducted interviews with
    officials from FBI, General Services Administration's (GSA) Federal
    Systems Integration and Management Center (FEDSIM), Department of the
    Interior (DOI), and Trilogy contractors. We also performed walkthroughs to
    gain an understanding of the processes used to review and approve invoices
    and account for property. While we identified some payments for
    questionable contractor costs,9 our work was not designed to identify all
    questionable payments or to estimate their extent.

    We provided FBI a draft of this report and GSA a draft of applicable
    sections of this report for review and comment. FBI and GSA provided
    written comments, which are reprinted in appendixes III and IV,
    respectively. FBI and GSA also provided technical comments, which we have
    incorporated as appropriate. We also discussed with Trilogy contractors
    any findings that related to them. We performed our work in accordance
    with generally accepted government auditing standards in Washington D.C.
    and at two FBI field sites and various other GSA and contractor locations
    in Virginia from May 2004 through December 2005. Our scope and methodology
    are discussed in greater detail in appendix II.

    Results in Brief

    FBI's internal controls did not provide reasonable assurance that payments
    to contractors for unallowable costs would not be made or would be
    detected in the normal course of business. Our review found that FBI's
    review and approval process for Trilogy contractor invoices, which
    included GSA's review in its role as contracting agency, did not provide
    an adequate basis to verify that goods and services billed were actually
    received by FBI or that the amounts billed were appropriate. This occurred
    in part because responsibility for the review and approval of invoices was
    not clearly defined in the contracts and interagency agreements related to
    Trilogy project oversight. In addition, certain contactor invoices lacked
    certain detailed information required by the Trilogy task orders and other
    additional information that would be needed to facilitate an adequate
    invoice review process. Despite this, invoices were paid without requests
    for additional supporting documentation necessary to validate the charges.
    These weaknesses made FBI highly vulnerable to payments of unallowable and
    questionable costs with Trilogy funds. Until significant improvements are
    made, these vulnerabilities will continue for future projects where FBI
    uses contractors for the delivery and deployment of goods and services.

    We used forensic auditing techniques, including data mining and document
    analysis, to assess the validity of selected payments and identified $10.1
    million of questionable contractor costs paid by FBI. We found instances
    of first-class travel and other excessive airfare costs, incorrect
    billings for overtime hours worked, potential overcharging of labor rates,
    and other questionable costs. For example, one contractor could not
    provide adequate documentation to substantiate about $5.5 million of
    subcontractor charges for other direct costs billed to FBI. Given FBI's
    poor control environment over invoice payments and the fact that we
    reviewed only selected FBI payments to Trilogy contractors, other
    questionable costs may have been paid for that have not been identified.
    Further, if these weaknesses go uncorrected, future contracts, including
    those related to FBI's new electronic information management system
    initiative, referred to as Sentinel, will be highly exposed to improper
    payments.

    FBI did not maintain adequate accountability for all computer equipment
    purchased for the Trilogy project. FBI relied extensively on contractors
    to account for Trilogy assets while they were being purchased, warehoused,
    and installed. However, FBI did not have controls or data to verify the
    accuracy and completeness of the contractor records it ultimately relied
    on and to ensure that it received all the items purchased through its
    contractors. Moreover, once FBI took possession of the Trilogy equipment,
    it did not establish adequate physical control over the assets. FBI failed
    to record accountable assets-equipment with a value of $1,000 or more, or
    deemed by FBI to be susceptible to theft-into its property system in a
    timely manner, did not properly use its bar codes to individually track
    accountable assets, and did not effectively use its inventory process to
    identify all potentially missing assets. These breakdowns in control over
    Trilogy assets created an environment in which equipment could be lost or
    stolen without detection.

    Given the serious nature of these control weaknesses, we performed
    additional test work to determine whether all accountable assets purchased
    with Trilogy funds could be accounted for by FBI. FBI was unable to locate
    over 1,200 of these assets, which we estimate are valued at approximately
    $7.6 million, including items such as computer desktops, laptops,
    printers, and servers. In addition to the items we found missing, as a
    result of its physical inventory procedures, FBI reported 37 pieces of
    contractor-purchased Trilogy equipment valued at about $167,000 that had
    been lost or stolen. Due to the significant weaknesses we identified in
    FBI's Trilogy property controls, the actual amount of missing or stolen
    equipment could be even higher. Until FBI strengthens its asset
    accountability controls it will remain highly vulnerable to continued loss
    of existing assets, as well as those it may acquire in the future.

    We are making 27 recommendations to address the issues identified in this
    report. Regarding FBI's and GSA's processes for reviewing and approving
    contractor invoices, we are making 6 recommendations to FBI and 5 to GSA
    to develop or strengthen these types of internal control procedures. We
    are making 4 additional recommendations to GSA in coordination with FBI,
    to take actions to resolve certain of the questionable costs we
    identified. And we are making 12 other recommendations to help FBI improve
    its accountability over existing Trilogy assets and those that will be
    purchased in connection with future projects such as Sentinel.

    In written comments on a draft of this report, FBI stated that it
    concurred with our recommendations and that it has made and continues to
    make significant structural and procedural changes to address our
    recommendations. FBI also provided additional information related to
    Trilogy assets we identified as missing. In written comments on a draft of
    applicable sections of this report, while GSA stated that it accepted our
    recommendations, it did not believe that 1 of them was needed, and
    described some of the improvements to its internal controls and other
    business process changes already implemented. GSA also expressed concern
    with some of our observations and conclusions related to the invoice
    review and approval process and our analysis of airfare costs. We continue
    to believe that our report is accurate and that all of the recommendations
    should be implemented. Our responses to these comments are provided in the
    Agency Comments and Our Evaluation section of this report and in appendix
    IV, immediately following the reprinted GSA comments.

    Background

    Recognizing the need to modernize its IT systems, FBI proposed a major
    technology upgrade plan to Congress in September 2000. FBI's Information
    Technology Upgrade Project, which FBI subsequently renamed Trilogy, was
    FBI's largest automated information systems initiative to date. Trilogy
    consisted of three parts: (1) the Information Presentation Component (IPC)
    to upgrade FBI's computer hardware and software, (2) the Transportation
    Network Component (TNC) to upgrade FBI's communication network, and (3)
    the User Application Component (UAC) to upgrade and consolidate FBI's five
    most important investigative applications.

    To expedite the contracting process, FBI entered into an interagency
    agreement with GSA to support FBI's use of the FEDSIM Millennia
    governmentwide acquisition contract10 for the implementation of Trilogy's
    three functional components, IPC, TNC, and UAC. FEDSIM, serving as
    contracting agency, was to provide all contract administrative services
    necessary to support the task orders. Because the Trilogy project was so
    large, DOJ required FBI to use two contractors for the three Trilogy
    components. FBI combined the IPC and TNC portions of Trilogy into one task
    order because both components involved physical infrastructure
    enhancements. IPC provided for new desktop computers, servers, and
    commercial-off-the-shelf automation software, including Web-browser and
    e-mail to enhance usability by the agents. TNC upgraded the complete
    communication infrastructure, including high-capacity wide-area and
    local-area networks, authorization security, and encryption of data
    transmission and storage. The IPC/TNC task order was awarded in May 2001
    to DynCorp (now Computer Sciences Corporation (CSC)).11 The IPC/TNC
    upgrades would provide the physical infrastructure needed to run the
    applications developed under UAC, the third Trilogy component.

    The third component of Trilogy-the UAC task order-was awarded in June 2001
    to Science Applications International Corporation (SAIC). The goal of UAC
    was to replace FBI's paper case files with electronic files and improve
    efficiency. The heart of the UAC portion became the development of the VCF
    system to replace the obsolete Automated Case Support system, FBI's
    primary investigative application that uploads and stores case files
    electronically.

    The above two Trilogy contracts12 were awarded on a cost-plus-award fee
    basis for labor charges, meaning that the contractor's costs incurred are
    reimbursed and fees13 may be awarded to the contractor based on
    performance. The FAR states that cost-reimbursement type contracts may
    only be used if appropriate government surveillance during performance
    will provide reasonable assurance that efficient methods and effective
    cost controls are used. The aspects of these contracts related to the
    purchase of equipment were based on fixed-price arrangements, meaning that
    a set price for the equipment is agreed to up front.

    In addition to the two primary contracts discussed above, FBI awarded two
    additional contracts to assist with the technical oversight, monitoring,
    and integration of the two primary Trilogy contracts described above. The
    first of the two additional contracts was awarded in February 2001, also
    through GSA FEDSIM, to Mitretek for Systems Engineering and Technical
    Assistance (SETA) services.14 Under the SETA contract, Mitretek was
    required to assist FBI with a wide array of tasks, including program and
    contract management, fiscal and budgetary oversight, cost estimating, and
    several other technical aspects of the Trilogy project. The second of the
    two additional contracts was awarded to SAIC for the integration15 of the
    three Trilogy components.16

    In July 2004, the VCF was scaled back to the Initial Operating Capability
    and the remaining deliverables were cancelled after the (1) initial
    deliverable was rejected by FBI and (2) VCF was determined to be
    infeasible and cost prohibitive to implement as originally envisioned.
    After a 90-day limited pilot that ended in March 2005, VCF offline and the
    pilot results were then to be analyzed by FBI for requirements development
    of its new electronic information management system initiative. In August
    2005, FBI released a solicitation for proposals to develop FBI's new
    electronic information management system, referred to as Sentinel. The
    solicitation was sent to more than 40 eligible companies under a National
    Institutes of Health governmentwide acquisition contract. Similar to VCF,
    the goal of Sentinel is to replace FBI's legacy case management
    capabilities with an integrated, paperless file management and workflow
    system.

    FBI's Asset Accountability Procedures

    According to FBI policy, assets valued at $1,000 or more, as well as
    certain sensitive items, such as firearms, laptop computers, and central
    processing units, are considered to be "accountable" assets, regardless of
    cost, and must be accounted for individually in FBI's Property Management
    Application (PMA). PMA is an automated management system that allows FBI
    to track the cost, location, and history of its accountable assets. PMA
    includes a variety of data fields to identify each item, including the
    acquisition date, received date, acquisition cost, last inventory date,
    bar code number, serial number, cost center for the office where the item
    is located, description of the item, and other information.

    Ongoing deficiencies in FBI's management of property have been identified
    by DOJ's OIG and FBI's independent financial statement auditor. In August
    2002, the DOJ OIG issued a report that revealed significant problems with
    FBI's management of laptop computers, including findings that FBI did not
    reconcile its property management data with purchase data from its
    accounting system, did not have an inventory record for accountable assets
    not in PMA that were lost or stolen, and could not verify whether the
    number of items purchased agreed with the number of items recorded in
    PMA.17 Additionally, in September 2004, the DOJ OIG reported on weaknesses
    in FBI's controls over nonaccountable property at FBI's Baltimore field
    office after an employee pleaded guilty to the theft and sale of FBI
    photography equipment.18 Annually, since fiscal year 1999, FBI's
    independent financial statement auditors have identified internal control
    weaknesses in the area of property management. They specifically reported
    that FBI needed to improve its procedures related to the timely and
    accurate recording, reconciling, and reporting of property and equipment
    in PMA.

    Internal Control

    Internal control is a major part of managing any organization. As required
    by 31 U.S.C. 3512(c),(d), commonly referred to as the Federal Managers'
    Financial Integrity Act of 1982, the Comptroller General issues standards
    for internal control in the federal government.19 These standards provide
    the overall framework for establishing and maintaining internal control
    and for identifying and addressing major performance and management
    challenges and areas at greatest risk of fraud, waste, abuse, and
    mismanagement. According to these standards, internal control comprises
    the plans, methods, and procedures used to meet missions, goals, and
    objectives. Internal control is the first line of defense in safeguarding
    assets and preventing and detecting fraud and errors. Internal control,
    which is synonymous with management control, helps government program
    managers achieve desired results through effective stewardship of public
    resources.

    Control activities are the policies, procedures, techniques, and
    mechanisms that enforce management's directives and help ensure that
    actions are taken to address risks. Control activities are an integral
    part of an entity's planning, implementing, reviewing, and accountability
    for stewardship of government resources and achieving effective results.
    They include a wide range of diverse activities. Some examples of control
    activities include (1) establishing physical controls over vulnerable
    assets to reduce the risk of loss or unauthorized use and periodically
    counting and comparing such assets to control records; (2) ensuring that
    documentation and records are properly managed and maintained and that
    transactions are appropriately documented and readily available for
    examination; (3) assigning accountability for the custody and use of
    resources and records to help reduce the risk of errors, fraud, misuse, or
    unauthorized alteration; and (4) implementing management level reviews at
    the functional level to ensure that appropriate control activities are
    being employed, such as reconciliations of summary information to
    supporting detail.

    Insufficient Invoice Review and Approval Process Increased FBI's
    Vulnerability to Payment of Unallowable Contractor Costs

    FBI's review and approval process for Trilogy contractor invoices, which
    was carried out by a review team consisting of officials from FBI, GSA,
    and Mitretek, did not provide an adequate basis to verify that goods and
    services billed were actually received by FBI or that payments were for
    allowable costs. This occurred in part because responsibility for the
    review and approval of invoices was not clearly defined in the Mitretek
    contract and in the interagency agreements related to Trilogy project
    oversight. In addition, contactor invoices frequently lacked detailed
    information required by the contracts and other additional information
    that would be needed to facilitate an adequate invoice review process.
    Despite this, invoices were paid without requests for additional
    supporting documentation necessary to determine the validity of the
    charges. These weaknesses in FBI's review and approval process made the
    agency highly vulnerable to payment of unallowable or questionable
    contractor costs.

    Invoices Were Approved for Payment without Validation that Goods and
    Services Were Received

    While the review and approval process differed for each contractor and
    type of invoice charge, in general the process carried out by the review
    team lacked key procedures to reasonably ensure that goods and services
    billed were actually received by FBI or that the amounts billed and paid
    were for allowable costs. Internal control guidance requires agencies to
    establish controls that reasonably ensure, among other things, that funds,
    property, and other assets are safeguarded against waste, loss,
    unauthorized use, or misappropriation.20

    Contractor invoices included costs for labor, including related overhead
    costs; travel; other direct costs (ODC); subcontractor labor; and
    purchased equipment. Table 1 provides a summary of total payments made to
    Trilogy contractors for these categories, as well as total Trilogy costs
    in each category.

    Table 1: Payments for Trilogy by Contractor and Category (in millions)

    Source: GAO analysis of contractor invoices, Mitretek Spend Plan, and FBI
    records.

    aSubcontractor charges for travel were included in CSC and SAIC travel
    invoices and are included under the travel category.

    bSubcontractor charges for ODC were included in CSC's ODC invoices and are
    included under the ODC category.

    cFor CSC, the "other" category represents miscellaneous maintenance
    charges and $10 million in awards and fees. For SAIC, the "other" category
    represents $5 million in awards and fees. For Mitretek, the "other"
    category represents $1.1 million in fees. We did not assess the propriety
    of these payments.

    dTrilogy costs beyond those billed by contractors primarily related to
    direct purchases of equipment by FBI.

    Each member of the review team-which included personnel from FBI; GSA, the
    contracting agency; and Mitretek-was to perform some level of review of
    the invoices submitted by the contractors for payment. During the project,
    each of the review team members, at times, worked on-site with the
    contractors. As is discussed later, the specific roles of each party were
    not clearly defined, which limited the effectiveness of the invoice review
    and approval process. Figure 1 illustrates this invoice review and
    approval process.

    Figure 1: Invoice Review and Approval Process

    aGSA facilitated the payment of contractor invoices and was subsequently
    reimbursed by FBI with funds appropriated for the Trilogy project.

    Our review disclosed serious gaps in the review process for each of the
    major categories of contractor costs, as follows.

    Labor-According  to GSA, it typically reviewed labor charges by looking
    for unusual or excessive hours worked or rates charged and recalculating
    some amounts to ensure mathematical accuracy. GSA also stated that its
    personnel generally compared average fully burdened labor rates (labor,
    overhead, fringe benefits, and general and administration costs) charged
    to ceiling rates (maximums) established in the Trilogy contracts. However,
    GSA was not able to provide us with an explanation for or evidence of how
    they resolved clearly questionable labor charges we identified, including
    hours billed far in excess of a normal pay period. For example, we
    identified one individual who charged 371 hours for one 4-week period (an
    average of 93 hours per week) and 359 in the following 5-week period (an
    average of 70 hours per week). There was no evidence that GSA had
    questioned whether these seemingly excessive hours were valid. GSA stated
    that these types of issues were usually resolved on the telephone and
    therefore they usually did not maintain any documentation of their
    inquiries.

    On-site members of the review team indicated that they generally knew the
    contractor employees working on the project and reviewed the hours billed
    for reasonableness. However, the review team did not have a systematic
    process in place to help ensure that individuals listed on invoices had
    actually worked on Trilogy the number of hours being billed or that the
    job classifications and related billing rates were appropriate. In
    addition, there was no documented assessment of whether the overall hours
    being billed for a particular activity were in line with expectations.

    Subcontractor Labor-The review team paid contractor invoices for
    subcontractor labor without any attempt to assess the validity of the
    charges. The GSA official responsible for paying the invoices stated that
    the review team relied on the contractors to properly bill for the costs
    related to their subcontractors and to validate the subcontractor
    invoices. However, the review team had no process in place to assess
    whether or not the contractors were properly validating their
    subcontractor labor charges or to assess the allowability of those
    charges. In addition, we found that CSC, which billed the bulk (i.e.,
    about $116 million) of the subcontractor labor costs,21 did not always
    have sufficient documentation of subcontractor charges to enable CSC, or
    anyone else, to perform any assessment of the allowability of those costs.
    For example, the only supporting documentation CSC could provide us for
    about $2 million in subcontractor labor charges we selected for review
    were subcontractor invoices that lacked some of the basic information
    needed to assess the labor costs, such as the names of the subcontractor
    employees, hours billed, or individual labor rates.

    Travel-These charges were reviewed differently by the review team for SAIC
    and CSC invoices. For SAIC travel, GSA told us they compared invoiced
    amounts to travel authorizations and verified the per diem and lodging
    rates in the authorizations to those prescribed under the Federal Travel
    Regulation. However, travel authorizations were not always submitted and
    approved before travel occurred and in some cases were based on actual
    amounts. The review team told us that they reviewed SAIC travel vouchers
    or receipts in a few instances over 4 years when amounts billed were
    higher than expected to verify the amounts charged on the travel invoices.
    However, there was no systematic process to review travel costs billed to
    the Trilogy project. For CSC travel, because CSC's travel authorizations
    did not include details by employee or the estimated cost for each trip
    and frequently covered several trips, the GSA official who paid the
    invoices told us she relied on members of the review team that worked
    on-site to review the travel invoices. These on-site review team officials
    indicated that their review process was based on their general
    understanding of who was traveling. However, we determined that no one on
    the review team obtained travel vouchers or receipts to verify that
    amounts billed by CSC were a necessary and proper charge to the Trilogy
    project and were reasonable based on the location and length of travel
    required.

    Other Direct Costs (ODC)-These charges were paid without validation of the
    actual amounts included in the invoices. The review team relied on
    contractors to obtain purchase orders for ODC charges. For SAIC ODC
    invoices, the review team generally tracked actual charges billed on
    invoices compared to purchase order amounts. However, there was no review
    of receipts or other documentation to validate the actual charges on
    invoices. CSC ODC invoices were paid without matching the charges to a
    purchase order or documentation of the actual cost incurred. Therefore,
    the review team had no basis for confidence that CSC ODC charges were
    approved ahead of time or appropriately billed. CSC ODC charges also
    included subcontractor ODC. We asked CSC for supporting documentation for
    selected subcontractor ODC and found that CSC's only support was
    subcontractor invoices that included only a brief description of the
    nature of the charge and the amount. No supporting receipts or other
    documentation necessary to verify the charges was provided. For example,
    CSC billed FBI for ODC of $456,211 on an invoice submitted in November
    2003. The only description on the invoice for these charges was "other
    direct costs." We requested from CSC any documentation they had in their
    files to support this charge from its subcontractor, CACI Inc. - Federal
    (CACI). CSC was able to provide an invoice with one line entitled
    "facilities/materials" and a spreadsheet with a general summary of the
    charges. Further, the e-mail exchange presented in figure 2 shows that CSC
    recognized that they did not have enough detail to review the ODC charge,
    but approved the invoice anyway. As noted below, the final entry in the
    exchange is, "It's not what we asked for but at this point it doesn't
    really matter. Approve it."

    Figure 2: CSC E-mail Approval of Subcontractor ODC Charge

    Equipment-Charges for equipment  purchased by contractors and billed to
    FBI were reviewed merely by tracking the total cost of equipment invoices
    to ensure that the total amount did not exceed the approved amount on
    purchase orders. However, neither GSA, FBI, nor Mitretek performed
    procedures to ensure that individual equipment items billed by the
    contractors were actually received before payment. Discussions with the
    contractors revealed that this was a high-risk area because some of the
    invoices they submitted were for equipment that had not yet been delivered
    to FBI. The review team approved and paid these invoices without question.
    In addition, FBI purchased some IPC/TNC equipment directly from vendors
    and delivered the equipment to contractor locations, but did not have a
    mechanism in place to physically verify receipt of that equipment at FBI
    sites before paying the related invoices. There was also no subsequent
    verification by the review team that all equipment purchased through
    contractors and vendors was ultimately received by FBI.

    Invoice Review Responsibilities Were Not Clearly Defined

    The insufficient invoice review and approval process was at least in part
    the result of a lack of clarity in the interagency agreement between FBI
    and GSA FEDSIM, as well as in FBI's oversight contract with Mitretek. We
    have identified the management of interagency contracting as a high-risk
    area, in part because it is not always clear with whom the responsibility
    lies for critical management functions in the interagency contracting
    process, including contract oversight.22

    The lack of clarity in roles and responsibilities was evident in our
    interviews with the review team, where each party indicated that they
    believed another party was responsible for a more detailed review. While
    contract management and oversight teams were identified in the interagency
    agreements, key roles and responsibilities for the review and approval of
    invoices were not clearly defined. For example, the terms and conditions
    of the interagency agreement with GSA only vaguely described GSA's role in
    contract administration. However, the agreement did not specify the
    invoice review and approval steps to be performed. Likewise, the Mitretek
    contract provided a general description of its oversight duties, but did
    not specifically mention its responsibilities related to the invoice
    review and approval process. We did note, however, that FBI did not
    approve an invoice for payment until after it was notified by Mitretek
    that it had reviewed the invoice. Based on our discussions with the review
    team, Mitretek would review its own invoices before sending them forward
    to FBI for payment approval.

    Invoices Did Not Provide Adequate Support for All Charges

    The failure to establish an effective review process was compounded by the
    fact that not all invoices provided detailed information required by the
    contracts and other information that would be needed to perform adequate
    reviews. Trilogy contractors were required to comply with various
    invoicing provisions of the FAR and the Trilogy contracts, including
    requirements to provide labor and various overhead rates, travel costs by
    trip, transaction detail for ODC, and purchase orders for equipment
    purchases. However, we found that the contractors, particularly CSC, often
    did not meet these requirements. For example:

    o CSC labor invoices did not include information related to individual
    labor rates or indicate which overhead rates were applicable to each
    employee-information needed to verify mathematical accuracy and to
    determine that the components of the labor charges were valid.

    o CSC invoices provided a summary of travel charges by category (airfare,
    lodging, etc.), but did not provide required information related to an
    individual traveler's trip costs. The travel invoices also did not provide
    cost detail by travel authorization number. Therefore, there was no way to
    determine that the trips billed were approved in advance or that costs
    incurred were proper and reasonable based on the location and length of
    travel.

    o CSC and SAIC invoices for ODC provided a summary of charges by category
    (shipping, office supplies, etc.); however, CSC did not provide required
    cost detail by transaction. In some cases, the category of charges was not
    even identified. For example, as shown in figure 3, within the ODC
    invoice, a subcategory entitled "other direct costs" made up $1.907
    million of the $1.951 million invoice current billing total. No additional
    information was provided in the invoice to explain what made up these
    "other direct costs."

    Figure 3: Example of CSC ODC Invoice

    o For purchased equipment, CSC invoices included a summary
    sheet-indicating the total price billed, a brief description of items
    purchased, and the quantity of each item purchased-and a copy of the
    related "Bill of Material" (BOM).23 However, they did not individually
    identify each asset being billed by bar code, serial number, or some other
    method that would allow verification of assets billed to assets received.
    SAIC invoices also lacked the detailed information necessary to
    individually identify assets. This severely impeded FBI's ability to
    determine whether it had actually received the assets included on invoices
    and to subsequently track individual accountable assets on an item-by-item
    basis.

    We also found that Mitretek, a member of FBI's review team, submitted
    invoices that did not include detailed information needed to perform
    adequate reviews. For example, Mitretek's invoices did not include
    individual labor rates needed to verify rates charged with salary
    information or overhead rates needed to recalculate labor costs.24 As
    previously noted, Mitretek reviewed its own invoices before sending them
    forward to FBI for payment approval.

    Even though contractor invoices, particularly those from CSC, frequently
    lacked key information needed to review charges, we found through
    inquiries with the review team and the contractors that invoices were
    generally paid without requesting additional supporting documentation.25

    Some Payments Made to Contractors Were for Questionable Costs

    Because of the lack of fundamental internal controls over the process used
    to pay Trilogy invoices, FBI was highly vulnerable to payment of
    unallowable contractor charges. In an attempt to determine the validity of
    FBI's payments, we used forensic auditing techniques, including data
    mining and document analysis, to select certain contractor costs and
    requested supporting documentation from the contractors. We identified
    about $10.1 million of questionable contractor costs paid by FBI. These
    included payments for first-class travel and other excessive airfare
    costs, incorrect billings for overtime hours, potentially excessive labor
    rates, and other questionable subcontractor costs.

    The following sections provide additional information on the payments for
    questionable costs we found. Given FBI's poor control environment and the
    fact that we only reviewed selected FBI payments to Trilogy contractors
    that we identified with data mining and other forensic auditing
    techniques, other payments for questionable costs may have been made that
    have not been identified.

    First-Class Travel and Other Excessive Airfare Costs

    During our review of CSC's supporting documentation for selected travel
    charges we found 19 first-class airline tickets purchased26 costing a
    total of $20,025, many of which exceeded the basic coach-class fares by
    significant margins.27 For example, in one case a traveler flew first
    class round trip between Providence, Rhode Island and San Francisco,
    California for $2,159. We estimated that a coach-class ticket for this
    same trip would have cost $1,119. In addition, 1 day after returning to
    Providence, this traveler flew back to San Francisco. The documentation
    provided by CSC did not explain or justify this first-class travel or
    unusual travel itinerary. The CSC contract called for airfare to be
    reimbursed to the extent allowable pursuant to the Joint Travel
    Regulations (JTR), which state that travelers must use basic economy or
    coach class unless the use of first-class travel is properly authorized
    and justified.28 Because the documentation provided by CSC for the 19
    first-class tickets costing $20,025 that we identified did not contain
    authorizations or justifications, the cost of this travel in excess of a
    coach-class ticket is potentially unallowable. Table 2 provides specific
    examples of these potentially unallowable first-class travel costs.

    Table 2: Examples of CSC's Potentially Unallowable First-Class Travel

    Source: GAO analysis of supporting documentation provided by CSC.

    aBecause historical costs for coach-class tickets were not available, we
    estimated the costs of coach-class tickets based on an average of current
    prices for a similar itinerary purchased 3 days in advance (which was
    CSC's average based on the trips we reviewed) and adjusted for inflation
    applicable to airfare.

    During our review of FBI's payments for travel costs, we also identified
    75 unusually expensive coach-class tickets that were purchased by the
    contractors for $100,847, which exceeded basic coach-class fares by
    approximately $49,848. Upon further inquiry with several airlines, we
    determined that most of these tickets were for "full fare" coach-class
    tickets. We noted that the airlines used most often by the contractors
    indicated that it is possible to obtain a free upgrade to first class with
    the purchase of the more expensive full-fare coach ticket. We found that
    in some instances, the current price of a full-fare coach ticket was
    higher than the current price of a first-class ticket. As discussed above,
    the JTR requires travelers to use basic economy or coach class unless the
    use of first-class travel is properly authorized and justified. The JTR
    defines economy class as basic accommodations that include a service level
    available to all passengers regardless of fare paid. Since full-fare coach
    tickets allow a traveler to upgrade to first class at no additional cost,
    full-fare coach class does not appear to be basic accommodations available
    to all passengers regardless of fare paid. As such, the purchase of
    full-fare coach-class tickets is a questionable cost. While the contracts
    incorporated the JTR, we determined that the JTR applies to civilian
    employees of the Department of Defense and is not considered appropriate
    "travel regulations" for contractors. The FAR, which would be appropriate
    for contractors, requires the use of the lowest customary standard, coach,
    or equivalent airfare29 and indicates that costs in excess of the lowest
    standard, coach, or equivalent airfare are unallowable. Had these
    provisions of the FAR been applied, the excessive cost of these tickets
    would have been potentially unallowable.

    We noted 62 full-fare coach tickets billed by CSC for $85,336, compared to
    an estimated cost of $41,978 for the basic fully refundable coach-class
    fares.  We also identified 6 full-fare coach tickets billed by SAIC. In
    addition, we noted 5 trips billed by SAIC for subcontractor travel with
    excessive airfare costs for which the airfare class was not included in
    the supporting documentation provided by SAIC. Therefore, we could not
    determine whether these 5 trips were first class, full-fare coach, or some
    other class of travel that exceeded basic coach-class fares. These 11
    tickets cost $11,610, compared to an estimated cost of $7,897 for the
    basic fully refundable coach-class fare. We further found 2 excessive
    airfare coach tickets billed by Mitretek that were upgraded to first
    class. These 2 tickets cost $3,901, compared to an estimated cost of
    $1,123 for the basic restricted coach-class fares.30 In total, the
    additional cost of $49,848 for the full-fare coach tickets and other
    excessive airfare are considered questionable. Table 3 provides examples
    of the excessive airfare travel costs of CSC, SAIC, and Mitretek.

    Table 3: Examples of Questionable Excessive Airfare Travel Costs

    Source: GAO analysis of supporting documentation provided by contractors.

    aBecause historical costs for coach-class tickets were not available, we
    estimated the costs of coach-class tickets based on an average of current
    prices for a similar itinerary purchased 3 days in advance (which was the
    average based on the trips we reviewed) and adjusted for inflation
    applicable to airfare.

    bThe fare basis code for this ticket indicated that a first-class upgrade
    was obtained. We could not verify whether this ticket was purchased as a
    full-fare coach or some other class of travel that exceeded the basic
    coach-class fares.

    cWe could not determine the airfare class of the ticket purchased because
    the supporting documentation provided did not include the fare basis code.

    Excess Overtime Charges

    During our review of labor charged by SAIC, we found that SAIC billed the
    Trilogy project for overtime hours worked by employees that exceeded the
    hours that would have been charged if SAIC followed the overtime policy
    informally agreed to by SAIC and FBI.31 Our calculations indicate that FBI
    may have overpaid an estimated $400,000 for these excess overtime charges.

    SAIC's task order, awarded in June 2001, stated that if work beyond the
    standard 40-hour work week was necessary to support the requirements of
    the task order, the government would not object to SAIC employees working
    an extended work week (EWW) (hours in excess of 40 per week). For
    designated EWW periods, exempt staff (professional staff normally not
    eligible for overtime compensation) would be paid a pro rata share
    (straight time) of their weekly salary based on the extended hours worked.
    EWW periods required SAIC management approval and were used when exempt
    staff were required to work extended hours for short periods of time due
    to special circumstances, such as accelerated project schedules or
    circumstances where employees could not dictate their work schedule.

    The first EWW period started August 31, 2002, and throughout the Trilogy
    project SAIC management approved 11 EWW periods for employees working on
    various Trilogy tasks. In March 2003, after the fourth EWW period started,
    SAIC implemented an EWW policy, agreed to with FBI, which decreased the
    amount of hours that would be billed to FBI. This policy stated that
    exempt staff would be compensated for hours worked that were greater than
    90 hours in a 2-week pay period on an hour-for-hour basis. That meant that
    the first 10 hours of overtime would be uncompensated. In addition, a
    ceiling of 120 hours was established, meaning that employees would not be
    compensated for hours worked in excess of 120 in a pay period. SAIC agreed
    that it would not bill FBI for this uncompensated overtime.

    During our review of employee labor billings for the Trilogy project, we
    found that SAIC employees who charged EWW time after the March 2003 policy
    frequently charged for all hours worked beyond 80 in a pay period and that
    the cost of these hours was billed to and paid by FBI. We also noted some
    instances where employees charged EWW beyond the 120-hour ceiling per pay
    period, which were also billed. We discussed this issue with SAIC
    management and they agreed that their billing of EWW costs was not
    consistent with the policy that was established in March 2003 and
    indicated that they would research the issue further to determine whether
    corrections are necessary.32

    Based on our review of the labor charges, it appears that FBI may have
    overpaid for more than 4,000 hours of EWW labor charges.33 Using average
    fully burdened labor rates for employees incorrectly billing EWW, we
    estimated that FBI may have overpaid EWW costs by approximately
    $400,000.34

    Questionable Labor Rates

    During our review of labor charged by CSC/DynCorp, we found that DynCorp
    Information Systems (DynIS), a subsidiary of DynCorp that billed about $42
    million or 94 percent of DynCorp's direct labor, charged actual labor
    rates that may have exceeded rates that GSA asserts were established
    ceiling rates pursuant to the task order. CSC asserts that ceiling rates
    were never established. If ceiling rates were established, we estimated
    that FBI overpaid CSC by approximately $2.1 million.

    When DynCorp entered into the GSA FEDSIM Millennia contract, it agreed to
    ceiling rates that would be charged for its various labor categories, such
    as clerical and senior technician. The Millennia contract also stated that
    ceiling rates applicable to subcontractors would be negotiated separately
    for each task order awarded under the Millennia contract. After entering
    into the Millennia contract, DynCorp acquired a company that was renamed
    DynIS. Because DynIS' labor rates were not considered when DynCorp's
    ceiling rates were established under Millennia, DynCorp's proposal for the
    Trilogy task order listed DynIS as a subcontractor.

    In May 2001, GSA issued a Trilogy task order award document to DynCorp
    that had a section entitled "Ceiling Rates Applicable to DynIS" that
    included the following statement: "Ceilings are placed on all labor
    category and indirect rates used to establish the total cost for this task
    order...These ceiling rates are subject to negotiation pending the results
    of [Defense Contract Audit Agency] DCAA's35 audit."

    GSA officials told us they believed that DynIS labor category hourly rates
    in DynCorp's Trilogy proposal represented established labor category
    ceiling rates. GSA officials stated that they negotiated DynIS labor
    category ceiling rates with DynCorp.36 However, CSC stated that labor
    category ceiling rates were never established because they were never
    negotiated with GSA.37

    In March 2003, CSC/DynCorp submitted and GSA approved a modification to
    the task order that, according to GSA, increased labor rates for several
    categories.38 However, CSC claims that this modification did not affect
    the ceiling rates because the ceilings were never established.

    Based on our review of DynCorp's labor invoices, we noted that several of
    DynIS' rates charged exceeded the labor rates that GSA contended were
    ceiling rates. For example, DynIS billed over 14,000 hours for work
    performed during 2001 for senior IT analysts working on the Trilogy
    project based on an average hourly rate of $106.14. However, if ceiling
    rates were established, the DynCorp proposal indicated that the Trilogy
    project would be charged a maximum of $68.73 per hour for a senior IT
    analyst working in the field or $96.24 per hour for a senior IT analyst
    working at headquarters. If ceiling rates were established, we estimated
    that FBI overpaid CSC/DynCorp by approximately $2.1 million for DynIS
    labor costs.39

    Other Questionable Costs

    We identified certain other payments to contractors that were for
    questionable costs. These costs were not supported by sufficient
    documentation to enable an objective third party to determine if each
    payment was a valid use of government funds.40 We further identified costs
    that were questionable as to whether they were necessary. Table 4
    summarizes these questionable costs, which totaled about $7.5 million.

    Table 4: Other Questionable Costs Paid by FBI

    Source: GAO analysis of contractor and subcontractor invoices and
    supporting documentation.

    Note: Examples 2 and 3, combined, represent $5.5 million of inadequately
    supported CSC ODC.

    A discussion of each of these questionable costs is provided below.

    Example 1-Subcontractor Labor Costs

    CSC did not provide us adequate supporting documentation for almost $2
    million of about $3.3 million of subcontractor labor charges we selected
    to review. The only documentation CSC could provide us for these charges
    were subcontractor invoices that lacked some of the basic information
    needed to assess the labor charges, such as the names of the subcontractor
    employees, hours billed, or individual labor rates. Therefore, CSC could
    not fully substantiate that the costs for services provided by the
    subcontractors that were charged to FBI's Trilogy project were
    appropriate.

    Example 2-Other Direct Costs/Training

    CSC hired a subcontractor, CACI, to schedule and conduct training related
    to the Trilogy project. CACI billed more than $17 million ($13 million for
    labor and $4 million for facilities, equipment rentals, and other direct
    costs) to provide FBI agents and employees basic, intermediate, and
    advanced training in Microsoft Office applications, including Word, Excel,
    PowerPoint, and Outlook. FBI officials stated that FBI decided to conduct
    off-site, hands-on training for employees (instead of internal or CD-based
    training) because of the number of employees who had limited experience
    using computers and because FBI had insufficient space to set up training
    labs at their existing facilities.

    During our review of CSC ODC, we selected $4.7 million of these training
    charges from CACI and found that CSC was unable to provide us with
    adequate support for these charges. Subsequently, we requested supporting
    documentation from CACI for selected charges totaling about $3.5 million
    of these training costs. Our examination identified the following issues:

    o CACI could not adequately support almost $3 million that it paid to one
    event planning company. Since FBI decided to conduct their training
    off-site, CACI hired an event planner, which it paid almost $3.2 million
    to reserve hotel conference rooms, rent computer equipment for training
    sessions, and set up the conference rooms for the training. The bulk of
    the $3.2 million related to one purchase order for training at 72 sites
    over 3 months, which stated that costs could not exceed $2,992,526. This
    purchase order provided for payment of 50 percent of this amount to the
    event planner at the time the purchase order was issued (to cover costs
    that include prepayments for obtaining training facilities) and four equal
    monthly payments for the remaining balance. CACI provided us with the
    purchase order, which included a description of the services to be
    performed by the event planner. They also provided us copies of invoices
    from the event planner that included general descriptions of the services
    billed. CACI could not provide any further evidence of the actual costs of
    goods or services that were provided by the event planner, such as hotel
    invoices for the rental of conference rooms.41 CACI stated that
    documentation supporting actual costs of the event planner was not
    applicable because its agreement with the event planner was "fixed
    priced." CACI stated that the payment terms in the purchase order required
    only that CACI pay the event planner a series of payments in fixed
    amounts. However, CACI's assertion that supporting documentation of actual
    costs was not applicable was not supported by the terms of the purchase
    order, which included a related statement of work that specifically
    required documentation to support costs claimed by the event planner.
    According to the statement of work, the event planner was required to (1)
    provide data on actual costs incurred twice a month, (2) make every
    attempt to obtain the best pricing with respect to all costs, and (3)
    charge CACI only for services rendered, allowing for any cost savings from
    advance payments to be returned to CACI upon request.42

    o CACI purchased about 30,000 ink pens and 30,000 highlighters for
    training sessions, at a cost of $19,705 and $32,314, respectively. The
    pens were custom made for the Trilogy training program. While there was
    supporting documentation for these costs and FBI officials stated that
    they preapproved the purchases as part of their acceptance of the Trilogy
    Pre-Training Education Plan, we question whether these purchases were
    necessary.

    Example 3-Other Direct Costs/Equipment Disposal

    CSC was unable to provide us adequate supporting documentation for
    $762,262 in equipment disposal costs billed by two subcontractors. The
    documentation provided consisted of a spreadsheet that summarized costs of
    the subcontractors, but did not include receipts or other support to prove
    that these costs were actually incurred.

    Example 4-Subcontractor Labor Invoice-Duplicate Payment

    Our review of SAIC's subcontractor labor charges found that FBI was billed
    twice for the same subcontractor invoice totaling $26,335. SAIC officials
    agreed that they double billed and stated that they would make a
    correction.

    Major Lapses in Accountability Resulted in Millions of Dollars of Missing
    Trilogy Equipment

    FBI did not adequately maintain accountability for computer equipment
    purchased for the Trilogy project. FBI relied extensively on contractors
    to account for Trilogy assets while they were being purchased, warehoused,
    and installed. However, FBI did not establish controls to verify the
    accuracy and completeness of contractor records it was relying on, to
    ensure that only the items approved for purchase were acquired by the
    contractors, and to ensure that it received all those items acquired
    through its contractors. Moreover, once FBI took possession of the Trilogy
    equipment, it did not establish adequate physical control over the assets.
    Consequently, we found that FBI could not locate over 1,200 assets
    purchased with Trilogy funds, which we valued at approximately $7.6
    million. In addition, during its physical inventory counts for fiscal
    years 2003 through 2005, FBI identified over 30 pieces of Trilogy
    equipment valued at about $167,000 that it reported as having been lost or
    stolen. Due to the significant weaknesses we identified in FBI's property
    controls, the actual amount of lost or stolen equipment could be even
    higher.

    FBI's Overreliance on Contractors Diminished Its Ability to Properly
    Account for Trilogy Assets

    FBI relied on contractors to maintain records related to the purchasing,
    warehousing, and installation of about 62 percent of the equipment
    purchased for the Trilogy project.43 FBI's primary contractor responsible
    for delivering computer equipment to FBI sites was CSC. FBI officials told
    us they met regularly with CSC and its subcontractors to discuss FBI's
    equipment needs and a deployment strategy for the delivery of equipment.
    Based on these meetings, CSC instructed its subcontractors to purchase
    equipment, which was subsequently shipped to and put under the control of
    the subcontractors. Once equipment arrived at the subcontractors'
    warehouses, they were responsible for affixing bar codes on accountable
    items-all items valued above $1,000 and certain others considered
    sensitive that are required by FBI policy to be tracked individually. In
    addition, FBI directly purchased about $19.1 million of equipment for the
    Trilogy project that was shipped directly to CSC or its subcontractors.

    When equipment was shipped from subcontractor warehouses to FBI sites, the
    shipment included two CSC subcontractor-prepared reports. The first
    report, similar to a bill of lading, included all items shipped, including
    nonaccountable items such as cables. However, there was no requirement for
    FBI officials receiving the items to verify that the items included on
    this report were actually received. The second report listed accountable
    assets that were delivered such as desktop computers, scanners, printers,
    and network equipment that were available for installation at that
    location. This report was then used by the subcontractor during the
    installation of equipment at each FBI location to prepare the "Site
    Acceptance Listing" documenting equipment that had been accepted and
    installed at the site. At the completion of the site installation, both
    FBI and subcontractor officials were required to sign this Site Acceptance
    Listing. According to FBI headquarters officials, verification of the
    subcontractor-prepared Site Acceptance Listings represented a key control
    over Trilogy equipment, providing assurance that FBI received what it
    should have. However, based on our inquiries at two field offices we
    visited, we found that FBI officials who received equipment and signed the
    Site Acceptance Listing, may not have always verified the accuracy and
    completeness of these lists.

    An official from the Baltimore field office acknowledged that he signed
    these lists without verifying that the items included had actually been
    delivered and installed at his site. In addition, officials from the
    Newark field office said they felt comfortable that they had received all
    the items they were supposed to because of their close working
    relationship with the subcontractor who performed the installation;
    however, they acknowledged that they did not independently verify
    equipment included on the contractor lists that they had signed. FBI did
    not prepare its own independent lists of ordered, purchased, or paid-for
    assets, and therefore, it had no choice but to rely solely on the
    contractor lists to account for its Trilogy assets.

    Furthermore, when FBI received shipments from contractors, it did not
    compare purchasing and billing documentation to receiving documentation to
    verify that all items purchased were received as required by FBI's
    accountable asset manual. According to FBI policy, when shipments are
    received, a designated property custodian is responsible for ensuring that
    the items received are the same as those that were ordered and for
    determining whether a complete or partial shipment was received. However,
    FBI did not require that these procedures be followed for the Trilogy
    project because purchasing and billing documentation for the project was
    not site specific; instead, the program office instructed FBI staff to
    only verify the number of boxes received and not to open the boxes to
    verify the assets received until the deployment team arrived. In addition,
    FBI did not perform an overall reconciliation of total assets ordered and
    paid for to those received. Such a reconciliation would have been made
    difficult by the fact that invoices FBI received from CSC did not include
    item-specific information-such as bar codes, serial numbers, or shipping
    location. However, failure to perform such a reconciliation left FBI with
    no assurance that it had received all of the assets it paid for.

    FBI Lacked Adequate Physical Control over Trilogy Assets

    Assets that were delivered to FBI sites by contractors were not entered
    into FBI's Property Management Application (PMA) in a timely manner,
    increasing the risk that assets could be lost or stolen without detection.
    FBI policy requires property management personnel to identify accountable
    items and enter them into PMA within 30 days of receipt. However, FBI
    officials acknowledged that Trilogy equipment had not been entered into
    PMA within 30 days, as required. We compared installation dates recorded
    in CSC's database44 of assets deployed to dates assets were recorded in
    PMA. As shown in table 5, we found that 71.6 percent of the CSC items that
    were recorded in PMA, representing 84 percent of the dollar value, were
    entered more than 30 days after receipt, contrary to FBI policy. In
    addition, 16.9 percent of the assets, representing 37 percent of the
    dollar value, were entered more than a year after receipt. When an asset
    is not recorded in the property system, there is no systematic means of
    identifying where it is located or when it is moved, transferred, or
    disposed of and no record of its existence when physical inventories are
    performed. This severely limits the effectiveness of the physical
    inventory in detecting missing assets.

    Table 5: Analysis of Time Taken by FBI to Enter CSC-Purchased Assets into
    PMA

    Source: GAO analysis of PMA records and FBI's deployment schedule.

    In an effort to identify the assets that should have been entered into
    PMA, FBI attempted to create, in 2005, an after-the-fact inventory listing
    of accountable and nonaccountable assets deployed. Because FBI had not
    prepared its own independent inventory listing of Trilogy assets ordered
    and paid for, it used the CSC-prepared list of equipment deployed as its
    basis to determine accountable assets.45 According to FBI, this list was
    supposed to include all CSC-deployed equipment that had been affixed with
    a bar code. However, FBI's ability to accurately identify accountable
    assets was hampered by its loss of control over bar codes. FBI policy
    identifies the use of bar codes as "the key control"46 for maintaining
    individual asset accountability and requires that bar codes be affixed to
    all accountable assets. Despite the importance of maintaining a reliable
    bar code system, FBI relied on contractors to affix the bar codes, but
    then did not track the bar code numbers given to contractors, the bar code
    numbers they used, or the bar code numbers returned. Moreover, FBI
    provided incorrect instructions to contractors, initially directing them
    to bar code certain types of nonaccountable computer pieces.47

    An FBI official stated that when creating its after-the-fact listing of
    accountable and nonaccountable assets from the CSC listing, FBI tried to
    identify and list as nonaccountable those items that had been mistakenly
    bar coded. However, we found that FBI's accountable asset listing still
    included some nonaccountable assets that had been bar coded in error.
    Further, we noted that FBI's listing of nonaccountable assets incorrectly
    included some accountable items such as uninterruptible power supplies and
    network switches.48 As a result, FBI could not reliably determine the
    complete universe of Trilogy assets that should have been bar coded and
    designated as accountable property to be tracked separately by PMA.

    We also compared FBI's after-the-fact listing of accountable assets
    identified from the CSC-prepared listing to the asset records in FBI's
    PMA. We found that FBI's listing and or PMA included several errors and
    omissions in the listings, including:

    o accountable assets for which there was no listed bar code or serial
    number;

    o incorrect bar codes (for example, text bar codes or bar codes with too
    many digits);

    o items for which locations were listed as "unknown";

    o assets with the same bar code with different serial numbers and/or
    locations;

    o incomplete and inaccurate asset descriptions;

    o items that matched to PMA by bar code but not by serial number; and

    o items that matched to PMA by serial number but not by bar code.

    The FBI official who prepared the accountable asset listing said he gave
    this listing to each site with instructions to ensure that all of the
    assets had been entered into PMA in preparation for a 2005 physical
    inventory count. However, FBI did not follow up to determine whether all
    of the records in the inventory listing were actually entered into PMA.
    For site officials using the listing, the lack of complete and accurate
    information included in the inventory listing may have limited their
    ability to track some of the assets and ensure they were accounted for in
    PMA.

    FBI policy requires complete physical inventories of all accountable
    assets at least once every 2 years. Annually, a complete physical
    inventory of all accountable assets that are also capitalized assets
    (i.e., those with an acquisition cost of $25,000 or more) and "sensitive"
    property (e.g., laptop computers and weapons which are susceptible to
    theft) is performed. FBI's most recent biennial inventory of accountable
    assets occurred in the spring of 2005. To complete its inventory, FBI used
    scanner technology, directing employees responsible for performing the
    inventory to scan all items found at FBI locations that contained a bar
    code. PMA was updated to reflect the items that were located and scanned
    during the inventory and generated reports to identify new accountable
    assets that were not previously entered in the system.49 However, FBI did
    not compare the results of its inventory to its listing of accountable
    assets purchased under Trilogy to ensure that all of these assets were
    actually located during the inventory. Failing to perform this elemental
    step undermines the fundamental purpose of conducting physical
    inventories.

    FBI Is Unable to Locate Millions of Dollars of Trilogy Assets

    Given that FBI did not ensure that all accountable Trilogy assets that
    should have been in its possession (i.e., those it paid for) were located
    during the physical inventory, we undertook several procedures in an
    attempt to do so. To perform this test work, we used FBI's inventory
    listing of CSC-purchased accountable equipment as well as similar FBI
    listings of assets FBI purchased directly (government furnished equipment
    or GFE) and that were purchased by SAIC. Although FBI's inventory listing
    of CSC-purchased accountable equipment included inaccurate and incomplete
    information, as previously discussed, we were able to reconcile the total
    number of items for selected types of equipment from its listing of
    accountable CSC-purchased equipment to the number of these assets invoiced
    by CSC. This provided some assurance that the listing of accountable
    CSC-deployed equipment purchased by both CSC and FBI for those asset types
    includes all accountable assets FBI paid for and that should be in FBI's
    possession. This was done for selected CSC-purchased accountable assets,50
    which represented approximately 76 percent of the total number of
    CSC-purchased equipment, and all SAIC-purchased assets. Therefore, we used
    these asset listings to determine whether accountable assets were located
    during FBI's most recent physical inventory.

    We obtained several iterations of PMA listings and inventory reports from
    FBI and attempted to trace the assets to these reports. Collectively,
    these listings and reports should have included all accountable Trilogy
    assets in FBI's possession at the time of its 2005 inventory. Based on
    this comparison, we identified 1,205 accountable Trilogy assets, with an
    estimated value of approximately $7.6 million that FBI has been unable to
    locate or otherwise account for. We estimated this value using the lowest
    per-unit-cost based on the Trilogy equipment-pricing sheets that were
    prepared by FBI and used in recording the cost of the same types of assets
    in PMA. If we could not identify a price for a certain type of accountable
    asset in FBI's equipment-pricing sheets, we identified the lowest price on
    the accountable and capitalized assets spreadsheet prepared by FBI's
    finance division. When the cost was not available on either of these
    documents, or when the item was unknown, we did not attempt to estimate
    the asset's value. As a result, our estimated value of lost or stolen
    equipment does not include 103 of the 926 CSC-purchased items we
    identified, such as Paradyne frame savers and Optical HBA drivers, and
    therefore is understated. Table 6 provides a description and estimated
    value for the assets for which we could identify unit cost.

    Table 6: Trilogy-Purchased Items Not Located by FBI

    Source: GAO analysis of FBI listings of accountable assets and PMA
    reports.

    aThis equipment includes electronic testing tools used to ensure proper
    installation of network equipment.

    bAn uninterrupted power source is a constantly charging battery pack which
    powers the computer.

    As of November 30, 2005, FBI was unable to sufficiently explain why these
    items were not accounted for in PMA and/or could not provide adequate
    documentation that the assets had been located. An FBI official stated
    that some of the assets included in the listing of CSC-purchased equipment
    would not be expected to be in PMA because some were replaced. For
    example, according to the official, some of the CSC-purchased switches
    were replaced due to a heating malfunction. However, FBI did not provide
    us with documentation related to replaced items, and therefore we could
    not determine which units if any were replaced and/or which units should
    still be on hand.

    The FBI official also told us that, even though he attempted to remove all
    nonaccountable items from the listing of CSC-purchased equipment, some
    nonaccountable items may still have been included. For example, FBI told
    us some purchased components that were a part of an accountable asset unit
    may have been bar coded even though the item by itself was not an
    accountable item. Using FBI guidance on accountable property,51 we
    determined that 103, or about 11.1 percent, of the missing 926
    CSC-purchased assets may represent nonaccountable units. Because FBI was
    unable to provide us with location information for these items, we could
    not definitively determine whether they represent nonaccountable
    components or are separate accountable assets that were not in PMA and
    could not be located. FBI had no further explanation for why it could not
    locate the missing assets we identified or whether the missing assets we
    identified may expose confidential and sensitive information and data to
    unauthorized users.

    In addition to the missing items discussed above, FBI could not initially
    locate another 25 purchased assets-highly-sensitive encryption
    equipment-in its PMA system. Subsequently, FBI officials were able to
    provide the bar codes, locate the encryption equipment,52 and provide
    evidence that all of the items were now in its PMA system.53 The officials
    stated the equipment was not originally required to be bar coded or
    tracked in PMA, but that it was tracked several different ways by serial
    number. The officials also explained that the problem resulted mostly from
    FBI modifications to the equipment that required revisions to the serial
    numbers listed in the invoices. Regardless of the fact that the equipment
    was subsequently located after research and inquiries, such highly
    sensitive equipment needs to be properly and timely accounted for to
    ensure the precise location of the equipment can be immediately determined
    at all times.

    In addition to the items we found missing, FBI's property management
    division reported 37 CSC-purchased Trilogy assets, totaling approximately
    $167,000, that were determined to be lost or stolen during its physical
    inventory counts for fiscal years 2003 through 2005.54 The assets reported
    as lost or stolen included computers and servers, which may have contained
    sensitive and confidential information. According to FBI policy, for items
    in PMA that cannot be located during the inventory, a "Report of Lost or
    Stolen Property" must be submitted to FBI headquarters. Due to security
    concerns, FBI did not provide us copies of these reports for the property
    items that were not located during the 2003, 2004, and 2005 inventories.
    Therefore, it is unclear what type of security risk if any these
    lost/stolen assets represent.

    Conclusions

    FBI's Trilogy IT project spanned 4 years and the reported costs exceeded
    $500 million. Our review disclosed that there were serious internal
    control weaknesses over the process used by FBI and GSA to approve
    contractor charges related to Trilogy, which made up the vast majority of
    the total reported project cost. While our review focused specifically on
    the Trilogy program, the significance of the issues identified during our
    review may be indicative of more systemic contract and financial
    management problems at FBI and GSA, in particular when using
    cost-reimbursable type contracts and interagency contracting vehicles.
    These weaknesses resulted in the payment of millions of dollars of
    questionable contractor costs, which may have unnecessarily increased the
    overall cost of the project. Unless FBI strengthens its controls over
    contractor payments, its ability to properly control the costs of future
    projects involving contractors, including its new Sentinel project, will
    be seriously compromised. Additionally, to the extent that GSA enters into
    similar interagency agreements, it will continue to be exposed to
    oversight lapses until it reassesses its procedures. Further, weaknesses
    in FBI's controls over the equipment acquired for Trilogy resulted in
    millions of dollars in missing equipment, and call into question FBI's
    ability to adequately safeguard its equipment, as well as confidential and
    sensitive information that could be accessed through that equipment from
    unauthorized use.

    Recommendations for Executive Action

    We are making the following 27 recommendations to the Director of FBI and
    the Administrator of General Services to (1) facilitate the effective
    management of interagency contracting, (2) mitigate the risks of paying
    unallowable costs in connection with cost-reimbursement type contracts,
    and (3) improve FBI's accountability for and safeguarding of its computer
    equipment.

    To improve FBI's controls over its review and approval process for
    cost-reimbursement type contract invoices, we recommend that the Director
    of FBI instruct the Chief Financial Officer to establish policies and
    procedures so that:

    o Future interagency agreements establish clear and well-defined roles and
    responsibilities for all parties included in the contract administration
    process, including those involved in the invoice review process, such as
    contracting officers, technical point of contacts, contracting officer's
    technical representatives, and contractor personnel with oversight and
    administrative roles.

    o Appropriate steps are taken during the invoice review and approval
    process for every invoice cost category (i.e., labor, travel, other direct
    costs, equipment, etc.) to verify that the (1) invoices provide the
    information required in the contract to support the charges, (2) goods and
    services billed on invoices have been received, and (3) amounts are
    appropriate and in accordance with contract terms.

    o The resolution of any questionable or unsupported charges on contractor
    invoices identified during the review process is properly documented.

    o Labor rates, ceiling limits, treatment of overtime hours, and other key
    terms for cost determination are clearly specified and documented for all
    contracts, task orders, and related agreements.

    o Future contracts clearly reflect the appropriate Federal Acquisition
    Regulation travel cost requirements, including the purchase of the lowest
    standard, coach, or equivalent airfare.

    o An appropriate process is in place to assess the adequacy of
    contractor's review and documentation of submitted subcontractor charges
    before such charges are paid by FBI.

    In light of the findings in this report, we recommend that the
    Administrator of General Services instruct the director of FEDSIM to
    reassess its procedures in connection with (1) interagency contracts and
    (2) delegated contract administration responsibilities, including the
    following:

    o Clearly defining the roles and responsibilities of each party in
    interagency agreements, and particularly those related to reviewing and
    approving invoices.

    o Assessing the adequacy of its invoice review and approval polices,
    including specific steps to be performed by each party so that (1)
    invoices provide the information required in the contract to support the
    charges, (2) goods and services billed on invoices have been received, (3)
    amounts are appropriate and in accordance with contract terms, and (4) the
    resolution of any questionable or unsupported charges on contractor
    invoices identified during the review process is clearly documented.

    o Clearly documenting labor rates, ceiling limits, treatment of overtime
    hours, and other key terms for cost determination for all contracts, task
    orders, and related agreements.

    o Clearly reflecting in future contracts the appropriate Federal
    Acquisition Regulation travel cost requirements, including the purchase of
    the lowest standard, coach, or equivalent airfare.

    o Confirming that contractors properly review and support submitted
    subcontractor charges.

    To address issues on the Trilogy project that could represent
    opportunities for recovery of costs, we recommend that the Administrator
    of General Services, in coordination with the Director of FBI:

    o Confirm SAIC's informal Extended Work Week policy and work with SAIC to
    determine and resolve any overpaid amounts.

    o Further investigate whether DynIS' labor rates exceeded ceiling rates
    and pursue recovery of any amounts determined to have been overpaid.

    o Determine whether other contractor costs identified as questionable in
    this report should be reimbursed to FBI by contractors.

    o Consider engaging an independent third party to conduct follow-up audit
    work on contractor billings, particularly areas of vulnerability
    identified in this report.

    To improve FBI's accountability for purchased assets, we recommend that
    the Director of FBI instruct the Chief Financial Officer to:

    o Establish policies and procedures so that (1) purchase orders are
    sufficiently detailed so that they can be used to verify receipt of
    equipment at FBI sites, and (2) contractor invoices are formatted to tie
    directly to purchase orders and facilitate easy identification of
    equipment received at each FBI site.

    o Reinforce existing policies and procedures so that when assets are
    delivered to FBI sites, they are verified against purchase orders and
    receiving reports. Copies of these documents should be forwarded to FBI
    officials responsible for reviewing invoices as support for payment.

    o Establish policies and procedures so that invoices are paid only after
    all verified purchase order and receipt documentation has been received by
    FBI payment officials and reconciled to the invoice package.

    o Establish a policy to require that, upon receipt of property at FBI
    sites, FBI personnel immediately identify all accountable assets and affix
    bar codes to them.

    o Revise FBI's policies and procedures to require that all bar codes are
    centrally issued and tracked through periodic reconciliation of bar codes
    issued to those used and remaining available. Assigned bar codes should
    also be noted on a copy of the receiving report and forwarded to FBI's
    Property Management Unit.

    o Revise FBI policies and procedures to require that accountable assets be
    entered into PMA immediately upon receipt rather than within the current
    30-day time frame.

    o Require officials inputting data into PMA to enter (1) the actual
    purchase order number related to each accountable equipment item bought,
    (2) asset descriptions that are consistent with the purchase order
    description, and (3) the physical location of the property.

    o Establish policies and procedures related to the documentation of
    rejected or returned equipment so that the (1) equipment that is rejected
    immediately upon delivery is notated on the receiving report that is
    forwarded to FBI officials responsible for invoice payment; and (2)
    equipment that is returned after being accepted at an FBI site (e.g.,
    items returned due to defect), is annotated in PMA, including the serial
    number and location of any replacement equipment, under the appropriate
    purchase order number.

    o Reassess overall physical inventory procedures so that all accountable
    assets are properly inventoried and captured in the PMA system and that
    all unlocated assets are promptly investigated.

    o Expand the next planned physical inventory to include steps to verify
    the accuracy of asset identification information included in PMA.

    o Establish an internal review mechanism to periodically spot check
    whether the steps listed above-including verifications of purchase orders
    and receiving reports against received equipment, immediate identification
    and bar coding of accountable assets, maintenance of accurate asset
    listings, prompt entry of assets into PMA, documentation of rejected and
    returned equipment, and improved bar coding and inventory procedures-are
    being carried out.

    o Investigate all missing, lost, and stolen assets identified in this
    report to (1) determine whether any confidential or sensitive information
    and data may be exposed to unauthorized users; and (2) identify any
    patterns related to the equipment (e.g., by location, property custodian,
    etc.) that necessitates a change in FBI policies and procedures, such as
    assignment of new property custodians or additional training.

    Agency Comments and Our Evaluation

    In written comments reprinted in appendix III, FBI stated that it
    concurred with our recommendations and that it has made and continues to
    make significant structural and procedural changes to address our
    recommendations, taking critical steps to strengthen internal controls.
    FBI also provided additional information related to Trilogy assets we
    identified as missing. In written comments reprinted in appendix IV, GSA
    stated that it accepted our recommendations, did not believe that 1 of
    them was needed, and described some of the improvements to its internal
    controls and other business process changes already implemented. GSA also
    expressed concern with some of our observations and conclusions related to
    the invoice review and approval process and our analysis of airfare costs.
    FBI and GSA also provided technical comments, which we have incorporated
    as appropriate.

    In its comments, FBI stated that executive management at FBI has directed
    a sustained effort to address and correct weaknesses identified in our
    report and other Trilogy reviews. FBI further stated that attention is
    being focused on four areas: (1) audit capability, (2) property
    management, (3) contracting services, and (4) IT investments. If properly
    implemented, the activities outlined in FBI's letter should help improve
    FBI accountability for future IT acquisitions and other contract services.
    In this regard, vigilant oversight will be needed to ensure controls are
    correctly designed and operating effectively to protect assets and prevent
    improper payments.

    Further, in its comments, FBI stated that more than 44,000 pieces of
    accountable property were successfully deployed and tracked in the FBI's
    PMA during the Trilogy project. FBI also stated that the 1,404 items we
    initially reported as missing or improperly documented represented
    approximately 3 percent of the accountable assets. We question both of
    these statements. Because of the control weaknesses discussed in our
    report, FBI does not have a reliable basis to know the number of Trilogy
    assets it purchased or how many should have been tracked as accountable
    assets. Further, since we did not test all the assets purchased, more may
    be missing.

    FBI also stated that as of January 2006, it had accounted for more than
    1,000 of the 1,404 items we reported as missing or improperly documented.
    During our agency comment period, FBI indicated that it found 237 items we
    previously identified as missing and provided evidence, not made available
    during our audit, to sufficiently account for 199 of these items. We
    adjusted the missing assets listing in our report to reflect 1,205 (1,404
    - 199) assets as still missing. In February 2006, FBI informed us that the
    approximately 800 remaining items noted in its official agency response
    included (1) accountable assets not in PMA because they were either
    incorrectly identified as nonaccountable assets or mistakenly omitted, (2)
    defective accountable assets that were never recorded in PMA and
    subsequently replaced, and (3) nonaccountable assets or components of
    accountable assets that were incorrectly bar coded.

    We considered these same issues during our audit and attempted to
    determine their impact. For example, as stated in our report, FBI told us
    that components of some nonaccountable assets that were part of a larger
    accountable item may have been mistakenly bar coded. Using FBI guidance on
    accountable property, we determined that 103 or about 11 percent of the
    926 missing assets purchased by CSC may have represented nonaccountable
    components. Because FBI could not provide us with the location
    information, we could not definitively determine whether the items were
    accountable assets or not. During the course of our audit, FBI was not
    able to provide us with any evidence to support their other statements
    regarding the reasons the assets could not be located. While we are
    encouraged by FBI's current efforts to account for these assets, its
    ability to definitively determine their existence has been compromised by
    the numerous control weaknesses identified in our report. Further, the
    fact that assets have not been properly accounted for to date means that
    they have been at risk of loss or misappropriation without detection since
    being delivered to FBI-in some cases for several years.

    While GSA said it accepted all of our recommendations, it expressed
    reservations regarding our recommendation that GSA should clearly reflect
    appropriate FAR travel cost requirements in future contracts. GSA stated
    in its comment letter that it believed that the requirements outlined in
    the applicable FAR section 31.205-46 and stipulated in the task orders
    were more than adequate. In a subsequent conversation, we asked the GSA
    contracting officer why the language in the CSC and SAIC task orders and
    the Mitretek contract, which stated that long-distance travel would be
    reimbursed to the extent allowable pursuant to the JTR, was considered
    appropriate by GSA. The GSA contracting officer stated that, while not
    specified in the contract language, the reference to the JTR related only
    to per diem rates and allowances when determining the reasonableness of
    the travel costs, such as lodging and mileage reimbursements. She further
    stated that the FAR would apply to all other travel reimbursement
    determinations.

    We do not agree that our recommendation is unnecessary. In our view, the
    references to the JTR create ambiguity. The FAR cost allowability clause
    52.216-7 states that when determining allowability, in addition to FAR
    cost principles, the terms of the contract also apply. Therefore, the
    reference to the allowability under the JTR could have caused confusion
    with the contractors regarding what long-distance travel costs were
    allowed, including airfare costs. We continue to believe that the task
    orders should have more clearly described the applicable travel
    requirements.

    Regarding the invoice review and approval process, GSA stated that each
    member of the review team-FBI, GSA, and Mitretek-played a unique and
    mutually understood role. In particular, GSA stated that Mitretek's role
    in the invoice review and approval process was significant and that it was
    reasonable for GSA to have relied on input from FBI, via Mitretek, in
    approving invoices for payment. GSA also referred to procedures to
    preapprove ODC and equipment purchases. Further, GSA stated that it
    believed that the procedures to process invoices were generally sound and
    that contractors are required to maintain records to adequately
    demonstrate that costs claimed have been incurred, and are reasonable,
    allowable, and allocable. GSA also stated that it will have DCAA audit the
    contract costs to determine if any costs are unallowable, unreasonable, or
    unallocable and will use the audit results as a basis to pursue remedies
    to recoup funds and assess penalties as may be applicable.

    We disagree with GSA regarding review team roles and the review process.
    Based on discussions with members of the review team, our review of
    supporting documentation, and our assessment of the outcomes of the review
    process, it is clear that the invoice review and approval process was
    inadequate. The roles and responsibilities of the review team members were
    not clearly defined or documented and this led to confusion among the
    review team members about each member's role. Regarding Mitretek's role,
    Mitretek officials stated that they performed a limited review of only
    labor invoices. Before relying on others, GSA should have verified its
    understanding of each member's roles and responsibilities and confirmed
    that the appropriate functions were being performed. In addition, while
    there were procedures to preapprove ODC and equipment purchases, the
    review team did not effectively link the preapproval and the invoice
    review and approval processes, especially in relation to CSC, in part
    because CSC invoices lacked detailed information needed to verify that
    charges billed were in fact preapproved.

    Further, while contractors are required to maintain records to adequately
    support costs, we found that the review team generally did not request
    additional documentation such as travel vouchers or subcontractor invoices
    to support amounts billed. If the review team had a systematic process in
    place to review costs, it may have questioned some of the excessive
    airfare we identified and found, as we did, that CSC lacked documentation
    to adequately support subcontractor charges. It is a management function
    and sound business practice to have a process in place to ensure that
    contractors have such documentation. Having such processes and questioning
    amounts billed would also allow for corrective measures to be implemented
    as, and if, problems were found. In addition, while we agree that a DCAA
    audit of contract costs can provide a detective control to help determine
    whether contractor costs were proper, reliance on an after-the-fact audit
    is not an acceptable replacement for the type of real-time monitoring and
    oversight of contractor costs-preventative controls-we recommend in this
    report. Further, a DCAA audit of civilian contractor costs is not
    automatic and would require an additional cost to the government to
    procure. The review team largely operated in an environment of trust
    without an adequate basis for knowing whether the contractor billings were
    reasonable and costs claimed were allowable. Effective internal control
    calls for a sound, on-going invoice review and approval process as the
    first line of defense in preventing unallowable costs.

    Regarding our analysis of travel costs, GSA stated that our conclusions
    did not account for the ever-changing travel schedules and itineraries
    necessitated by changes in FBI requirements. GSA also stated that a
    hypothetical standard coach-class ticket does not provide a benchmark to
    make a valid price comparison, even if adjusted for inflation, because the
    airline travel industry has had significant changes with respect to
    pricing of airline tickets. GSA also stated that they believe it is
    impossible at this date to look back over 5 years and estimate what may
    have been a reasonable airfare price.

    We disagree. Our analysis and conclusions related to travel did take into
    account the possible conditions that could justify airfare costs in excess
    of the lowest customary coach-class fare. The FAR requires supporting
    documentation for first-class and other excessive airfare costs of the
    nature we identified to justify the higher airfare costs. No such
    documentation was provided to us to justify the excessive costs we
    identified. To estimate the cost of the coach-class tickets, we assumed
    that tickets were purchased 3 days in advance (which was the average based
    on the trips we reviewed) and did not include a Saturday night stay over.
    Specifically, we (1) used the Web sites of the airlines used by each
    traveler, (2) searched for standard fully-refundable55 coach-class tickets
    with the same destinations, (3) calculated an average cost based on the
    lowest and highest ticket prices available at the time of our search, and
    (4) adjusted the average cost for inflation applicable to airfare. We
    believe that this approach, which closely approximated what travelers were
    doing at that time, resulted in reasonable estimates as to how much the
    travel should have cost. We also believe that adjusting current fares for
    inflation applicable to airfare results in a reasonable benchmark to
    compare to historical prices, since it does take into account price
    changes as a result of changes in the airline industry, including the
    effects of competition. Lastly, we fully agree with GSA that the passage
    of time makes it difficult to determine historical airfare costs, which is
    another reason that costs should be reviewed real time instead of as part
    of an after-the-fact audit. An after-the-fact analysis is no substitute
    for the contemporaneous monitoring and oversight that we recommend in this
    report.

    More specific discussions are provided following GSA's comments, which are
    reprinted in appendix IV.

    As agreed with your offices, unless you publicly announce its contents
    earlier, we plan no further distribution of this report until 30 days from
    its date. At that time, we will send copies of this report to the Director
    of the FBI, the Acting Administrator of GSA, and interested congressional
    committees. Copies will also be made available to others upon request. In
    addition, the report will be available at no charge on the GAO Web site at
    http://www.gao.gov .

    If you or your staffs have any questions about this report, please contact
    me at (202) 512-9508 or calboml@gao.gov . Contact points for our offices
    of Congressional Relations and Public Affairs may be found on the last
    page of this report. Major contributors to this report are acknowledged in
    appendix V.

    Linda M. Calbom Director, Financial Management   and Assurance

    Key Trilogy Milestones Appendix I

    The Federal Bureau of Investigation's (FBI) Trilogy project experienced
    several delays, as shown in figure 4.

    Figure 4: Key Trilogy Milestones

    Scope and Methodology Appendix II

    To determine whether the Federal Bureau of Investigation's (FBI) internal
    controls provided reasonable assurance that improper payments to
    contractors would not be made or would be detected in the normal course of
    business, we used the Standards for Internal Control in the Federal
    Government, Guide for Evaluating and Testing Controls Over Sensitive
    Payments, and The Executive Guide on Strategies to Manage Improper
    Payments: Learning from Public and Private Sector Organizations as a basis
    for assessing FBI's internal control structure over its Trilogy program.
    We also reviewed our prior reports, as well as those by the Department of
    Justice's Office of Inspector General on Trilogy issues; Trilogy contracts
    and interagency agreements; and contractor invoices and other
    documentation supporting goods provided and services rendered. In
    addition, we conducted interviews with officials from FBI, the General
    Services Administration, the Department of the Interior, and the
    contractors, and performed walkthroughs to gain an understanding of the
    processes used to review and approve invoices.

    Validity of Payments

    To determine whether FBI's payments to contractors were properly supported
    as a valid use of government funds, we performed data mining, document
    analysis, and other forensic auditing techniques to select transactions to
    test. We reviewed documentation maintained by the review team,
    contractors, or subcontractors to assess the allowability of costs based
    on Trilogy contract documents and applicable federal regulations, such as
    the Federal Acquisition Regulation, Federal Travel Regulation, and Joint
    Travel Regulations. While we identified some payments for questionable
    costs, our work was not designed to identify all questionable payments or
    to estimate their extent. The following provides more details on our
    testing of payments to FBI's contractors-Science Applications
    International Corporation (SAIC), Computer Sciences Corporation (CSC), and
    Mitretek-for labor, subcontractor labor, travel, and other direct costs
    (ODC).

    Payments to SAIC

    o To test payments for labor costs, we obtained from SAIC a database of
    hours charged to the Trilogy project by employee and pay period. Using
    this database and labor invoice detail, we selected 21 employees based on
    either (1) a high number of hours worked, (2) a high dollar amount billed,
    or (3) billing in more than one labor category. For these 21 employees, to
    test the labor rates billed, we compared rates billed to salary
    information. In addition, for subsets of these 21 employees, we compared
    the hours billed to hours reported in SAIC's labor database and tested the
    mathematical accuracy of the labor costs billed. To determine the
    reasonableness of extended work week (EWW) hours charged to the Trilogy
    project, using the database we analyzed the total, EWW, and uncompensated
    hours charged by employee. We also compared the average fully burdened
    labor rates charged to ceiling rates to determine whether the rates were
    below the ceilings.

    o To test payments for subcontractor labor costs, we obtained from SAIC a
    database of all subcontractor labor charges. In order to determine whether
    the database was complete, we verified that the database reconciled with
    SAIC's subcontractor billings. We then selected subcontractor invoices to
    review based on a high dollar amount billed or unusual billing patterns.
    We analyzed supporting documentation such as subcontractor invoices and
    time sheets from SAIC for about $17.2 million, or 37 percent, of payments
    for SAIC subcontractor labor.

    o To test payments for travel costs, using detail included in SAIC's
    travel invoices and copies of travel authorizations provided by SAIC, we
    selected transactions to review based on (1) high airfare costs, (2)
    actual costs that exceeded authorized amounts, and (3) unusual billing
    patterns. We analyzed supporting documentation, such as travel vouchers,
    receipts, and subcontractor invoices, from SAIC for about $154,000, or 45
    percent, of payments for SAIC travel costs.

    o To test payments for ODC, using detail included in SAIC's ODC invoices,
    we selected transactions with unusually large amounts within a category or
    with an unusual category description. We analyzed supporting
    documentation, such as invoices or other documentation, from SAIC for
    about $307,000, or 61 percent, of payments for SAIC ODC.

    Payments to CSC

    Because CSC was unable to readily provide us transaction-level detail for
    all labor, travel, and ODC charges, we selected 11 invoices based on the
    amounts billed and the time periods covered. CSC was able to provide us
    transaction-level detail for these 11 invoices, which represented $14.7
    million or about 33 percent of labor costs;1 $3.1 million or about 33
    percent of travel costs; and $2.4 million or about 27 percent of ODC
    charges. Using these 11 invoices as our data source we performed the
    following tests of CSC labor, travel, and ODC.

    o We recalculated the total labor charged for three labor categories in 7
    of the 11 invoices to verify that the invoice amounts were calculated
    correctly.2 We also selected 11 employees based on either (1) high number
    of hours worked, (2) a high dollar amount billed, or (3) billing in more
    than one labor category. For these 11 employees, we compared the hours
    billed to time sheets and verified hourly rates by reviewing each
    employee's salary history. In total, the 11 selected employees billed
    around $850,000 on the 11 invoices we reviewed. We tested the reliability
    of the detail provided by comparing the hours and amounts to labor
    invoices. We compared the average fully burdened rates charged to ceiling
    rates.

    o We selected travel charges that were high in amount or exhibited unusual
    billing patterns. We reviewed travel vouchers for these selected charges.
    Because we identified possible first-class and unusual coach-class travel
    in these selections, we obtained and reviewed additional supporting
    documentation for CSC-purchased airline tickets beyond the initial 11
    invoices selected for review.

    o We selected ODC transactions with unusually large amounts within a
    category or in an unusual category (such as computer hardware). Because of
    anomalies we identified in our initial review, we selected additional
    transactions to review beyond the initial 11 invoices. In total, we
    analyzed supporting documentation for about $7.0 million, or about 80
    percent, of payments for CSC ODC during the Trilogy project.

    o To test payments for subcontractor labor costs, we obtained from CSC
    transaction-level detail for 12 of its subcontractors during the Trilogy
    project. From the transaction-level detail, we selected charges to review
    based on (1) high number of hours worked, (2) a high amount billed, and
    (3) other unusual billing patterns. We obtained and analyzed supporting
    documentation, such as subcontractor invoices, from CSC for about $3.3
    million, or 4 percent, of the $75 million charged by CSC as subcontractor
    labor costs during the Trilogy project.3

    Payments to Mitretek

    o To test payments for labor costs, we obtained transaction detail for
    three labor invoices, which represented $1.5 million or 8 percent of the
    payments for Mitretek labor. We tested the mechanical accuracy of the
    invoice calculation and selected one of the invoices and verified hours
    billed compared to time cards and hourly rates charged compared to salary
    histories.

    o To test payments for travel costs, we obtained and analyzed the
    supporting documentation, such as travel vouchers, for all travel costs on
    two invoices. These invoices represented $11,211 or about 13 percent of
    payments to Mitretek for travel costs.

    o To test payments for ODC,4 we obtained and analyzed the supporting
    documentation, such as invoices and receipts, for all ODC costs on two
    invoices. These invoices represented $139,083 or about 8 percent of
    payments to Mitretek for ODC.

    FBI's Asset Accountability

    To determine whether FBI maintained proper accountability for assets
    purchased with Trilogy project funds, we used our Standards for Internal
    Control as a basis to assess FBI's control structure over its Trilogy
    assets. We interviewed FBI, contractor, and subcontractor staff to
    identify and assess the controls in place over the ordering, purchasing,
    and receipt of Trilogy equipment. The following provides more details on
    our testing of Trilogy equipment purchased for FBI by CSC and SAIC, or
    directly by FBI:

    o To determine whether FBI approved for purchase all assets acquired for
    the Trilogy project, we obtained FBI consents to purchase, Bills of
    Material, and invoices and compared the total assets approved to be
    purchased to assets actually purchased.

    o To determine whether FBI Trilogy accountable assets listed in PMA were
    recorded in a timely manner, we obtained documentation from FBI and
    contractors for accountable assets purchased by CSC that identified the
    bar codes assigned to accountable assets and the date the equipment was
    received by FBI. We did not perform this test for SAIC-purchased assets
    because the assets represented only .8 percent of the total assets
    purchased with Trilogy funds. We also did not perform this test for FBI
    direct purchases since the supporting documentation did not provide bar
    codes or serial numbers for individual assets. We compared the bar codes
    on the listings to FBI's Property Management Application (PMA) which
    included the date the asset was entered into PMA.

    o To assess the accuracy and completeness of the FBI-prepared listings of
    CSC- and SAIC-purchased assets, we (1) analyzed the listings to identify
    any irregularities such as duplicate bar codes or missing information; (2)
    obtained the CSC equipment invoices and compared the total number of
    pieces billed on the CSC invoices for four selected accountable asset
    types that represented about 76 percent of the total CSC assets purchased
    to FBI's listing; and (3) obtained the SAIC listings of Trilogy equipment
    returned to FBI, SAIC's equipment invoices, and FBI's listing of VCF
    assets and compared for each item the amount of equipment per the invoices
    to the SAIC listing and then to FBI's VCF listing.

    o To determine whether FBI had in its possession all accountable assets
    purchased for it by CSC and SAIC, we compared the complete listing of bar
    codes from FBI's VCF and CSC listings to PMA to identify any bar codes not
    recorded in PMA.

    o To test the accuracy of the data included in the PMA accountable asset
    records, we compared the data for each accountable asset, such as bar code
    number, serial number, asset description, and asset location, to FBI's
    listing and followed up on any discrepancies.

    o To identify Trilogy assets that had been reported as lost or stolen by
    FBI, we obtained a listing of all assets identified as lost or stolen by
    FBI during its annual inventories for years 2003, 2004, and 2005. We then
    compared this listing, by bar code, to FBI's CSC and VCF equipment
    listings to determine which of these assets had been acquired for the
    Trilogy project.

    The scope of our review covered all assets purchased from the inception of
    the Trilogy contracts (May 2001) through December 2004 and included
    Trilogy assets that were either purchased directly by FBI or by one of the
    two primary Trilogy contractors, CSC and SAIC.

    We provided FBI a draft of this report and GSA a draft of applicable
    sections of this report for review and comment. The FBI Finance Division
    Acting Assistant Director and General Services Acting Administrator
    provided written comments, which are reprinted in appendixes III and IV,
    respectively. FBI and GSA also provided technical comments, which we have
    incorporated as appropriate. We also discussed with Trilogy contractors
    any findings that related to them. We performed our work in accordance
    with generally accepted government auditing standards in Washington, D.C.
    and at two FBI field sites and various other GSA and contractor locations
    in Virginia from May 2004 through December 2005.

    Comments from the Federal Bureau of Investigation Appendix III

    Comments from the General Services Administration Appendix IV

    1.We referred to the Department of Justice Office of Inspector General
    report only to provide background information related to previously
    reported issues with the Trilogy project.

    2.See "Agency Comments and Our Evaluation" section.

    3.Processing invoices timely as envisioned by the Prompt Payment Act does
    not lessen the government's responsibility to verify costs billed by
    contractors. It is conceivable that the essential validation work could
    have been performed immediately after payment and any adjustments to
    correct prior billing errors could have been made to future invoices.

    4.No documentation of any such inquiries was provided to support the
    General Services Administration's (GSA) comment. Documenting such
    inquiries allows a subsequent reviewer to draw similar conclusions and
    would be beneficial to any subsequent audit, including by the Defense
    Contract Audit Agency (DCAA).

    5.Contrary to GSA's comment, the review team-Federal Bureau of
    Investigation (FBI), GSA, and Mitretek-approved CSC's invoices that lacked
    information required by its task order, including employee billing rates
    and detail for subcontractor labor. We were not provided documentation
    indicating that any Computer Sciences Corporation (CSC) invoice had been
    rejected.

    6.While the review team compared billed labor rates against Millennia
    ceiling rates for certain labor costs, it did not evaluate labor rates
    compared to ceiling rates for subcontractor labor, which represented about
    $163 million of Trilogy costs. Had the review team reviewed labor charges
    more thoroughly, it may have identified the potential overcharging of
    labor rates discussed in this report related to DynCorp Information
    Systems (DynIS).

    7.According to the Federal Acquisition Regulation (FAR), it is the
    contractor's responsibility to maintain supporting documentation for costs
    billed, including subcontractor labor costs.

    8.Based on our discussion with on-site members of the review team, CSC
    travel vouchers were not obtained to review amounts billed on travel
    invoices. Had the vouchers been reviewed, the review team would have had a
    basis for questioning the first-class and excessive airfare costs we
    identified.

    9.The travel administrator's obligation to obtain the best fare does not
    relieve the government of its responsibility to review travel costs. In
    addition, we noted instances where the itinerary from the travel
    administrator indicated that a full-fare ticket was obtained at the
    traveler's request, even though the ticket cost more than twice as much as
    the lowest logical fare that was also noted on the itinerary.

    10.The approval process discussed by GSA relates to the travel
    authorization, which was the request to travel. We found that the review
    team lacked an adequate process to review travel vouchers that include the
    traveler's receipts to confirm that the authorized trips were taken and
    that the costs were in accordance with applicable travel regulations. Also
    see comments 8 and 9.

    11.The next sentence of the relevant section of the FAR cited by GSA
    states, "However, in order for airfare costs in excess of the above
    standard airfare to be allowable, the applicable condition(s) set forth
    above must be documented and justified." No such documentation was
    provided to us for any of the first class or other excessive airfares we
    identified.

    12.Our report stated that other direct costs (ODC) were paid without
    validation of the actual amounts included in the invoices and that the
    review team relied on the contractors to obtain purchase orders for ODC
    charges. It further stated that neither GSA, FBI, nor Mitretek performed
    procedures to ensure that equipment billed by the contractors was actually
    received before payment.

    13.CSC ODC invoices lacked sufficient detail to validate amounts billed
    compared to what was approved and we were not provided documentation
    indicating that such information was requested by the review team.
    Further, the CSC invoices did not include the detail necessary for the
    review team to specifically identify the items purchased. We also found
    that some assets were paid for before they were received and that the FBI
    did not perform an overall reconciliation of total assets ordered and paid
    for to those received.

    14.A GSA contracting officer representative told us that he was aware of
    the informal extended work week policy agreement, but could not provide
    documentation of the policy.

    15.Our report stated that DynIS charged labor rates that may have exceeded
    rates that GSA asserts were established ceiling rates pursuant to the task
    order.

    16.Based on GSA's acceptance of our recommendations on page 1 of its
    comments, we assume that the intent was to state that "GSA accepts each of
    GAO's recommendations."

    GAO Contacts and Staff Acknowledgments Appendix V

    Linda Calbom, (202) 512-9508, or calboml@gao.gov

    Staff members who made key contributions to this report include Steven
    Haughton (Assistant Director), Marie Ahearn, Brooks Bare, Ed Brown, Marcia
    Carlsen, Richard Cambosos, Lisa Crye, Tyshawn Davis, Bonnie Derby, Abe
    Dymond, Lori Ryza, Kara Scott, Brooke Whittaker, and Matt Wood.

    (190140)

    www.gao.gov/cgi-bin/getrpt? GAO-06-306 .

    To view the full product, including the scope

    and methodology, click on the link above.

    For more information, contact Linda Calbom at (202) 512-9508 or
    calboml@gao.gov.

    Highlights of GAO-06-306 , a report to congressional requesters

    February 2006

    FEDERAL BUREAU OF INVESTIGATION

    Weak Controls over Trilogy Project Led to Payment of Questionable
    Contractor Costs and Missing Assets

    The Trilogy project-initiated in 2001-is the Federal Bureau of
    Investigation's (FBI) largest information technology (IT) upgrade to date.
    While ultimately successful in providing updated IT infrastructure and
    systems, Trilogy was not a success with regard to upgrading FBI's
    investigative applications. Further, the project was plagued with missed
    milestones and escalating costs, which eventually totaled nearly $537
    million. In light of these events, Congress asked GAO to determine whether
    (1) internal controls provided reasonable assurance that improper payment
    of unallowable contractor costs would not be made or would be detected in
    the normal course of business, (2) payments to contractors were properly
    supported as a valid use of government funds, and (3) FBI maintained
    proper accountability for assets purchased with Trilogy project funds.

    What GAO Recommends

    GAO makes 27 recommendations to help improve (1) FBI's and GSA's controls
    over their invoice review and approval processes and to address
    questionable billing issues, and (2) FBI's accountability for assets. FBI
    concurred with our recommendations. While GSA accepted our
    recommendations, it did not believe that 1 of them was needed and
    expressed concern with some of our findings. GAO reaffirms its position on
    all of its findings and recommendations.

    FBI's review and approval process for Trilogy contractor invoices, which
    included a review role for the General Services Administration (GSA) as
    contracting agency, did not provide an adequate basis to verify that goods
    and services billed were actually received and that the amounts billed
    were appropriate, leaving FBI highly vulnerable to payments of unallowable
    costs. This vulnerability is demonstrated by FBI's payment of about $10.1
    million in questionable contractor costs we identified using data mining,
    document analysis, and other forensic auditing techniques. These costs
    included first-class travel and other excessive airfare costs, incorrect
    charges for overtime hours, potentially overcharged labor rates, and
    charges for which the contractors could not provide adequate supporting
    documentation to substantiate the costs purportedly incurred.

    FBI also failed to establish controls to maintain accountability over
    equipment purchased for the Trilogy project. These control lapses resulted
    in more than 1,200 missing pieces of equipment valued at approximately
    $7.6 million that GAO identified as part of its review. In addition, in
    its own inventory counts, FBI identified 37 pieces of Trilogy equipment
    valued at approximately $167,000 that had been lost or stolen. The table
    below summarizes questionable contractor costs and missing assets that GAO
    identified.

    Questionable Costs and Missing Assets

    Source: GAO.

    Given the poor control environment and the fact that GAO reviewed only
    selected FBI payments to Trilogy contractors, other questionable
    contractor costs may have been paid that have not been identified. If
    these control weaknesses go uncorrected, future contracts, including those
    related to Sentinel-FBI's new electronic information management system
    initiative-will be highly exposed to improper payments. In addition, the
    lack of accountability for Trilogy equipment calls into question FBI's
    ability to adequately safeguard its existing assets as well as those it
    may acquire in the future.
    *** End of document. ***