National Laboratories: DOE Needs to Assess the Impact of Using
Performance-Based Contracts (Letter Report, 05/07/1999, GAO/RCED-99-141).

The Department of Energy (DOE) contracts with private companies and
educational institutions to run 18 of its 22 laboratories. These are
cost reimbursement contracts under which DOE pays all of its
contractors' allowable costs. In 1994, DOE switched to performance-based
contracts as part of its contract reform program. This report provides
information on the status of performance-based contracting in DOE's
national laboratory contracts and efforts to determine the impact of
performance-based contracting. GAO found that DOE's use of
performance-based contracting for its laboratories is in a state of
transition. GAO found a wide variance in the number of performance
measures and the types of fees negotiated. Also, the contractors
reported that the main benefits from performance-based contracting were
that it has helped DOE clarify what it expects from the contractors and
that it has improved communication.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  RCED-99-141
     TITLE:  National Laboratories: DOE Needs to Assess the Impact of
	     Using Performance-Based Contracts
      DATE:  05/07/1999
   SUBJECT:  Contract performance
	     Contract terms
	     Laboratories
	     Management and operating contracts
	     Cost control
	     Performance measures
	     Fixed price contracts
	     Fixed price incentive contracts
	     Contractor payments

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  This text was extracted from a PDF file.        **
** Delineations within the text indicating chapter titles,      **
** headings, and bullets have not been preserved, and in some   **
** cases heading text has been incorrectly merged into          **
** body text in the adjacent column.  Graphic images have       **
** not been reproduced, but figure captions are included.       **
** Tables are included, but column deliniations have not been   **
** preserved.                                                   **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************

    United States General Accounting Office GAO                Report
    to the Committee on Science, House of Representatives May 1999
    NATIONAL LABORATORIES DOE Needs to Assess the Impact of Using
    Performance-Based Contracts GAO/RCED-99-141 GAO
    United States General Accounting Office Washington, D.C. 20548
    Resources, Community, and Economic Development Division B-282356
    May 7, 1999 The Honorable F. James Sensenbrenner, Jr. Chairman The
    Honorable George E. Brown, Jr. Ranking Minority Member Committee
    on Science House of Representatives The Department of Energy (DOE)
    contracts with private companies and educational institutions to
    manage and operate 18 of its 22 laboratories. These are cost
    reimbursement contracts under which DOE pays all of its
    contractors' allowable costs. DOE can also provide a fee, or
    profit, to a contractor for managing a laboratory. Responding to
    criticism that its historical contracting practices were costly
    and inefficient, DOE switched to performance-based contracts in
    1994 as part of its contract reform program. Use of these
    contracts allows DOE to structure each contract to provide a clear
    statement of what needs to be accomplished-rather than providing
    broad statements of work-and to rely on performance measures to
    evaluate a contractor's progress toward meeting its objectives. An
    important feature of performance-based contracting is providing
    incentives, including fees, to the contractor's achievement of
    objectives as a means of encouraging superior performance and
    lowering costs. Concerned about the progress made to implement
    performance-based contracting at the national laboratories, you
    asked us to * assess the status of performance-based contracting
    in DOE's national laboratory contracts, and * identify efforts
    being made to determine the impact of performance-based
    contracting. Results in Brief      DOE's use of performance-based
    contracting for its laboratories is in a state of transition.
    While all laboratory contracts we examined had some performance-
    based features, we found wide variance in the number of
    performance measures and the types of fees negotiated. About half
    of the 18 laboratory contracts have performance fees to encourage
    superior performance-a major goal of performance-based
    contracting. Most of the remaining laboratory contracts are still
    based on DOE's traditional fixed-fee arrangement in which the fees
    are paid regardless of performance. Page 1
    GAO/RCED-99-141 Performance-Based Contracting B-282356 DOE has not
    evaluated the impact of performance-based contracting on its
    laboratory contractors and, as a result, does not know if this new
    form of contracting is achieving the intended results of improved
    performance and lower costs. Specifically, DOE has not determined
    whether giving higher fees to encourage superior performance by
    its laboratory contractors is advantageous to the government,
    although we recommended in 1994 that DOE develop criteria for
    measuring the costs and benefits to the government of using higher
    fees.1 Fees for the laboratories totaled over $100 million for
    fiscal year 1998. While the contractors were unable to cite
    measurable benefits achieved by switching to performance-based
    contracting, they support its goals. The main benefits from
    performance-based contracting cited by laboratory contractors was
    that it has helped DOE clarify what it expects from the
    contractors and that it has improved communication. Background
    DOE manages the largest laboratory system of its kind in the
    world. Since the early days of the World War II Manhattan Project,
    DOE's laboratories have played a major role in maintaining U.S.
    leadership in research and development. DOE is responsible for
    ensuring that the laboratory system-with 22 laboratories in 14
    states, a combined budget of over $10 billion a year, and a staff
    of about 60,000-is managed in an effective, efficient, and
    economical manner. DOE contracts with educational institutions and
    private sector organizations for the management and operation of
    18 of its laboratories. (App. I lists DOE's national
    laboratories.) The remaining four laboratories are staffed by
    federal employees. DOE pays its laboratory contractors all
    allowable costs. DOE can also pay contractors a separate fee, or
    profit, as compensation for operating the laboratories. Fees are
    based on the contract value and the technical complexity of the
    work to be performed at a laboratory, but also on the degree of
    financial liability or risk that a contractor is willing to
    assume. Under performance-based contracting principles, fees can
    include both a fixed amount and an amount that is linked to
    achieving performance objectives. One of DOE's major goals in
    performance-based contracting is to develop performance objectives
    for each contractor that are specific, results-oriented,
    measurable, and reflect the most critical activities. 1Energy
    Management: Modest Reforms Made in University of California
    Contracts, but Fees Are Substantially Higher (GAO/RCED-94-202,
    Aug. 25, 1994). Page 2
    GAO/RCED-99-141 Performance-Based Contracting B-282356
    Performance-Based                DOE's implementation of
    performance-based contracting for its Contracting at the
    laboratories is in a state of transition. While most of its
    laboratory contracts contain some performance-based features, the
    contracts National Laboratories            negotiated by DOE vary
    from contract to contract. For example, DOE is Is an Evolving
    Process incorporating performance-based features in all of its
    laboratory contracts, although measures vary substantially in
    number, ranging from a low of 7 in one laboratory contract to
    about 250 in another. Also, DOE has negotiated performance fees in
    only 9 of its 18 laboratory contracts because the remaining
    laboratories are still operating under DOE's traditional approach
    in which fees are not linked to performance. We found that similar
    laboratories managed by similar contractors have different
    contracts. The wide diversity of contract features reflects DOE's
    philosophy of relying on DOE field units to tailor contracts to
    local conditions and contractors' preferences. Developing the
    Right             Since introducing performance-based contracting
    in 1994, DOE and its Performance Measures Is a        laboratory
    contractors have struggled to find the right mix of measures
    Challenge                        that accurately and reliably
    capture the contractors' performance. According to DOE field
    staff, in the early years of contract reform, DOE encouraged its
    field units to construct as many measures as they could, but
    provided limited guidance on how to accomplish this task. As a
    result, early attempts led to large numbers of performance
    measures. A large number of measures diminishes the importance of
    any single measure, whereas a small number results in measures
    that are too broad to be meaningful. For example, a DOE field
    official told us, "The original guidance from DOE Headquarters was
    to [develop performance measures] as much as possible.
    Unfortunately, there was inadequate guidance on how to do this. .
    . . The number of performance measures . . . is too large.
    However, if we fail to cover an activity [with a measure] the
    contractor may not give the attention needed to the activity." DOE
    and its laboratories are still attempting to develop the right
    number of measures. For example, we found that the number of
    performance measures in the laboratory contracts we examined
    ranged from a low of 7 measures at the Ames Laboratory in Iowa to
    about 250 at the Idaho National Engineering and Environmental
    Laboratory in Idaho. DOE and its contractors are also working to
    develop measures that reliably address the most important
    activities of the laboratories. According to a Page 3
    GAO/RCED-99-141 Performance-Based Contracting B-282356 field
    official, DOE's early attempts at developing performance measures
    resulted in contractors focusing only on those activities that
    were tied to performance fees, while neglecting other important
    activities. Another DOE site official stated, "[P]erformance-based
    contracting tends to focus too much on the monetary reward . . .
    and less on an analysis of performance. The incentive at the labs
    should be [for] good science, not more dollars." Developing the
    right number and type of performance measures is an evolving
    process between DOE and its contractors. Most DOE and contractor
    representatives told us that they are making progress in finding
    measures that accurately and reliably reflect performance,
    particularly in management and operations activities. Measuring a
    contractor's performance in science and technology is more
    difficult. Science and technology measures are broader in scope
    and typically rely on peer reviews and a contractor's self-
    assessment for evaluating performance. Types of Fees Paid to
    Although performance fees are a major feature of performance-based
    Contractors Vary Widely    contracting, only 9 of the Department's
    18 laboratory contracts have them. Nine of the remaining
    laboratory contracts operate under DOE's traditional fixed-fee
    arrangement, and one laboratory contract has no fee. Fixed fees
    are earned regardless of performance and were commonly used before
    DOE adopted performance-based contracting as its normal business
    practice. Appendixes I and II summarize laboratory fee
    arrangements and illustrate the wide variety of fee arrangements
    in use. In commenting on a draft of this report, DOE said that by
    the end of calendar year 1999, the majority of laboratory
    contracts that provide fees will have performance-based fee
    structures. Performance fees were introduced as a way of
    encouraging superior performance and can include an incentive and
    an award fee. An incentive fee is usually applied to activities
    for which progress can be accurately measured, for example,
    cleaning up 40 barrels of toxic waste within a prescribed period
    of time. An award fee is usually applied to tasks that are harder
    to measure and require a more subjective judgment of performance,
    for example, assessing a contractor's attention to community
    relations. Performance fees represent the amount of a contractor's
    total fee placed "at risk" since the fee that could be earned is
    determined by how well the contractor performs. Page 4
    GAO/RCED-99-141 Performance-Based Contracting B-282356 As the
    following examples show, some laboratory contracts include both
    types of performance fees, while others rely solely on an
    incentive fee or an award fee. Still others have neither and use
    only fixed fees. * At the Sandia National Laboratories in New
    Mexico and the Oak Ridge National Laboratory in Tennessee, DOE
    negotiated fixed-fee contracts. Both of these laboratories are
    operated by subsidiaries of the Lockheed Martin Corporation-a for-
    profit company. DOE officials told us they were confident that
    incentive fees were not needed for these laboratories because the
    existing Lockheed Martin contractors' performance is superior and
    introducing incentive fees might distract the contractors from
    performing all essential work. * At the Idaho National Engineering
    and Environmental Laboratory in Idaho, operated by Lockheed Martin
    Idaho Technologies Company, DOE uses a combination of fixed,
    incentive, and award fees. DOE officials told us that incentive
    fees were used because of the many different tasks that could be
    identified and measured, but that award fees were also needed to
    assess activities that required more subjective judgments. * At
    the Stanford Linear Accelerator Center in California, operated by
    Stanford University, DOE negotiated a no-fee contract, the only
    such arrangement in the laboratory system. According to DOE, the
    laboratory contractor does not want a fee for operating this
    laboratory because a fee would not motivate performance and may be
    a detriment to the conduct of outstanding science, which is the
    primary mission of this laboratory. * The Lawrence Berkeley
    National Laboratory and Lawrence Livermore National Laboratory in
    California and the Los Alamos National Laboratory in New Mexico
    are operated by the University of California. The contracts
    contain a fixed fee and an incentive fee for meeting expectations,
    plus another amount for exceeding expectations. A senior DOE
    official acknowledged the variability in laboratory contracts but
    said that imposing uniform practices throughout the laboratory
    system would not necessarily improve the overall performance and
    accountability of the contractors. According to DOE and laboratory
    officials, there are several reasons for the variability in the
    contracts. First, the laboratories engage in different activities
    with different levels of technical complexities. Second, some
    contractors are willing to assume greater financial risk or
    liability and thus expect a higher or different fee arrangement.
    Finally, DOE field officials who negotiate the contracts employ
    features that they believe are best suited for their particular
    circumstances. However, we found that similar laboratories
    operated by similar contractors have different fee arrangements.
    For example, both the Page 5
    GAO/RCED-99-141 Performance-Based Contracting B-282356 Lawrence
    Berkeley and Argonne national laboratories have similar research
    missions and are both managed by university contractors. However,
    Lawrence Berkeley's contractor, the University of California,
    works under a fixed-fee plus performance fee arrangement, while
    Argonne's contractor, the University of Chicago, works under a
    performance fee arrangement only. We also found substantial
    variations in contracting philosophy among DOE field officials.
    DOE relies on field units to negotiate its contracts, including
    whether to use performance-based fees, and how performance
    objectives and measures will be accomplished. Some of these
    officials told us that performance fees are important motivators,
    while others said performance fees can distract the contractor
    from other important work. In commenting on a draft of this
    report, DOE provided us with additional reasons for the
    variability in contracts, including the timing of when contractors
    first converted to performance-based contracting, the nature of
    the proposals received in competitive awards, and the negotiated
    terms in contract extensions. In addition, DOE cited other
    motivations for laboratory contractors, such as their reputations
    in the scientific community and contract extensions. The Impact of
    DOE's guidance states that the purpose of performance-based
    contracting is Performance-Based      to obtain better performance
    or lower costs or both. DOE has not analyzed the impact of
    performance-based contracting on its laboratory Contracting
    Remains    contractors. As a result, it has not determined whether
    performance-based Unknown                contracting is achieving
    the intended objectives of reducing costs and improving
    performance. DOE officials told us that the amounts of fees paid
    to laboratory contractors have generally increased with the
    implementation of performance-based contracting but that it is
    difficult to determine the return on this investment since
    contractors are also assuming more risk or liability for costs
    previously paid by DOE. Increased liabilities include costs due to
    a failure to exercise prudent business judgment on the part of the
    contractor's managerial personnel. DOE has not analyzed the
    relative costs and benefits to the government of using higher fees
    in performance-based contracts. We previously recommended that DOE
    ensure that the fees paid to contractors for incurring increased
    financial risks are cost-effective by developing criteria for
    measuring the costs and benefits to the government Page 6
    GAO/RCED-99-141 Performance-Based Contracting B-282356 of this
    approach.2 DOE officials told us that while they have not
    conducted a comprehensive cost-benefit analysis of fees, they try
    to negotiate fees that make sense for individual contracts, taking
    into account the financial risks and incentives needed to motivate
    performance. Without such an overall analysis, however, it is
    difficult to determine the value to the government of the over
    $100 million spent on contractor fees for fiscal year 1998.
    Although DOE has not assessed the impact of performance-based
    contracting, limited reviews have found both progress and
    problems, as these examples show: * Since 1997, DOE's Office of
    Inspector General has issued three reports on problems the
    Department had in implementing performance-based incentives at
    three facilities (one of which was a laboratory).3 Problems
    reported by the Inspector General included contracts with poorly
    developed performance measures and fees that were paid to
    contractors before agreement was reached on the performance
    incentives. * In 1997, DOE's Office of Procurement issued a report
    on the use of performance-based incentives. The report noted that
    the use of incentives has been effective in directing contractors'
    attention to performance outcomes and has improved communications
    concerning performance expectations. The report also noted that
    DOE field units are improving the quality of their contracts.
    However, the report pointed out that implementation was sometimes
    inconsistent and that performance objectives sometimes were overly
    focused on process milestones rather than on outcomes. DOE's
    laboratories were not the focus of this review, however.4 * Our
    July 1998 report on DOE's performance-based incentive contracts
    noted that the Department had taken steps to correct many of the
    problems cited in the Inspector General's reports, including
    issuing guidance, conducting training, and incorporating lessons
    learned into fiscal year 1998 contract incentives.5 We noted that
    although DOE 2GAO/RCED-94-202, Aug. 25, 1994. 3Inspection of the
    Performance Based Incentive Program at the Richland Operations
    Office (DOE/IG-0401, Mar. 10, 1997); Audit of the Contractor
    Incentive Programs at the Rocky Flats Environmental Technology
    Site (DOE/IG-0411, Aug. 13, 1997); and Inspection Report: The
    Fiscal Year 1996 Performance Based Incentive Program at the
    Savannah River Operations Office (DOE-INS-O-98-03, May 1998).
    4Contract Reform Self Assessment Report, Office of Contract Reform
    and Privatization, DOE (Sept. 1997). 5Department of Energy:
    Lessons Learned Incorporated Into Performance-Based Incentive
    Contracts (GAO/RCED-98-223, July 15, 1998). Page 7
    GAO/RCED-99-141 Performance-Based Contracting B-282356 maintained
    that its performance-based incentives have been effective in
    achieving the desired end results, it had not been clear whether
    these successes were due to performance-based incentives or to an
    increased emphasis on program management. None of these
    assessments focused exclusively on laboratory contracts. In our
    discussions, DOE field staff generally credited performance-based
    contracting with improving their ability to set expectations for
    the Department's laboratories, and several laboratory contractors
    concurred that this was a benefit. In addition, both DOE and
    laboratory officials cited improved communication as a benefit of
    performance-based contracting. Laboratory contractors also
    credited DOE for focusing its oversight on evaluating results and
    away from dwelling on strict compliance with DOE's rules and
    regulations. In addition, contractors told us they have increased
    productivity and lowered costs, especially for the support and
    overhead functions. However, most of these officials also said
    that these advances were more the result of other initiatives,
    such as internal streamlining actions, than of performance-based
    contracting. Conclusions    DOE and its laboratory contractors
    told us that they are committed to making performance-based
    contracting work effectively and that the contracts are including
    more specific and reliable performance measures. However, since
    DOE has not evaluated the impact of performance-based contracting
    on its laboratories-owing in part to the wide variance in fee
    arrangements-there is limited evidence on how performance fees
    ensure a high level of performance by contractors at lower cost.
    As a result, DOE cannot show how the higher fees it is paying to
    contractors under performance-based contracting are of value to
    the government and to the taxpayers. We previously recommended
    that the Secretary of Energy ensure that the fees paid to
    contractors for incurring increased financial risk are cost-
    effective by developing criteria for measuring the costs and
    benefits to the government of this approach. DOE did not implement
    our recommendation and has no plans to measure the overall costs
    and benefits of performance-based contracting for its
    laboratories. DOE officials maintain that performance-based
    contracting is working, but this is based on anecdotal evidence.
    Moreover, the fees DOE negotiates are based on its best judgment
    of what is needed to motivate contractors and to compensate them
    for increased risk, but DOE's evidence is based primarily on non-
    laboratory contractors, and DOE has not quantified the Page 8
    GAO/RCED-99-141 Performance-Based Contracting B-282356 value of
    the increased risk assumed by contractors under performance-based
    conditions. Recommendation     Because DOE does not know whether
    performance-based contracting is improving performance at lower
    cost at its national laboratories and because our previous
    recommendation to develop criteria for measuring the costs and
    benefits of paying fees to contractors for incurring increased
    financial risk was not implemented, we recommend that the
    Secretary of Energy evaluate the costs and benefits from using
    performance-based contracting at the national laboratories. While
    we recognize that each laboratory contract is individually
    negotiated, DOE should nevertheless ensure that the fees it
    provides to motivate contractors and to compensate them for
    increased financial risk is based on an analysis of costs and
    benefits. The need for this type of evaluation is consistent with
    the principles of the Government Performance and Results Act of
    1993 that require agencies to measure outcomes against their
    goals.6 Agency Comments    We provided a draft of this report to
    DOE for review and comment. DOE disagreed with our conclusion on
    the need for determining the costs and benefits of the fees it has
    negotiated with its laboratory contractors. DOE noted that its
    performance-based contracting experience is in transition but that
    its evaluations show that performance-based contracting is
    working. We acknowledge in our report that DOE's evaluations of
    performance-based contracting show promise, but we also point out
    that these evaluations did not focus on the laboratories'
    experiences with performance-based contracting. Because of this
    limitation and because of the higher fees being negotiated with
    the laboratories, we continue to believe it is desirable for DOE
    to determine if its performance-based contracting is improving
    performance at lower cost. DOE also commented that the variability
    we found in performance-based laboratory contracts reflects many
    different factors, including differences in the scope of work, the
    type of contractor, and the experiences the laboratories have with
    performance-based contracting features. Our report described the
    reasons for the variability in laboratory contracts, and we have
    included the additional reasons provided in DOE's comments. We
    also agree that DOE's use of performance-based contracting is
    evolving and that the variability we found in laboratory contracts
    (principally in 6The Results Act applies to agencies as defined in
    5 U.S.C. 306(f), which generally covers executive departments,
    government corporations, and independent establishments. Page 9
    GAO/RCED-99-141 Performance-Based Contracting B-282356 performance
    measures and fee arrangements) is in part due to an ongoing
    learning process associated with the transition to performance-
    based contracting. DOE also raised a number of issues regarding
    the use of fees in its laboratory contracts and strongly defended
    its use of performance fees. We agree with many of DOE's
    observations on the use of performance fees, and we are not
    suggesting that DOE should abandon its performance-based approach
    or that it should eliminate performance-based fees in its
    laboratory contracts. It is also not our intent to show that
    performance-based contracting should be abandoned if its impacts
    on the laboratories cannot be measured. We do believe, however,
    that effective implementation of performance-based contracting
    provisions is dependent on the ability to support the fee amounts
    paid through a cost and benefit analysis. DOE also provided a
    number of clarifications that we have incorporated in our report
    as appropriate. Appendix III includes the full text of DOE's
    comments and our response. Our review was performed from September
    1998 through April 1999 in accordance with generally accepted
    government auditing standards. See appendix IV for a description
    of our scope and methodology. As arranged with your offices,
    unless you publicly announce its contents earlier, we plan no
    further distribution of this report until 30 days after the date
    of this letter. At that time, we will send copies to Bill
    Richardson, Secretary of Energy, and Jacob J. Lew, Director,
    Office of Management and Budget. We will make copies available to
    others on request. If you or your staff have any questions about
    this report, please call me at (202) 512-3841. Major contributors
    to this report were Gary R. Boss and Tom Kingham. Susan D. Kladiva
    Associate Director, Energy, Resources, and Science Issues Page 10
    GAO/RCED-99-141 Performance-Based Contracting Page 11
    GAO/RCED-99-141 Performance-Based Contracting Contents Letter
    1 Appendix I
    14 Contract Amount and Fees Earned by DOE's Laboratory Contractors
    in Fiscal Year 1998 Appendix II
    16 Status of DOE Laboratory Contracts as of March 30, 1999
    Appendix III
    18 Comments From the Department of Energy Appendix IV
    29 Scope and Methodology Abbreviations DOE          Department of
    Energy GAO          General Accounting Office R&D
    research and development Page 12
    GAO/RCED-99-141 Performance-Based Contracting Page 13
    GAO/RCED-99-141 Performance-Based Contracting Appendix I Contract
    Amount and Fees Earned by DOE's Laboratory Contractors in Fiscal
    Year 1998 Contract amount Laboratory/contractor
    (millions)a      Fixed or base feeb    Performance feec
    Total fee earned Argonne National Laboratory/University of Chicago
    $466.9                                 $3,425,000
    $3,425,000 Brookhaven National Laboratory/Brookhaven Science
    Associates                                              385.9
    3,574,000                                     3,574,000 Idaho
    National Engineering and Environmental Laboratory/Lockheed Martin
    Idaho Technologies Company                       578.7
    9,848,000                9,848,000 Lawrence Berkeley National
    Laboratory/University of California                     320.0
    420,000            1,063,780                1,483,780 Lawrence
    Livermore National Laboratory/University of California
    1,100.0            1,680,000            4,482,000
    6,162,000 Los Alamos National Laboratory/University of California
    1,345.0            2,100,000            5,550,000
    7,650,000 Oak Ridge National Laboratory/Lockheed Martin Energy
    Research                                                475.6
    7,220,000                                     7,220,000 Pacific
    Northwest Laboratory/Battelle Memorial Institute
    461.0             5,600,000
    5,600,000 Sandia National Laboratories/Sandia Corp. (Lockheed
    Martin)                                1,397.6
    14,347,000                                   14,347,000 Ames
    Laboratory/Iowa State University                    25.4
    TBDd                     TBDd Fermi National Accelerator
    Laboratory/University Research Associates, Inc.
    279.6             2,750,000
    2,750,000 National Renewable Energy Laboratory/Midwest Research
    Institute                   199.4             3,522,500
    3,522,500                7,045,000 Princeton Plasma Physics
    Laboratory/Princeton University                          59.4
    10,000                                        10,000 Stanford
    Linear Accelerator Center/Stanford University
    187.0 Thomas Jefferson National Accelerator Facility/Southeastern
    University Research Associates, Inc.
    70.2             1,874,633
    1,874,633e Bettis Atomic Power Laboratory/Bechtel Group
    305.1             8,686,000
    8,686,000 Knolls Atomic Power Laboratory/KAPL, Inc. (Lockheed
    Martin)                                  265.0
    7,300,000                                     7,300,000 Savannah
    River Technology Centerf/Westinghouse Savannah River Co.
    1,248.0                                51,570,100
    51,570,100 (Table notes on next page) Page 14
    GAO/RCED-99-141 Performance-Based Contracting Appendix I Contract
    Amount and Fees Earned by DOE's Laboratory Contractors in Fiscal
    Year 1998 aContract amounts for fiscal year 1998 include some
    estimates. bA base fee, used in a performance fee contract, is the
    part of the fee not at risk and is similar to a fixed fee. cA
    performance fee can be either an award fee or an incentive fee or
    a combination. dThe fee is to be determined after DOE reviews of
    the contractor's self-assessment and after the issuance of final
    DOE reports. eThe fee is considered a management allowance, which
    is similar to a fixed fee. fThe contract and fee amounts shown are
    for the entire Savannah River Site, including the Savannah River
    Technology Center. Source: GAO based on data from DOE's
    headquarters and operations offices. Page 15
    GAO/RCED-99-141 Performance-Based Contracting Appendix II Status
    of DOE Laboratory Contracts as of March 30, 1999
    Laboratory/contractor                                        Type
    and status of contract Argonne National Laboratory/University of
    Chicago            Incentive-fee contract. DOE plans to extend
    this contract 5 years, but is renegotiating to make it consistent
    with the federal acquisition regulations format and to incorporate
    all contract reform features, including performance-based
    provisions. Brookhaven National Laboratory/Brookhaven Science
    Fixed-fee contract. The new contract was signed in January 1998
    with a Associates
    fixed fee through September 1998. DOE is still negotiating the
    contract for fiscal year 1999. DOE plans to negotiate a
    performance fee. Idaho National Engineering and Environmental
    Incentive-fee contract. DOE plans to recompete this contract in
    fiscal Laboratory/Lockheed Martin Idaho Technologies Company
    year 1999. The current contractor, Lockheed Martin, announced it
    will not bid. Lawrence Berkeley National Laboratory/University of
    Incentive-fee contract. The amount of the annual available fee
    remains California
    the same for each year of the 5-year contract. Lawrence Livermore
    National Laboratory/University of         Incentive-fee contract.
    The amount of the annual available fee remains California
    the same for each year of the 5-year contract. Los Alamos National
    Laboratory/University of California      Incentive-fee contract.
    The amount of the annual available fee remains the same for each
    year of the 5-year contract. Oak Ridge National
    Laboratory/Lockheed Martin Energy         Fixed-fee contract. DOE
    is recompeting this contract. DOE plans to Research
    convert this contract to a performance fee. Lockheed-Martin
    announced that it will not bid as a prime contractor on the new
    contract. The other two contracts for this site-for environmental
    cleanup and production-are performance-fee Pacific Northwest
    Laboratory/Battelle Memorial Institute     Incentive-fee contract.
    DOE converted this contract from a fixed-fee to an incentive-fee
    type and made available $7.1 million in potential fees geared to
    incentives in four areas-science and technology excellence,
    operational excellence, leadership and management, and community
    relations. Sandia National Laboratories/Sandia Corp. (Lockheed
    Martin) Fixed-fee contract. The contract that expired in September
    1998 was renegotiated and extended noncompetitively for 5 years.
    The new contract remains a fixed-fee arrangement but now includes
    performance objectives, measures, and criteria. DOE decided that
    the contractor's superior performance could be sustained with a
    fixed fee. Ames Laboratory/Iowa State University
    Incentive-fee contract. DOE is renegotiating this contract and
    plans to extend noncompetitively for 5 years. DOE plans to make
    the contract consistent with the federal acquisition regulations
    format and to incorporate all contract reform conditions,
    including performance-based provisions. Fermi National Accelerator
    Laboratory/University Research    Fixed-fee contract. DOE has not
    announced whether it will recompete or Associates, Inc.
    extend this contract. DOE rates the contractor's performance as
    outstanding. National Renewable Energy/Midwest Research Institute
    Award-fee contract. DOE recompeted this contract in 1998. The new
    contract was effective on Oct. 1, 1998, and is fixed fee until
    March 1999, at which time DOE intends to includes an award fee for
    the remainder of the contract period. Princeton Plasma Physics
    Laboratory/Princeton University     Fixed-fee contract. The
    contractor did not want any fee, but DOE negotiated a small fee of
    $10,000. (continued) Page 16
    GAO/RCED-99-141 Performance-Based Contracting Appendix II Status
    of DOE Laboratory Contracts as of March 30, 1999
    Laboratory/contractor
    Type and status of contract Stanford Linear Accelerator
    Center/Stanford University         No-fee contract. The contract
    term ended on March 31, 1998, and was extended noncompetitively on
    a month-by-month basis during negotiations to incorporate
    performance-based incentives. The contract was then extended
    noncompetitively for 5 years in January 1999. The contract
    includes performance measures and expectations, but no fee. Thomas
    Jefferson National Accelerator Facility/Southeastern No fee
    contract (with management allowance). The contract is currently
    University Research Associates, Inc
    being renegotiated so that it can be extended noncompetitively for
    5 years. DOE plans the new contract to be a fixed-fee arrangement.
    Objectives of the negotiations are to structure the contract to be
    consistent with the federal acquisition regulations format and to
    incorporate all contract reform conditions, including performance-
    based features. Bettis Atomic Power Laboratory/Bechtel Group
    Fixed-fee contract. The contract was recompeted in 1998. The new
    contractor was selected (the Bechtel Group) but the incumbent
    contractor, Westinghouse Electric Corporation, protested the
    award. The existing contract extended non-competitively pending
    the result of a bid protest to GAO. The bid protest was denied by
    GAO. The new contract with Bechtel was effective February 1, 1999.
    Knolls Atomic Power Laboratory/KAPL, Inc. (Lockheed
    Fixed-fee contract. Martin) Savannah River Technology
    Center/Westinghouse Savannah Incentive-fee contract. River Co.
    Source: GAO based on data from DOE's operations offices. Page 17
    GAO/RCED-99-141 Performance-Based Contracting Appendix III
    Comments From the Department of Energy Note: GAO comments
    supplementing those in the report text appear at the end of this
    appendix. Page 18      GAO/RCED-99-141 Performance-Based
    Contracting Appendix III Comments From the Department of Energy
    See comment 1. See comment 2. See comment 1. Now on p. 1. See
    comment 1. See comment 3. See comment 4. Page 19
    GAO/RCED-99-141 Performance-Based Contracting Appendix III
    Comments From the Department of Energy See comment 5. See comment
    6. Now on p. 6. See comment 7. Page 20
    GAO/RCED-99-141 Performance-Based Contracting Appendix III
    Comments From the Department of Energy See comment 8. Page 21
    GAO/RCED-99-141 Performance-Based Contracting Appendix III
    Comments From the Department of Energy See comment 9. Page 22
    GAO/RCED-99-141 Performance-Based Contracting Appendix III
    Comments From the Department of Energy Now on p. 4. See comment
    10. Now on p. 4. See comment 11. Now on pp. 5 and 6. See comment
    12. Page 23                             GAO/RCED-99-141
    Performance-Based Contracting Appendix III Comments From the
    Department of Energy Now on p. 6. See comment 13. Now on p. 8. See
    comment 14. Page 24                             GAO/RCED-99-141
    Performance-Based Contracting Appendix III Comments From the
    Department of Energy See comment 1. Page 25
    GAO/RCED-99-141 Performance-Based Contracting Appendix III
    Comments From the Department of Energy GAO Comments    The
    following are GAO's comments on the Department of Energy's letter
    dated April 22, 1999. 1. We have made changes to the report as
    appropriate in response to DOE's comments. 2. Our wording is drawn
    from DOE's guidance on performance-based contracting, and we have
    made changes to our report to reflect DOE's comments. DOE
    recommends that its laboratory contracts contain performance-based
    features, which include clear expectations described in terms of
    results, not how the work is to be accomplished. 3. As we stated
    in our report, DOE's evaluations did not focus on the laboratory
    contractors, nor did these evaluations focus on the costs and
    benefits of performance-based contracting features, including the
    impact of fees. 4. We recognize that one of the purposes of
    providing fees is to reflect the financial risk associated with
    work performance, and we make this point in our report. Our 1994
    recommendation questioned the cost-benefit of the increased fees,
    regardless of whether they were related to performance or
    financial risk. We continue to believe that our recommendation is
    relevant because DOE has not evaluated the cost and benefit of the
    fees it is providing to laboratory contractors. 5. We believe our
    wording adequately reflects the conditions discussed. Information
    on the laboratory fees and total contract costs is presented in
    appendix I. 6. We have made changes to the report as appropriate
    in response to DOE's comments on contract type. We stated in our
    report that DOE's performance-based contracting is in a state of
    transition. We also stated that there are wide variations in
    performance measures and fee arrangements negotiated by DOE and
    its laboratory contractors. This material is presented as facts
    describing the conditions that presently exist. Our report also
    describes the reasons for the variability in laboratory contracts
    and includes most of the reasons given in DOE's comments. We have
    made changes in the report to reflect these additional reasons for
    the variability in DOE's laboratory contracts. 7. Our statement
    that contract differences are the product of DOE's relying on its
    field units to tailor contracts to local conditions is based on
    Page 26                             GAO/RCED-99-141 Performance-
    Based Contracting Appendix III Comments From the Department of
    Energy interviews with numerous DOE field officials. This
    statement is not an implied criticism of how DOE negotiates
    contracts. Also, we disagree with DOE's characterization that
    contractors' preferences are "generally irrelevant" when
    accounting for the variations that exist among laboratory
    contractors. As DOE noted, contractors' preferences are reflected
    in the negotiation process. In our discussions with DOE field
    officials responsible for negotiating contracts, laboratory
    contractors' preferences on fees were cited as a critical factor
    in determining fee structures. 8. Our report recognizes that
    developing the optimum number of performance measures is a
    challenge, as reflected in the wide range of performance measures
    in use even among similar laboratories. We are not suggesting that
    any two contractors should have the same measures or the same
    number of measures. Our point is that DOE continues to struggle
    with finding the right number of measures. To further illustrate,
    the University of California's fiscal year 1998 contracts for its
    two weapons laboratories-Lawrence Livermore and Los Alamos-contain
    83 and 120 performance measures, respectively, even though these
    laboratories are very similar in budget and scope. They are,
    however, managed by different DOE field units. 9. Our purpose in
    including comments we received from DOE field units is to
    illustrate the wide differences in philosophy about the use of
    fees to motivate laboratory contractors. Several DOE field staff,
    as well as contractors, told us that they strongly believe that
    providing fees does not motivate contractors, including both for-
    profit and not-for-profit contractors. Moreover, our statement
    that performance-based contracting has tended to focus in some
    instances on monetary rewards at the expense of good science was a
    frequent comment from both DOE field officials and laboratory
    contractors. Thus, it is very important to identify the need for
    monetary incentives where they are appropriate. Other motivations
    that DOE cited for laboratory contractors, such as their
    reputations in the scientific community and desire for contract
    extensions, were added to our report. These differences in
    philosophy account for some of the variation in contracts. 10. Our
    report reflects information provided directly from DOE field
    staff, who we were advised by DOE headquarters were the proper
    source for this information. The data in DOE's comments are
    reflected in the appendixes to our report. We have also revised
    our report to show that there are now 18 laboratory contractors,
    reflecting a recent change in how DOE defines its laboratories.
    Page 27                             GAO/RCED-99-141 Performance-
    Based Contracting Appendix III Comments From the Department of
    Energy 11. DOE field officials told us that performance fees are
    used to encourage superior performance. Asserting that fees are
    used to link performance to financial reward is self-evident in
    this context. 12. We agree with DOE that no single approach in
    contracting has proven to be optimum, and we reflected this view
    in our report. Regarding the wide variability in fee arrangements,
    we stated that there was very little consistency among the
    contracts of similar laboratory contractors conducting similar
    work. We also stated that local conditions influence the
    variability in laboratory contracts. 13. Our wording was taken
    from DOE's guidance on performance-based contracting. As we state
    in our report, prior assessments of performance-based contracting
    have not focused on laboratory contractors. We also stated in our
    report that DOE believes that the results from its assessments of
    performance-based contracting have been positive. We believe it is
    a logical and desirable step for DOE to determine whether
    performance-based contracting is improving performance at lower
    cost in its national laboratories. Also, we are not suggesting
    that DOE should abandon its performance-based approach or that it
    should eliminate performance-based fees in its laboratory
    contracts. It is also not our intent to show that performance-
    based contracting should be abandoned if its impacts on the
    laboratories cannot be measured. We believe that effective
    implementation of performance-based contracting provisions is
    dependent on the ability to support the fee amounts paid through a
    cost and benefit analysis. While it may appear intuitively obvious
    that defining performance expectations and measuring results are
    effective management tools, it is not intuitively obvious that the
    government is receiving a reasonable return on its investments in
    fee amounts for laboratory contractors. Likewise, while DOE
    commented that increases in fees reflect, in part, the increased
    financial risks being borne by contractors, no cost-benefit
    analysis quantifying this increased financial risk has been
    completed; thus it is not possible to determine if the proper
    level of fee is appropriate for the risk assumed. 14. We recognize
    that laboratory contractor fees are relatively small percentages
    of the total contract amounts. However, these percentages, which
    translated into $100 million in fees for fiscal year 1998, must be
    considered in light of the fact that DOE's laboratories are
    government owned and that a laboratory contractor's financial risk
    is limited. Page 28                             GAO/RCED-99-141
    Performance-Based Contracting Appendix IV Scope and Methodology To
    obtain information on the national laboratories' contracts, we
    interviewed officials from the following laboratories: Sandia
    National Laboratories and Los Alamos National Laboratory in New
    Mexico; Lawrence Berkeley National Laboratory, Lawrence Livermore
    National Laboratory, and the Stanford Linear Accelerator in
    California; the National Renewable Energy Laboratory in Colorado;
    the Idaho National Engineering and Environmental Laboratory in
    Idaho; the Oak Ridge National Laboratory in Tennessee; and the
    Argonne National Laboratory in Illinois. We also spoke with
    laboratory officials in other locations to obtain cost and status
    information. We asked officials at these laboratories to comment
    on the impact of performance-based contracting on their
    operations. We also interviewed Department of Energy (DOE)
    officials responsible for overseeing these laboratories. These
    officials were from DOE's operations offices in Albuquerque, New
    Mexico; Oakland, California; Oak Ridge, Tennessee; and Chicago,
    Illinois. We also interviewed DOE area and site office staff
    located at each of the operations offices we visited. To obtain a
    broader perspective, we interviewed DOE headquarters officials
    responsible for developing contracting policy. We conducted our
    review from September 1998 through April 1999 in accordance with
    generally accepted government auditing standards. (141265)
    Page 29                            GAO/RCED-99-141 Performance-
    Based Contracting Ordering Information The first copy of each GAO
    report and testimony is free. Additional copies are $2 each.
    Orders should be sent to the following address, accompanied by a
    check or money order made out to the Superintendent of Documents,
    when necessary. VISA and MasterCard credit cards are accepted,
    also. Orders for 100 or more copies to be mailed to a single
    address are discounted 25 percent. Orders by mail: U.S. General
    Accounting Office P.O. Box 37050 Washington, DC  20013 or visit:
    Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S.
    General Accounting Office Washington, DC Orders may also be placed
    by calling (202) 512-6000 or by using fax number (202) 512-6061,
    or TDD (202) 512-2537. Each day, GAO issues a list of newly
    available reports and testimony.  To receive facsimile copies of
    the daily list or any list from the past 30 days, please call
    (202) 512-6000 using a touchtone phone.  A recorded menu will
    provide information on how to obtain these lists. For information
    on how to access GAO reports on the INTERNET, send an e-mail
    message with "info" in the body to: [email protected] or visit
    GAO's World Wide Web Home Page at: http://www.gao.gov PRINTED ON
    RECYCLED PAPER United States General Accounting Office
    Bulk Rate Washington, D.C. 20548-0001     Postage & Fees Paid GAO
    Permit No. G100 Official Business Penalty for Private Use $300
    Address Correction Requested

*** End of document. ***