    Yucca Mountain: Quality Assurance at DOE's Planned Nuclear Waste
    Repository Needs Increased Management Attention (17-MAR-06,
    GAO-06-313).

    The Department of Energy (DOE) is working to obtain a license
    from the Nuclear Regulatory Commission (NRC) to construct a
    nuclear waste repository at Yucca Mountain in Nevada. The
    project, which began in the 1980s, has been beset by delays. In a
    2004 report, GAO raised concerns that persistent quality
    assurance problems could further delay the project. Then, in
    2005, DOE announced the discovery of employee e-mails suggesting
    quality assurance problems, including possible falsification of
    records. Quality assurance, which establishes requirements for
    work to be performed under controlled conditions that ensure
    quality, is critical to making sure the project meets standards
    for protecting public health and the environment. GAO was asked
    to examine (1) the history of the project's quality assurance
    problems, (2) DOE's tracking of these problems and efforts to
    address them since GAO's 2004 report, and (3) challenges facing
    DOE as it continues to address quality assurance issues within
    the project.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-06-313
        ACCNO:   A49318
      TITLE:     Yucca Mountain: Quality Assurance at DOE's Planned
    Nuclear Waste Repository Needs Increased Management Attention
         DATE:   03/17/2006
      SUBJECT:   Accountability
    	     Licenses
    	     Nuclear waste disposal
    	     Nuclear waste management
    	     Nuclear waste storage
    	     Performance measures
    	     Program evaluation
    	     Quality assurance
    	     Schedule slippages
    	     DOE Yucca Mountain Project (NV)
    	     Yucca Mountain (NV)

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-06-313

         * Report to the Chairman, Subcommittee on the Federal Workforce and
           Agency Organization, Committee on Government Reform, House of
           Representatives
              * March 2006
         * YUCCA MOUNTAIN
              * Quality Assurance at DOE's Planned Nuclear Waste Repository Needs
                Increased Management Attention
         * Contents
              * Results in Brief
              * Background
              * DOE Has a Long History of Quality Assurance Problems at Yucca
                Mountain and Is Relying on Costly and Time-Consuming Measures to
                Correct Problems Before Submitting Its License Application for
                the Repository
                   * DOE Has Had Problems Implementing and Maintaining an
                     Effective Quality Assurance Program
                   * Recurring Issues with Project Data, Models, and Software
                     Illustrate DOE's Difficulties Addressing Quality Assurance
                     Problems
                   * DOE Is Now Relying on Costly and Time-Consuming Rework to
                     Resolve Additional Problems
              * DOE Cannot Be Certain Its Efforts to Improve Quality Assurance
                Have Been Effective Because of Weaknesses in Tracking Progress
                and Identifying Problems
                   * The Panel's Focus and Frequent Changes Hindered the Tracking
                     of Progress with Management Concerns and Quality Problems
                   * Trend Evaluation Reports Have Not Specifically Tracked the
                     Initiatives' Management Concerns and Have Had Weaknesses
                     Tracking Significant and Recurrent Problems for Management
                     Attention
                   * DOE Has Not Adequately Tracked Problems Being Addressed by
                     Ongoing Management Actions
              * DOE's `New Path Forward' for Preparing to Submit Its License
                Application Faces Substantial Quality Assurance and Other
                Challenges
                   * Determining the Extent of Problems with Relevant Documents
                     Will Delay DOE's Submission of the License Application
                   * Ongoing Design and Requirements Management Issues Could
                     Delay DOE's Submission of the License Application
                   * DOE Faces Challenges in Managing the Transition, Complexity,
                     and Continuity of Its `New Path Forward'
              * Conclusions
              * Recommendations for Executive Action
              * Agency Comments
         * Objectives, Scope, and Methodology
         * Yucca Mountain Project Employee Concerns Programs
         * GAO Contact and Staff Acknowledgments
         * PDF6-Ordering Information.pdf
              * Order by Mail or Phone

    Report to the Chairman, Subcommittee on the Federal Workforce and Agency
    Organization, Committee on Government Reform, House of Representatives

    March 2006

    YUCCA MOUNTAIN

    Quality Assurance at DOE's Planned Nuclear Waste Repository Needs
    Increased Management Attention

    Contents

    Tables

    Figures

    March 17, 2006Letter

    The Honorable Jon C. Porter Chairman Subcommittee on the Federal Workforce
    and Agency Organization Committee on Government Reform House of
    Representatives

    Dear Mr. Chairman:

    The nuclear waste created as a by-product of the nuclear power process in
    reactors can remain highly radioactive for hundreds of thousands of years,
    and will require proper disposal to protect public health and the
    environment. Over 50,000 metric tons of this waste is currently being
    stored at 72 sites around the country, principally at commercial nuclear
    power plants. These wastes have been accumulating for several decades in
    surface storage designed to be temporary. The Nuclear Waste Policy Act of
    1982 obligated the Department of Energy (DOE) to construct an underground
    geological repository for permanent storage and begin accepting these
    wastes by January 31, 1998. However, it was not until 2002, after more
    than 15 years of scientific study, that Yucca Mountain in Nevada was
    approved by Congress as a suitable location for the repository. DOE is
    continuing to experience delays, and it does not currently have a schedule
    for when construction of the repository will begin. The project to build
    and operate a repository at Yucca Mountain is highly complex. It is also
    highly controversial among some of the public, in large part, because of
    their concern that the repository may not be adequate, over the long term,
    to prevent the release of radioactive material to the environment. DOE has
    established quality assurance procedures to ensure that its work relative
    to the project and the technical information it produces are of high
    quality and defensible. However, persistent problems with implementing
    these procedures and resulting questions about the quality of the work
    have significantly contributed to project delays. Resolving these quality
    issues is essential to proceeding with construction.

    To construct a repository at the Yucca Mountain site, DOE must obtain a
    license from the Nuclear Regulatory Commission (NRC). As part of the
    licensing process, DOE must demonstrate to NRC that its plans for the
    repository will meet standards for protecting public health and the
    environment from harmful exposure to the radioactive waste. The
    Environmental Protection Agency (EPA) set these standards in 2001, but as
    a result of a 2004 court ruling, EPA is proposing to revise the standards
    to extend the protection period from 10,000 years to 1 million years.1

    To demonstrate that it can meet these standards, DOE has been conducting
    scientific and technical studies at the Yucca Mountain site that will
    serve as supporting documentation for DOE's planned license application.
    For example, it has developed mathematical models to measure the
    probability that various combinations of natural and engineered
    (human-made) features of a repository could safely contain waste for the
    long term; the models take into account possible water infiltration
    through the mountain (see fig. 1), earthquakes, volcanic action, or other
    scenarios. Thus, one of DOE's most important tasks during the licensing
    process will be to demonstrate the adequacy of its data, software, and
    models. Accordingly, NRC requires nuclear facilities to develop a quality
    assurance program that ensures that the technical information submitted in
    support of a license application-such as scientific data, models, and
    details on design and construction-is well documented and defensible. More
    specifically, data used to support conclusions about the safety and design
    of the repository must meet transparency and traceability standards. That
    is, the data must be clear in justifying and explaining any underlying
    assumptions, calculations, and conclusions, and must be capable of being
    traced back to original source materials.

    Figure 1: A Yucca Mountain Project Scientist Conducts Water Infiltration
    Tests inside Yucca Mountain

    To meet NRC's requirements, DOE established a quality assurance program
    for the Yucca Mountain project. The program establishes requirements that
    scientific, design, engineering, and other work, such as procurement and
    record keeping, is to be performed under controlled conditions that ensure
    quality and enable the work to be verified by others. For example, the
    program establishes general requirements for calibrating equipment before
    conducting tests, stipulating when and how the equipment should be
    calibrated and how to document the results. The project's line
    organizations, which are responsible for carrying out various functions or
    aspects of the work, then create their own policies and procedures to
    implement the requirements.

    Project employees are required to follow such procedures to help ensure
    the reliability of project information. Quality assurance auditors
    periodically verify that the procedures have been followed. Project
    employees, including quality assurance auditors, are required to identify
    when procedures are not being followed or when they encounter problems
    with the procedures. These problems can be identified in "condition
    reports" under the project's Corrective Action Program, which establishes
    procedures for the prompt identification and correction of problems.
    Alternatively, project employees can submit problems for resolution
    through the Employee Concerns Program, which allows for submissions to be
    confidential or anonymous.

    Because quality assurance plays a key role in ensuring that the
    information DOE uses to support its license application is of high quality
    and fully defensible, problems in this area raise concerns about delays to
    DOE's submission and NRC's review of the license application. In April
    2004, for example, we reported that recurring quality assurance problems
    at the Yucca Mountain project could delay the licensing and operation of
    the repository.2 As we noted, a 2004 NRC evaluation found quality
    assurance problems such as data that could not be readily traced back to
    their sources. NRC indicated that unless DOE rectified such problems
    before submitting the license application, NRC could be in the position of
    requesting large volumes of additional information, which could prevent it
    from making a decision on the license within the time required by law.
    Then, in early 2005, DOE reported it had discovered a series of e-mail
    messages among some U.S. Geological Survey (USGS) employees working on the
    Yucca Mountain project under a contract with DOE that appeared to imply
    that workers had falsified records for scientific work. Several of these
    messages, written in the late 1990s, appeared to show disdain for the
    project's quality assurance program and its requirements. As a result of
    these e-mails, DOE is engaging in an extensive review of records to
    restore confidence in scientific documents that will be used to support
    its license application.

    DOE's recent efforts to better manage quality assurance problems include
    its Management Improvement Initiatives (Initiatives), which began in 2002
    and were reported completed in April 2004. The Initiatives' purpose was to
    ensure that work and products consistently met quality objectives and were
    fully defensible by establishing a foundation for continuous improvement
    in areas of identified management weaknesses. In our 2004 report, we
    concluded that, while DOE considered the Initiatives to have been
    completed, it could not assess their effectiveness in addressing the
    management weaknesses because its performance goals lacked objective
    measures and time frames for determining success.3 By the end of the
    Initiatives, DOE had established two tools to alert management about
    quality-related and other problems: (1) a one-page summary of performance
    indicators for key project activities and processes (the summary, which
    DOE refers to as a "panel," is prepared monthly for discussion and action
    by project managers) and (2) quarterly trend evaluation reports analyzing
    patterns and trends in problems identified through the Corrective Action
    Program. Then, in October 2005, DOE initiated planning for an aggressive
    series of changes to the facility design, organization, and management of
    the Yucca Mountain project. This effort, known as the "new path forward,"
    is intended to address quality assurance and other challenges prior to
    submission of a license application. According to the project's Acting
    Director, DOE will be considering changes in performance indicators and
    other management tools to better support the new path forward.

    In this context, you requested that we provide additional information on
    the project's quality assurance problems and DOE's efforts to correct
    them. As agreed with your office, this report discusses (1) the history of
    the project's quality assurance problems since its start in the 1980s, (2)
    DOE's tracking of quality problems and progress implementing quality
    assurance requirements since our April 2004 report, and (3) challenges DOE
    faces as it continues to address quality assurance issues at the project.
    In addition, you asked for information about concerns raised in recent
    years through the project's Employee Concerns Program, which is provided
    in appendix II.

    To determine the history of quality assurance problems, we reviewed
    previous GAO, DOE, and NRC documents, visited the project, and interviewed
    officials from DOE, NRC, and Bechtel/SAIC Company, LLC (BSC), which is
    DOE's management contractor for the Yucca Mountain project. To assess
    DOE's tracking of quality-related problems and progress in addressing
    them, we examined management tools and associated documentation, such as
    monthly indicator panels and quarterly trend reports, and interviewed BSC
    and DOE officials regarding those tools. To identify current quality
    assurance and other challenges, we attended quarterly NRC management
    meetings, interviewed the Acting Director and other senior managers of the
    DOE project, and gathered information on management turnover. Due to the
    criminal investigation under way related to possible falsification of
    records implied in USGS e-mail exchanges, we did not examine the
    investigated issues beyond confirming that a concern about the e-mails had
    been submitted to the Employee Concerns Program. However, to determine if
    concerns about other instances of potential falsification of records had
    been raised by project employees, we reviewed employee concerns filed with
    the project's Employee Concerns Program from January 2004 to December
    2005. More information on our scope and methodology is provided in
    appendix I. We conducted our work from July 2005 through January 2006 in
    accordance with generally accepted government auditing standards.

    Results in Brief

    DOE has had a long history of quality assurance problems at the Yucca
    Mountain project. In the late 1980s and early 1990s, DOE had problems
    assuring NRC that it had developed adequate plans and procedures related
    to quality assurance. For example, as GAO reported in 1988, NRC had found
    that DOE's quality assurance procedures were inadequate and its efforts to
    independently identify and resolve weaknesses in the procedures were
    ineffective. By the late 1990s, DOE had largely addressed NRC's concerns
    about its plans and procedures, but its own audits identified quality
    assurance problems with the data, software, and models used in the
    scientific work supporting its potential license application. For example,
    in 1998, a team of project personnel determined that 87 percent of the
    models used to simulate the site's natural and environmental conditions,
    and to demonstrate the future repository's performance over time, did not
    comply with requirements for demonstrating their accuracy in predicting
    geologic events. More recently, as it prepares to submit the license
    application for the planned repository to NRC, DOE has been relying on
    costly and time-consuming rework to resolve lingering quality assurance
    concerns. For example, to address problems with the transparency and
    traceability of scientific work in technical documents, DOE implemented,
    in the spring of 2004, a roughly $20 million, 8-month project called the
    Regulatory Integration Team. This effort involved about 150 full-time
    employees from DOE, USGS, and multiple national laboratories, such as
    Sandia, Los Alamos, and Lawrence Livermore, working to inspect technical
    documents to identify and resolve quality problems.

    DOE cannot be certain that its efforts to improve the implementation of
    its quality assurance requirements have been effective because it adopted
    management tools that did not target existing management concerns and did
    not track progress with significant and recurring problems. Although DOE
    announced, in 2004, that it was making a commitment to continuous quality
    assurance improvement and that its efforts would be tracked by performance
    indicators that would enable it to assess progress and direct management
    attention as needed, its adopted management tools have not been effective
    for this purpose. Specifically, the one-page summary, or "panel," of
    selected performance indicators that project managers used in monthly
    management meetings was not an effective tool for assessing progress. The
    indicators selected for the panel poorly represented the major management
    concerns and changed frequently. For example, the panel did not include an
    indicator to represent the management concern about unclear roles and
    responsibilities-a problem that could undermine accountability within the
    project. Use of the indicator panel was discontinued in late 2005, and DOE
    is deciding on a tool to replace it. Moreover, a second management
    tool-trend evaluation reports-also did not track relevant concerns. The
    reports generally had technical weaknesses for identifying recurrent and
    significant problems and inconsistently tracked progress in resolving the
    problems. For example, lacking reliable data and an appropriate
    performance benchmark for determining the significance of human errors as
    a cause of quality problems, DOE's trend reports offered no clear basis
    for tracking progress on such problems. In addition, under the trend
    reports' rating categories, the rating assigned to convey the significance
    of a problem was overly influenced by a judgment that there were already
    ongoing management actions to address the problem, rather than solely
    assessing the problem's significance. For example, the trend report's
    rating of one particular problem at the lowest level of significance did
    not accurately describe the problem or sufficiently draw management's
    attention to it.

    Before DOE submits a license application, its aggressive "new path
    forward" effort faces substantial quality assurance and other challenges.
    First, the March 2005 announcement of the discovery of USGS e-mails
    suggesting the possible falsification of quality assurance records has
    resulted in extensive efforts to restore confidence in scientific
    documents, and DOE is conducting a wide-ranging review of approximately 14
    million e-mails to determine whether they raise additional quality
    assurance issues. Such a review creates a challenge not just because of
    the sheer volume of e-mails to be reviewed, but also because DOE will have
    to decipher their meaning and determine their significance, sometimes
    without clarification from authors who have left the project. Furthermore,
    if any of the e-mails raise quality assurance concerns, further review,
    inspection, or rework may need to be performed to resolve any newfound
    problems. Second, DOE faces quality assurance challenges in resolving
    design control problems associated with an inadequate requirements
    management process-the process responsible for ensuring that broad plans
    and regulatory requirements affecting the project are tracked and
    incorporated into specific engineering details. In December 2005, DOE
    issued a stop-work order on some design and engineering work until DOE can
    determine that the requirements management process has been improved.
    Third, DOE continues to be challenged by managing a changing and complex
    program and organization. Significant project changes initiated in October
    2005 under the new path forward create the potential for confusion over
    accountability as roles and responsibilities change-a situation DOE found
    to contribute to quality assurance problems during an earlier transition
    period. For example, a proposed reorganization, establishing a lead
    laboratory to assist the project, not only would have to be effectively
    managed, but also would introduce a new player whose accountability DOE
    would have to ensure. DOE has also experienced turnover in 9 of 17 key
    management positions since 2001-including positions related to quality
    assurance-that has created management continuity challenges. For example,
    the director position for the project has been occupied by three
    individuals since 1999 and is currently occupied by an acting director.
    Since DOE is still formulating its plans, it is too early to determine
    whether its new effort will effectively resolve these challenges.

    We are making recommendations to DOE aimed at improving the effectiveness
    of its management tools for monitoring performance in key areas, including
    quality assurance, by improving the tools' ability to identify problems
    and track progress in addressing them. We provided DOE and NRC with draft
    copies of this report for their review and comment. In comments, DOE
    agreed with our recommendations. Both DOE and NRC provided technical and
    editorial comments that we incorporated into the report, as appropriate.

    Background

    Congress enacted the Nuclear Waste Policy Act of 1982 to establish a
    comprehensive policy and program for the safe, permanent disposal of
    commercial spent fuel and other highly radioactive wastes in one or more
    mined geologic repositories. The act charged DOE with (1) establishing
    criteria for recommending sites for repositories; (2) "characterizing"
    (investigating) three sites to determine each site's suitability for a
    repository (1987 amendments to the act directed DOE to investigate only
    the Yucca Mountain site); (3) recommending one suitable site to the
    President, who, if he considered the site qualified for a license
    application, would submit a recommendation to Congress; and (4) seeking a
    license from NRC to construct and operate a repository at the approved
    site. The act created the Office of Civilian Radioactive Waste Management
    within DOE to manage its nuclear waste program.

    Since the 1980s, DOE has spent years conducting site characterization
    studies at the Yucca Mountain site to determine whether it is suitable for
    a high-level radioactive waste and spent nuclear fuel repository. DOE, for
    example, has completed numerous scientific studies of the mountain and its
    surrounding region for water flow and the potential for rock movement,
    including volcanoes and earthquakes that might adversely affect the
    performance of the repository. To allow scientists and engineers greater
    access to the rock being studied, DOE excavated two tunnels for studying
    the deep underground environment: (1) a five-mile main tunnel that loops
    through the mountain, with several research areas or alcoves connected to
    it; and (2) a 1.7-mile tunnel that crosses the mountain (see fig. 2). This
    second tunnel allows scientists to study properties of the rock and the
    behavior of water near the potential repository area. In July 2002,
    Congress approved the President's recommendation of the Yucca Mountain
    site for the development of a repository.

    Figure 2: The 1.7-Mile Tunnel Built for Scientific Studies near the
    Potential Repository Area

    The Yucca Mountain project is currently focused on preparing an
    application to obtain a license from NRC to construct a repository. The
    required application information includes both repository design work and
    scientific analyses. DOE is engaged in necessary tasks such as compiling
    information and writing sections of the license application, and is
    conducting technical exchanges with NRC staff and addressing key technical
    issues identified by NRC to ensure that sufficient supporting information
    is provided. It also plans to further develop the design of the
    repository, including revised designs for the repository's surface
    facilities and canisters to hold the waste. DOE is also identifying and
    preparing potentially relevant documentary material that it is required to
    make available on NRC's Web-based information system, known as the
    Licensing Support Network. This is a critical step because DOE is required
    to certify that the documentary material has been identified and made
    electronically

    available no later than 6 months in advance of submitting the license
    application.4

    In February 2005, DOE announced that it does not expect the repository to
    open until 2012 at the earliest, which is more than 14 years later than
    the 1998 goal specified by the Nuclear Waste Policy Act of 1982. More
    recently, the conference report for DOE's fiscal year 2006 appropriations
    observed that further significant schedule slippages for submitting a
    license application are likely. Further delays could arise from factors
    such as the time needed for EPA to establish revised radiation standards
    for Yucca Mountain and for DOE to revise its technical documents in
    response. Such delays could be costly because nuclear utilities, which pay
    for most of the disposal program through a fee on nuclear power, have sued
    DOE, seeking damages for not starting the removal of spent nuclear fuel
    from storage at commercial reactors by the 1998 deadline. Estimates of the
    potential damages vary widely, from DOE's estimate of about $5 billion to
    a nuclear industry's estimate of about $50 billion, but the cost for the
    damages will likely rise if there are further delays to opening the
    repository.

    Given these schedule slippages, Congress has considered other options for
    managing existing and future nuclear wastes, such as centralized interim
    storage at one or more DOE sites. The conference report for DOE's fiscal
    year 2006 appropriations directed DOE to develop a spent nuclear fuel
    recycling plan to reuse the fuel. However, according to the policy
    organization of the nuclear energy industry, no technological option
    contemplated will eliminate the need to ultimately dispose of nuclear
    waste in a geologic repository.

    In October 2005, the project's Acting Director issued a memorandum calling
    for the development of wide-ranging plans for the "new path forward,"
    DOE's effort to address quality assurance and other challenges prior to
    applying for a license. To restore confidence in scientific documents that
    will support the license application, some of the plans will address the
    need to review and replace USGS work products, a requirement for USGS to
    certify its scientific work products, and establishing a lead national
    laboratory to assist the project. Other plans are focused on a new
    simplified design for the waste canisters and repository facilities, a
    design that is expected to improve the safety and operation of the
    repository by eliminating the need to directly handle and process the
    spent fuel at the repository. Further, this aggressive effort called for
    management changes, including a transition plan; more rigorous project
    management, including a new baseline schedule; rescoping existing
    contracts and developing new contracts; tracking project hiring actions; a
    financial plan; and new reporting indicators.

    After DOE submits the license application, NRC plans to take 90 days to
    examine the application for completeness to determine whether DOE has
    addressed all NRC requirements. One of the reviews for completeness will
    include an examination of DOE's documentation of the quality assurance
    program to assess whether it addresses all NRC criteria. These criteria
    include, among other things, organization, design control, document
    control, corrective actions, quality assurance records, and quality
    audits. If it deems any part of the application is incomplete, NRC may
    either reject the application or require that DOE furnish the necessary
    documentation before proceeding with the detailed technical review of the
    application. If it deems the application is complete, NRC will docket the
    application, indicating its readiness for a detailed technical review.5

    Once the application is accepted and placed on the docket, NRC will
    conduct its 18-month technical review of the application to determine if
    the application meets all NRC requirements, including the soundness of
    scientific analyses and preliminary facility design, and NRC quality
    assurance criteria. If NRC discovers problems with the technical
    information used to support the application, it may conduct specific
    reviews, including inspections, to determine the extent and effect of the
    problem. Because the data, models, and software used in modeling
    repository performance are integral parts of this technical review,
    quality assurance plays a key role since it is the mechanism used to
    verify the accuracy of the information DOE presents in the application.
    NRC may conduct reviews, including inspections, of the quality assurance
    program if technical problems are identified that are attributable to
    quality problems. NRC will hold public hearings chaired by its Atomic
    Safety and Licensing Board to examine specific topics. After completing
    the proceedings, the board will forward its initial decision to the NRC
    commissioners for their review. Finally, within 3 to 4 years from the date
    that NRC dockets the application, NRC will make a decision to grant the
    construction authorization, reject the application, or grant the
    construction authorization with conditions.6 NRC will grant a construction
    authorization only if it concludes from its reviews that the repository
    would meet its reasonable expectation that the safety and health of
    workers and the public would be protected.

    DOE Has a Long History of Quality Assurance Problems at Yucca Mountain and
    Is Relying on Costly and Time-Consuming Measures to Correct Problems
    Before Submitting Its License Application for the Repository

    DOE has repeatedly experienced quality assurance problems with its work on
    the Yucca Mountain project. In the late 1980s, DOE had been challenged to
    fix and develop adequate plans and procedures related to quality
    assurance. By the late 1990s, audits by GAO, DOE, and others identified
    recurring quality assurance problems with several aspects of key
    scientific data, models, and software. Currently, in preparing to submit
    the license application to NRC, DOE is relying on costly and
    time-consuming rework to resolve lingering quality assurance problems with
    the transparency and traceability of data and in project design and
    engineering documents uncovered during audits and after-the-fact
    evaluations.

    DOE Has Had Problems Implementing and Maintaining an Effective Quality
    Assurance Program

    DOE has a long-standing history of attempting to address NRC concerns
    about its quality assurance program. Although NRC will have responsibility
    for regulating the construction, operation, and decommissioning (closure)
    phases of the project, its regulatory and oversight role does not begin
    until DOE submits a license application. As a result, NRC's role in the
    project has been limited to providing guidance to DOE to ensure an
    understanding of NRC regulations and that the years of scientific and
    technical work will not later be found inadequate for licensing purposes.
    Specifically, since 1984, NRC has agreed to point out problems it
    identifies with the quality assurance program so that DOE can take timely
    corrective action. Initially, this NRC guidance was mainly focused on
    ensuring that DOE had the necessary quality assurance organization, plans,
    and procedures.

    As we reported in 1988, NRC had reviewed DOE's quality assurance plans and
    procedures comprising the principal framework of its quality assurance
    program, and concluded that they were inadequate and did not meet NRC
    requirements.7 NRC also concluded that DOE's efforts to independently
    identify and resolve weaknesses in the plans and procedures were
    ineffective. After observing DOE quality assurance audits, NRC determined
    that the audits were ineffective for measuring whether quality assurance
    procedures were being effectively implemented. Further, NRC identified
    additional concerns, during the 1980s, related to DOE management and
    organizational deficiencies relating to the quality assurance program.
    Specifically, among other things, NRC found the following:

    o DOE had a small staff and relied heavily on contractors to provide
    quality assurance oversight. Based on its experience in regulating nuclear
    power plants, NRC found that these types of organizations frequently
    developed major quality-related problems.

    o DOE had indirect project control, with administrative and functional
    control over the project split between different offices. NRC found that
    such project control arrangements tend to have serious quality
    assurance-related problems because conflicts can arise between quality and
    other organizational goals, such as cost and schedule.

    o During a 1984 NRC visit to Nevada, DOE project participants had
    expressed the opinion that quality assurance is "unnecessary, burdensome,
    and an imposition." Further, in 1986, DOE issued a stop-work order to the
    USGS based on a determination that USGS staff did not appreciate the
    importance of quality assurance and that USGS work would not meet NRC
    expectations. NRC believed that organizational attitudes can indicate
    whether a project is likely to experience problems relating to quality
    assurance and found such examples troublesome.

    Finally, based in part on the information obtained from its oversight
    activities, NRC concluded, in 1989, that DOE and its key contractors had
    yet to develop and implement an acceptable quality assurance program.

    However, by March 1992, NRC came to the conclusion that DOE had made
    significant progress in improving its quality assurance program. NRC noted
    that DOE had addressed many of its concerns, specifically that, among
    other things, (1) all of the contractor organizations had developed and
    were in the process of implementing quality assurance programs that met
    NRC requirements, (2) quality assurance management positions had been
    filled with full-time DOE personnel with appropriate knowledge and
    experience, and (3) DOE had demonstrated that it is capable of evaluating
    and correcting deficiencies in the overall quality assurance program.
    Nevertheless, in October 1994, NRC found problems with quality assurance,
    particularly with the site contractor's ability to effectively implement
    corrective actions and DOE's ability to oversee the site contractor's
    quality assurance program.

    Recurring Issues with Project Data, Models, and Software Illustrate DOE's
    Difficulties Addressing Quality Assurance Problems

    As DOE's quality assurance program matured, it resolved NRC concerns about
    its organization, plans, and procedures, and in the late 1990s began
    successfully detecting new quality assurance problems in three areas
    critical to the repository's successful performance: the adequacy of the
    data sources, the validity of scientific models, and the reliability of
    computer software developed at the site. These problems surfaced in 1998
    when DOE began to run the initial version of its performance assessment
    model. Specifically, DOE was unable to ensure that critical project data
    had been collected and tracked back to the original sources. In addition,
    DOE did not have a standardized process for developing scientific models
    used to simulate a variety of geologic events or an effective process for
    ensuring that computer software used to support the scientific models
    would work properly. As required by DOE's quality assurance procedures,
    the department conducted a root cause analysis and issued a corrective
    action plan in 1999. After corrective actions were taken, DOE considered
    the issues resolved.

    However, in 2001, similar deficiencies associated with models and software
    resurfaced. DOE attributed the recurrence to ineffective procedures and
    corrective actions, improper implementation of quality procedures by line
    managers, and personnel who feared reprisal for expressing quality
    concerns. Recognizing the need to correct these recurring problems, DOE
    conducted a comprehensive root cause analysis that included reviews of
    numerous past self-assessments and independent program assessments, and
    identified weaknesses in management systems, quality processes, and
    organization roles and responsibilities. Following the analysis, in July
    2002, DOE issued its Management Improvement Initiatives (Initiatives) that
    addressed quality problems with software and models. In addition, DOE
    added other corrective actions to address management weaknesses that it
    found in areas such as roles and responsibilities, quality assurance
    processes, written procedures, corrective action plans, and work
    environment.

    However, DOE continued to face difficulties in resolving quality assurance
    problems concerning the data, software, and modeling to be used in support
    of the licensing application:

    o Data management. As part of NRC's quality assurance requirements, data
    used to support conclusions about the safety and design of the repository
    must be either collected under a quality assurance program or subjected to
    prescribed testing procedures to ensure the data are accurate for their
    intended use. In addition, the data supporting these conclusions must also
    be traceable back to its original source. In 1998, DOE identified quality
    assurance problems with the quality and traceability of data-specifically
    that some data had not been properly collected or tested to ensure their
    accuracy and that data used to support scientific analysis could not be
    properly traced back to their source. DOE again found similar problems in
    April and September 2003, when a DOE audit revealed that some data sets
    did not have the documentation necessary to trace them back to their
    sources; the processes for data control and management were
    unsatisfactory; and faulty definitions were developed, which allowed
    unqualified data to be used.

    o Software management. DOE quality assurance procedures require that
    software used to support analysis and conclusions about the performance
    and safety of the repository be tested or created in such a way to ensure
    that it is reliable. From 1998 to 2003, multiple DOE audits found
    recurring quality assurance problems that could affect confidence in the
    adequacy of software codes. For example, in 2003, DOE auditors found
    problems related to software similar to those found previously in areas
    such as technical reviews, software classification, planning, design, and
    testing. Further, a team of industry professionals hired by DOE to assess
    quality assurance problems with software reported in February 2004 that
    these problems kept recurring because DOE did not assess the effectiveness
    of its corrective actions and did not adequately identify the root causes
    of the problems.

    o Model validation. Models are used to simulate natural and environmental
    conditions at Yucca Mountain, and to demonstrate the performance of the
    future repository over time. However, before models can be used to support
    the license application, DOE must demonstrate through a process called
    validation that the models are able to accurately predict geologic events.
    In 1998, a team of project personnel evaluated the models and determined
    that 87 percent did not comply with the validation requirements. In 2001,
    and again in 2003, DOE audits found that project personnel were not
    properly following procedures-specifically in the areas of model
    documentation, model validation, and checking and review. Further, the
    2003 audit concluded that previous corrective actions designed to improve
    validation and reduce errors in model reports were not fully implemented.

    After many years of working to address these quality assurance problems
    with data, software, and models, DOE had mostly resolved these problems
    and closed the last of the associated condition reports by February 2005.

    DOE Is Now Relying on Costly and Time-Consuming Rework to Resolve
    Additional Problems

    As DOE prepares to submit the Yucca Mountain project license application
    to NRC, it has relied on costly and time consuming rework to ensure that
    the documents supporting the application are accurate and complete.
    Specifically, DOE has relied on inspections and rework by DOE personnel to
    resolve quality assurance problems with the traceability and transparency
    of technical work products. These efforts to deal with quality problems at
    the end, rather than effectively ensuring that work organizations are
    producing quality products from the beginning, add to the project's cost
    and could potentially delay DOE's submission of the license application to
    NRC. In addition, DOE's efforts indicate that some corrective actions have
    been ineffective in resolving problems with the quality assurance process.
    Further, DOE is now detecting quality assurance problems in design and
    engineering work that are similar to the quality assurance problems it
    experienced with its scientific work in the late 1990s.

    Although DOE did not initiate its major effort to address these problems
    until 2004, the department and NRC for years had known of quality
    assurance problems with the traceability and transparency of technical
    work products called Analysis and Model Reports (AMR). AMRs are a key
    component of the license application, and contain the scientific analysis
    and modeling data demonstrating the safety and performance of the planned
    repository. Among other quality requirements, AMRs must be traceable back
    to their original source material and data, and must also be transparent
    in justifying and explaining their underlying assumptions, calculations,
    and conclusions. In 2003, based in part on these problems as well as DOE's
    long-standing problems with data, software, and modeling, NRC conducted an
    independent evaluation of three AMRs. The scope of the review was to
    determine if the AMRs met NRC requirements for being traceable,
    transparent, and technically appropriate for their use in the license
    application. NRC found significant problems.8 First, in some cases DOE was
    not transparent in explaining the basis on which it was reaching
    conclusions. For example, in some circumstances, DOE selected a single
    value from a range of data without sufficient justification. Other times,
    DOE did not explain how a range of experimental conditions were
    representative of repository conditions. Second, where DOE did
    sufficiently explain the basis for a conclusion, it did not always provide
    the necessary technical information, such as experimental data, analysis,
    or expert judgment, to trace the support for that explanation back to
    source materials. For example, DOE did not explain how information on one
    type of material provided an appropriate comparison for another material.
    Moreover, while DOE had identified similar problems in the past, the
    actions taken to correct them did not identify and resolve other
    deficiencies. NRC concluded that these findings suggested that other AMRs
    possibly had similar problems, and that if not resolved, such problems
    could delay NRC's review of the license application as it would need to
    conduct special inspections to resolve any issues it found with the
    quality of technical information.

    To address problems of traceability and transparency, DOE in the spring of
    2004 initiated an effort called the Regulatory Integration Team (RIT) to
    perform a comprehensive inspection and rework of the AMRs to ensure they
    met NRC requirements and expectations.9 According to DOE officials, the
    RIT involved roughly 150 full-time personnel from DOE, USGS, and multiple
    national laboratories such as Sandia, Los Alamos, and Lawrence Livermore.
    First, the RIT screened all of the approximately 110 AMRs and prioritized
    its efforts on 89 that needed additional rework. Ten AMRs were determined
    to be acceptable, and 11 were canceled because they were no longer needed
    to support the license application. According to DOE officials,
    approximately 8 months later, the RIT project was completed at a cost of
    about $20 million, with a total of over 3,700 problems and issues
    addressed or corrected. In February 2005, in a letter to DOE, the site
    contractor stated that the RIT effort was successful and that the AMRs had
    been revised to improve traceability and transparency.

    Subsequently, however, additional problems with traceability and
    transparency have been identified, requiring further inspections and
    rework. For example, after the March 2005 discovery of e-mails from USGS
    employees written between May 1998 and March 2000 implying that employees
    had falsified documentation of their work to avoid quality assurance
    standards, DOE initiated a review of additional AMRs that were not
    included in the scope of the 2004 RIT review. The additional AMRs
    contained scientific work performed by the USGS employees and had been
    assumed by the RIT to meet NRC requirements for traceability and
    transparency. However, according to DOE officials, DOE's review determined
    that these AMRs did not meet NRC's standards, and additional rework was
    required. Further, similar problems were identified as the focus of the
    project shifted to the design and engineering work required for the
    license application. In February 2005, the site contractor determined that
    in addition to problems with AMRs, similar traceability and transparency
    problems existed in the design and engineering documents that comprise the
    Safety Analysis Report-the report necessary for demonstrating to NRC how
    the facilities and other components of the repository site will meet the
    project's health, safety, and environmental goals and objectives. In a
    root cause analysis of this problem, the site contractor noted that
    additional resources were needed to inspect and rework the documents to
    correct the problems.

    DOE Cannot Be Certain Its Efforts to Improve Quality Assurance Have Been
    Effective Because of Weaknesses in Tracking Progress and Identifying
    Problems

    DOE cannot be certain that it has met continuous improvement goals for
    implementing its quality assurance requirements, a commitment DOE made at
    the closure of its Management Improvement Initiatives (Initiatives) in
    April 2004. At that time, DOE told us it expected that the progress
    achieved with the initiatives would continue and that its performance
    indicators would enable it to assess further progress and direct
    management attention as needed. However, DOE's performance indicators, as
    well as a second management tool-trend evaluation reports-have not been
    effective for this purpose. More specifically, the indicators panel did
    not highlight the areas of concern covered by the initiatives and had
    weaknesses in assessing progress because the indicators kept changing. The
    trend evaluation reports also did not focus on tracking the concerns
    covered by the Initiatives, had technical weaknesses for identifying
    significant and recurring problems, had inconsistently tracked progress in
    addressing problems, and could not fully analyze projectwide problems.10
    In addition, the trend reports' tracking of problems for which corrective
    actions were already being taken was at times overly influenced by
    judgments about whether additional management action was warranted rather
    than the problems' significance.

    The Panel's Focus and Frequent Changes Hindered the Tracking of Progress
    with Management Concerns and Quality Problems

    By the time that the actions called for by the Initiatives had been
    completed in April 2004, project management had already developed the
    indicators panel, which DOE refers to as the annunciator panel, to use at
    monthly management meetings to monitor project performance. The panel was
    a single page composed of colored blocks representing selected performance
    indicators and their rating or level of performance. A manager viewing the
    panel would be able to quickly see the color rating of each block or
    indicator. For example, red indicated degraded or adverse performance
    warranting significant management attention; yellow indicated performance
    warranting increased management attention or acceptable performance that
    could change for the worse; and green indicated good performance. The
    panel represented a hierarchy of indicators in which the highest level, or
    primary, indicators were shown; secondary indicators that determined the
    primary indicators' ratings were shown for some primary indicators; but
    lower third- or fourth-level indicators were not shown. Our review
    analyzed a subset of these indicators that DOE designated as the
    indicators that best predict performance in areas affecting quality. While
    we were conducting our review, DOE suspended preparation of the panel
    after August 2005 while it reconsiders its use of indicators to monitor
    project performance. DOE had also suspended preparation of the panel from
    late 2004 to early 2005 in order to make substantial revisions. These
    revisions were made, in part, to emphasize fewer, more important
    indicators for management attention.

    The Initiatives raised concerns about five key areas of management
    weakness as adversely affecting the implementation of quality assurance
    requirements:

    1.Roles and responsibilities were becoming confused as the project
    transitioned from scientific studies to activities supporting licensing.
    The confusion over roles and responsibilities was undermining managers'
    accountability for results. The Initiatives' objective was to realign
    DOE's project organization to give a single point of responsibility for
    project functions, such as quality assurance and the Corrective Action
    Program, and hold the project contractor more accountable for performing
    the necessary work in accordance with quality, schedule, and cost
    requirements.

    2.Product quality was sometimes being achieved through inspections by the
    project's Office of Quality Assurance rather than being routinely
    implemented by the project's work organizations. As a result, the
    Initiatives sought to increase work organizations' responsibility for
    being the principle means for achieving quality.

    3.Work procedures were typically too burdensome and inefficient, which
    impeded work. The Initiatives sought to provide new user-friendly and
    effective procedures, when necessary, to allow routine compliance with
    safety and quality requirements.

    4.Multiple corrective action programs existed, processes were burdensome
    and did not yield useful management reports, and corrective actions were
    not completed in a timely manner. The Initiatives sought to implement a
    single program to ensure that problems were identified, prioritized, and
    documented and that timely and effective corrective actions were taken to
    preclude recurrence of problems.

    5.The importance of a safety-conscious work environment that fosters open
    communication about concerns was not understood by all managers and staff,
    and they had not been held accountable when inappropriately
    overemphasizing the work schedule, inadequately attending to work quality,
    and acting inconsistently in practicing the desired openness about
    concerns. Through issuing a work environment policy, providing training on
    the policy, and improving the Employee Concerns Program, the Initiatives
    sought to create an environment in which employees felt free to raise
    concerns without fear of reprisal and with confidence that issues would be
    addressed promptly and appropriately.

    As shown in table 1, the Initiatives' effectiveness indicators for
    tracking progress in addressing these management weaknesses did not have
    equivalent performance indicators visible in the annunciator panel when it
    was prepared for the last time, using August 2005 data.

    Table 1: Visibility of Management Improvement Initiatives' Effectiveness
    Indicators in Annuciator Panel When Last Prepared (Using August 2005 Data)

           Key area of        Effectiveness     DOE response on  GAO comments and
    management weakness indicators from the  coverage of the    observations
        identified in the      Initiatives         management
           Initiatives                          weakness in the
                                                    panel's
                                                  performance
                                                   indicators
    Roles,              An improving trend   No integrated    No indicator was
    responsibilities,   in quality and work  analysis of      visible in, or
                           schedule             trends in        underlies, the
    accountability,     performance.         quality and      panel.
    authority                                schedule
                                                performance.     Some indicators
                                                                 measured aspects
                                                                 of quality or
                                                                 schedule, but
                                                                 provided no
                                                                 integrated
                                                                 analysis of
                                                                 these trends.
                           A consistently       No aspect        No indicator was
                           decreasing trend in  measured.        visible in, or
                           quality problems                      underlies, the
                           related to roles and                  panel.
                           responsibilities.
    Quality assurance   The numbers of       One indicator    No indicator,
    programs and        high-priority        looked at work   focused only on
    processes           (significant)        organizations'   significant
                           quality problems     identification   problems, was
                           that are             of problems,     visible in the
                           self-identified are  including less   panel.
                           at least 80 percent  significant
                           of all significant   ones.            One fourth-level
                           quality problems.                     indicator
                                                                 tracked work
                                                                 organizations'
                                                                 identification
                                                                 of significant
                                                                 problems.
                           A decreasing trend   A new timeliness No indicator was
                           in average time to   measure has been visible in
                           resolve significant  developed.a      panel.
                           quality
                                                                 Aspect of
                           problems and in                       fourth-level
                           number of delinquent                  indicator
                           corrective actions                    tracked average
                           for significant                       time of
                           quality problems.                     resolution.
    Work procedures     A decreasing number  No aspect        No indicator was
                           of quality problems  measured.        visible in, or
                           related to                            underlies, the
                           ineffective                           panel.

                           Procedures.
                           A decreasing trend   No aspect        No indicator was
                           in time needed to    measured.        visible in, or
                           revise procedures.                    underlies, the
                                                                 panel.
                           A decreasing trend   No aspect        No indicator was
                           in average time of   measured.        visible in, or
                           interim procedure                     underlies, the
                           changes.                              panel.
    Corrective Action   A decreasing trend   No aspect        No indicator was
    Program             in number of         measured.        visible in, or
                           repetitive quality                    underlies, the
                           problems.                             panel.
                           A decreasing trend   A new timeliness No indicator was
                           in average time to   measure has been visible in
                           resolve significant  developed.a      panel.
                           quality
                                                                 Aspect of
                           problems.                             fourth-level
                                                                 indicator
                                                                 tracked average
                                                                 time of
                                                                 resolution.
                           Less than 10 percent A new timeliness No indicator was
                           of quality problems  measure has been visible in the
                           are resolved late.   developed.a      panel.

                                                                 A third-level
                                                                 indicator
                                                                 tracked
                                                                 percentage of
                                                                 problems with
                                                                 timely
                                                                 resolution.
    Work environment    A decreasing number  Aspects of this  No indicator was
                           of substantiated     issue are        visible in
                           employee concerns    measured by work panel.
                           for harassment,      environment
                           retaliation,         indicators.      A third-level
                           intimidation, and                     indicator
                           discrimination.                       measured this
                                                                 performance.
                           Evaluation of        Goals have       No indicator was
                           routine employee     remained at 30   visible in
                           concerns in less     and 90 days.     panel.
                           than 30 days, or
                                                                 Third-level
                           90 days for complex                   indicators
                           employee concerns                     measured the
                           involving harassment                  timely
                           or intimidation.                      completion of
                                                                 routine and
                                                                 other concerns.
                           External evaluation  External         No indicator was
                           of work environment  evaluation is    visible in
                           shows positive       accomplished     panel.
                           changes.             through
                                                independent      Four third-level
                                                employee         indicators were
                                                surveys,         based on the
                                                reflected in     employee
                                                third-level      surveys.
                                                indicators.

    Source: GAO analysis of DOE data.

    aNew timeliness indicator was not implemented by the time of the final
    panel using August 2005 data.

    Two of the Initiatives' key areas of concern-(1) roles, responsibilities,
    authority, and accountability; and (2) work procedures-and their
    associated effectiveness indicators were not represented in the panel's
    visible or underlying indicators. The Initiatives' effectiveness indicator
    for tracking trends in recurring problems also was not represented. In
    other cases, the Initiatives' effectiveness indicators were represented in
    underlying lower-level indicators that had very little impact on the
    rating of the visible indicator. An example is the Initiatives' indicator
    for timely completion of employee concerns. The panel's related visible
    indicator was work environment, whose rating was based on 4 secondary and
    23 tertiary indicators. Of the third-level indicators, two were for
    timeliness of completion of employee concerns, and combined they
    contributed 3 percent toward the rating of the work environment indicator.
    As a result of the weighting of these many underlying indicators, ratings
    for individual lower-level indicators could be different from the visible
    indicator. For example, in August 2005, the work environment indicator
    showed good performance. However, the ratings of four underlying
    indicators from the project's employee survey on the work
    environment-collectively accounting for 25 percent of the work environment
    indicator's score-indicated the need for increased management attention.
    Moreover, some of the Initiatives' indicators, such as the work
    organizations' self-identification of significant problems, had their
    impact on visible indicators diluted by the inclusion of other indicators
    that were not focused solely on the detection of significant problems.

    Another shortcoming of the annunciator panel was that frequent changes to
    the indicators hindered the ability to identify problems for management
    attention and track progress in resolving them. The indicators could
    change in many ways, such as changes in their definition, calculation, or
    data sources used in calculations, or from the deletion or addition of a
    subindicator. When such changes were made to the indicators, progress
    became less clear because changes in reported performance levels may have
    been the result of the indicator changes rather than actual performance
    changes. Some of the indicators for key project processes with quality
    elements changed from one to five times during the 8-month period from
    April 2004 through November 2004. Even after the major revision of the
    panel in early 2005, most of the performance indicators tracking quality
    issues continued to change over the next 6 months-that is, from March 2005
    through August 2005. As shown in table 2, only one of the five relevant
    indicators did not change during this period. One indicator was changed
    four times during the 6-month period, resulting in it being different in
    more months than it remained the same.

    Table 2: Key Indicators for Processes with Quality Elements, Their
    Intended Focus, and Number of Times They Changed (March through August
    2005)

             Indicators                     Intended focus              Number of
                                                                           months
                                                                          changed
    Performance improvement Effectiveness of self-assessment of              4
                               quality and other issues, lessons
                               learned, and Corrective Action Program
    Work management         Quality of work products and documents           1
    Safety-conscious work   Worker confidence in management support          0
    environment             for raising quality and other concerns
                               without fear of retaliation; management
                               effectiveness in detecting and
                               preventing retaliation for raising
                               concerns; effectiveness of normal and
                               alternative problem resolution
    Human performance       Preventing, detecting, and correcting            3
                               human errors
    Quality performance     Composite of quality indicators, in              1
                               areas of engineering products,
                               self-assessment, Corrective Action
                               Program, and work products and documents

    Source: GAO analysis of DOE data.

    Moreover, the panel was not always available to identify problems and
    track progress. The panel was not created for December 2004, January 2005,
    and February 2005 because it was undergoing a major revision. At that
    time, DOE told NRC that the performance indicators for the panel were
    revised to reflect the change in the work as the project moved into the
    engineering, procurement, and construction phase. DOE also reduced the
    total number of visible indicators from 60 to 30 to focus on fewer, more
    critical aspects of project management. Panels with the new indicators
    were then produced for 6 months, starting with March 2005 and ending after
    August 2005. This second interruption of the panels resulted from another
    major revision to the indicators; this time, indicators are being made
    congruent with project work as designated by DOE's "new path forward,"
    again to focus on fewer, more important activities. In December 2005, a
    senior DOE official told us that the project would begin to measure key
    activities, but without use of the panel.

    Trend Evaluation Reports Have Not Specifically Tracked the Initiatives'
    Management Concerns and Have Had Weaknesses Tracking Significant and
    Recurrent Problems for Management Attention

    According to DOE, some of the Initiatives' areas of concern and their
    associated effectiveness indicators-for example, trends in quality
    problems related to roles and responsibilities-were being captured, at
    least partially, in the project's quarterly trend evaluation reports
    rather than in the performance indicators. However, the trend reports are
    a management tool designed more to identify emerging and unanticipated
    problems than to monitor progress with already identified problems, such
    as those addressed by the Initiatives. In developing these reports, trend
    analysts seek to identify patterns and trends in condition reports (CR),
    which document problematic conditions through the project's Corrective
    Action Program. The trend reports analyze CRs for more significant
    problems (Levels A and B) and minor problems (Level C), but not at Level D
    (opportunities for improvement). The trend analysis typically separates
    the reported problems into categories such as organizational unit, type of
    problem, and cause. These categories are intended to provide insights into
    the problems. For example, analysis might reveal that most occurrences of
    a particular type of problem are associated with a certain organization.

    In practice, DOE missed opportunities to use trend reports to call
    attention to progress in the Initiatives' areas of concern. For example,
    the Initiatives sought to clarify roles and responsibilities within and
    between DOE and BSC to ensure clear accountability for project results
    during the project's transition from scientific studies to the design and
    engineering activities necessary to license a repository. Similar
    organizational transition problems were identified in the November 2004
    trend report. While that report attributed increases in the number of
    causal factors associated with change management, supervisory methods, and
    work organization to recent BSC reorganizations and changes in the project
    from science-based to design and engineering activities, it did not
    specifically mention issues of roles and responsibilities or that roles
    and responsibilities was an Initiatives' area of concern. However, an
    analysis of the cause of the problems noted in various significant
    condition reports, which is performed for certain condition reports and
    outside of the process of developing trend reports, found evidence of
    weaknesses in the organizational interfaces among BSC organizations, as
    well as between BSC and DOE. According to this cause analysis, these
    organizational interface weaknesses were associated with some manner of
    change and represented weaknesses in the definition of roles and
    responsibilities. Trend reports are generally based on condition reports,
    and problems with roles and responsibilities seem to be identified in
    cause analyses rather than in the condition reports themselves.

    Similarly, DOE missed an opportunity to use trend reports to discuss the
    Initiatives' goal that the project's line or work organizations become
    more accountable for self-identifying significant problems. The August
    2005 trend report briefly cited an evaluation of a CR highlighting the low
    rate of self-identification of significant problems during the previous
    quarter and reported the evaluation's conclusion that it was not a problem
    warranting management attention. However, the trend report did not mention
    that about 35 percent of significant problems were self-identified during
    the previous quarter, while the Initiatives' goal was that 80 percent of
    significant problems would be self-identified. Thus, the trend report
    missed an opportunity to either raise a performance problem or pose the
    question of whether the Initiatives' goal needed to be reassessed.

    Beyond whether they effectively tracked the Initiatives' areas of concern,
    trend reports face important obstacles, in general, to adequately identify
    recurrent and significant problems:

    o Recurring or similar conditions can be difficult to clearly identify for
    management's attention and resolution. A trend report noted that there
    will be few cases where recurrent conditions are obvious because each
    condition slightly differs.

    o Trend analysis tends to focus on the number of CRs issued, but the
    number of CRs does not necessarily reflect the significance of a problem.
    For example, the number of CRs involving requirements management decreased
    by over half from the first quarter to the second quarter of fiscal year
    2005. However, this decrease was not a clear sign of progress. Not only
    did the number rise again in the third quarter, but the May 2005 trend
    report also noted that the number of all condition reports had dropped
    during the second quarter. According to the report, the volume of CRs in
    the first quarter had been high because of reviews of various areas,
    including requirements management. Another example is the records
    management problem. The November 2005 trend report stated that a records
    management problem identified in various CRs, despite accounting for about
    50 percent of all business administration problems, reflected an
    underlying error rate of less than 1 percent and thus was not a
    significant problem.

    o The lack of an increasing trend in the number of reported problems does
    not necessarily mean the lack of a significant problem for management
    attention. Knowing the appropriate level of performance, regardless of the
    trend, is difficult without having clearly appropriate benchmarks from
    organizations engaged in activities similar to the Yucca Mountain project.
    Such benchmarks would clarify, for example, whether a project's
    percentages of human performance errors compare favorably, regardless of
    whether the numbers are increasing. Similarly, the trend in the number and
    types of CRs during any period is not necessarily a sign of improvement or
    worsening conditions. Trends can be attributed to various factors,
    including increases in the number of audits or self-assessments, which can
    lead to more CRs being issued.

    o At the time of analysis, some trend data may not be sufficiently
    reliable or complete to ensure sound findings for management's attention.
    For example, although some actions were taken in December 2004 to ensure
    that cause and other codes were properly assigned, a BSC audit in June
    2005 again raised questions about the consistency of the coding. With
    respect to completeness, the fourth quarter report for 2005 noted that 28
    percent of the Level B CRs did not have a cause code at the time of the
    trend analysis, and one finding was presented even though two-thirds of
    the data was missing.

    Due, in part, to these obstacles and changes to how the analysis is done,
    trend reports have not consistently determined the significance of
    problems or performed well in tracking progress in resolving problems. For
    example, trend reports have questionably identified significant human
    performance problems and ineffectively tracked progress in resolving the
    problem because of no clearly appropriate or precise benchmark for
    performance, inconsistent focus on the problem, and unreliable data on
    cause codes.

    The February 2004 trend report identified a human performance problem
    based on Yucca Mountain project data showing the project's proportion of
    skill-based errors to all human performance errors was two times higher
    than benchmark data from the Institute of Nuclear Power Operations
    (INPO).11 The report used this comparison to suggest that the project
    needed to adopt successful commercial nuclear practices for addressing
    skill-based errors. However, the report cautioned that other comparisons
    with these INPO data may not be appropriate because of differences in the
    nature, complexity, and scope of work performed, but did not explain why
    the report's comparison of INPO data for skill-based errors to the Yucca
    Mountain project should be an exception to this caution. The May 2004
    trend report repeated this comparison to INPO, finding skill-based errors
    three times higher than the benchmark data. However, this INPO benchmark
    has not been used in subsequent reports.

    The November 2004 trend report redefined the problem as the predominance
    of human performance errors in general, rather than the skill-based
    component of these errors-but later reports reinterpreted this
    predominance as not a problem. The problem with skill-based errors was
    unclear in the November 2004 report because these errors were showing a
    decreasing trend, a finding that was attributed as likely the result of
    unreliable assignment of cause codes. Instead, the report cited an adverse
    trend based on the fact that the human performance cause category
    accounted for over half of the total number of causes for condition
    reports prepared during the quarter. Under the project's trend analysis
    guidelines, this large predominance of human performance causes-in
    contrast to management, communication or procedure, and other cause
    categories-was designated an adverse trend. Nevertheless, by February
    2005, trend reports began interpreting this predominance as generally
    appropriate, given the type of work done by the project. That is, the
    project's work involves mainly human efforts and little equipment, while
    work at nuclear power plants involves more opportunities for errors caused
    by equipment. In our view, this interpretation that a predominance of
    human performance errors would be expected implies an imprecise benchmark
    for appropriate performance.

    Although trend reports continued to draw conclusions about human
    performance problems, the February 2005 report indicated that any
    conclusions were hard to justify because of data reliability problems with
    cause coding. For example, the majority of problems attributed to human
    performance causes are minor, or Level C, problems that receive less
    rigorous cause analysis, such as not completing a form. This less rigorous
    analysis tends to reveal only individual human errors-that is, human
    performance problems-whereas more rigorous analysis tends to reveal less
    immediately obvious problems with management and procedures.

    Trend reports have also inconsistently tracked progress in resolving the
    problem associated with the "flow-down" of requirements into the project's
    procedures-that is, with ensuring that program, regulatory, and statutory
    requirements are identified, allocated, and assigned to the project
    organizations that are responsible for applicable activities. Such
    requirements management problems can result in inadequate control over
    design inputs and, possibly, inputs to scientific models. Progress with
    this problem was less clear because of inconsistent methods of
    categorizing requirements management problems over time. Initially, based
    on reviews of annual trends in condition reports, the September 2004 and
    November 2004 trend reports observed a systemic and continuing problem in
    the flow-down of requirements from BSC's Project Requirements Document and
    identified this as an adverse trend. In subsequent reports, the
    requirements flow-down problem was variously treated as an aspect of
    requirements management or records management, or as a latent management
    weakness or weak change management. When treated as an aspect of these
    broader problems, the significance of the original flow-down problem and
    any progress in resolving it became diluted and less clear. The primary
    focus eventually became requirements management, which the February 2005
    trend report designated as a potential trend, whereas the flow-down
    problem had earlier been designated an adverse trend. Consequently, as a
    result of this change, the flow-down of requirements got less direct
    attention and analysis-for example, receiving only a footnote in the
    August 2005 trend report stating that the April 2004 condition report
    issued to address the adverse trend was still overseeing implementation of
    corrective actions.

    In addition, because trend reports examine only condition reports issued
    to BSC, they do not always assess the projectwide significance of problems
    such as requirements management.12 When analyzing one category of issues
    associated with requirements management, the November 2005 report stated
    that BSC and DOE shared the process problems, which cannot be adequately
    addressed by just one of the organizations. However, for a second category
    of these issues, the report did not analyze most of the condition reports
    because 6 of the 10 relevant reports were assigned to DOE. For a third
    category of issues, no analysis or recommendation was provided because all
    of the reports were assigned to DOE and therefore did not fall within the
    scope of the trend report.

    DOE Has Not Adequately Tracked Problems Being Addressed by Ongoing
    Management Actions

    The tracking of problems for which corrective actions are already being
    taken appeared at times to be overly influenced by judgments, rather than
    the problems' significance, about whether additional management action is
    warranted. As a result, problems might be rated as less significant, or
    not tracked further.

    The situation of assigning a lower rating to a problem's significance was
    apparently caused by the fact that ratings were simultaneously an
    assessment of a problem's significance and of the need for management
    action. In its current formulation, DOE's rating categories cannot
    accurately represent both the assessment of a problem's significance and a
    judgment that additional actions are not needed because the designated
    rating category will distort one or the other. For instance, the November
    2005 trend report analyzed the four categories of requirements management
    issues and designated one category that included problems with
    requirements flow-down as a "monitoring trend"-defined as a small
    perturbation in numbers that does not warrant action but needs to be
    monitored closely. Describing this trend as a small perturbation, or a
    disturbance in numbers, did not accurately describe the report's
    simultaneous recognition that significant process problems spanned both
    BSC and DOE and the fact that the numbers and types of problems were
    consistently identified over the previous three quarters. A more
    understandable explanation for the low rating is that designating the
    problem at any higher level of significance would have triggered
    guidelines involving the issuance of a condition report, which, according
    to the judgment expressed in the report, was not needed. Specifically, the
    report indicated that existing condition reports have already identified
    and were evaluating and resolving the problem, thereby eliminating the
    need to issue a new condition report.

    By rating the problem at the lowest level of significance and not calling
    for additional actions, the trend report did not sufficiently draw
    management's attention to the problem. The trend report's assessment did
    not convey that other serious problems might have been raised by the
    additional condition reports. At about the same time that the trend report
    judged that no new condition reports were necessary, an Employee Concerns
    Program's investigation of requirements management resulted in 14 new
    condition reports-3 at the highest level of significance and 8 at the
    second-highest

    level of significance.13 For example, the Employee Concerns Program's
    investigation resulted in condition reports calling for an analysis of the
    collective significance of the numerous existing condition reports and an
    assessment of whether the quality assurance requirement for complete and
    prompt remedial action had been met.14 As a result of the investigation
    and a concurrent DOE root cause analysis, during the December 2005
    Quarterly Management Meeting with NRC, DOE stated that strong actions were
    required to address the problems with its requirements management system
    and any resulting uncertainty about the adequacy of its design products.15

    Trend reports identified significant problems in the February 2005 report
    but did not continue to track the problems after a separate analysis
    identified ongoing improvement actions. According to the trend report,
    Level B condition reports collectively indicated organizational weaknesses
    associated with change management involving cross-departmental interfaces.
    The trend report recommended that management focus on these problems, and
    cited a condition report that would further investigate them. The cause
    analysis for that condition report and a related condition report found
    that the problems were well-known, in part through a BSC review, and
    related to a variety of ongoing BSC improvement actions. Since this was a
    broad category of problems with many initiatives under way, the cause
    analysis recommended no new actions other than for management to remain
    aware of the problems. However, the trend reports that followed provided
    no further analyses to focus management's awareness on these problems or
    to assess progress in resolving them.

    DOE's `New Path Forward' for Preparing to Submit Its License Application
    Faces Substantial Quality Assurance and Other Challenges

    In October 2005, DOE announced an aggressive series of proposed changes to
    the design, organization, and management of the Yucca Mountain project,
    but this effort-known as the "new path forward"-will face substantial
    challenges. Some key challenges facing DOE are (1) determining the extent
    of problems and restoring confidence in the documents supporting the
    license application after the discovery of e-mails raising the potential
    of falsified records, (2) settling design issues and associated problems
    with requirements management, and (3) replacing key personnel and managing
    the transition of new managers and other organizational challenges. The
    current Acting Director of the Office of Civilian Radioactive Waste
    Management (OCRWM) stated that DOE will not announce a schedule for
    submitting a license application until DOE addresses these important
    quality assurance and other challenges. Since DOE is still formulating its
    plans, it is too early to determine whether the new path will resolve
    these challenges.

    Determining the Extent of Problems with Relevant Documents Will Delay
    DOE's Submission of the License Application

    In March 2005, after announcing the discovery of USGS e-mails suggesting
    the possible violation of quality assurance requirements, including the
    falsification of records, DOE has taken steps to address lingering
    concerns about the adequacy of the scientific work related to the flow of
    water into the repository and whether similar quality assurance problems
    are evident in other e-mails relevant to the licensing application.
    Specifically, DOE is (1) conducting an extensive review of approximately
    14 million e-mails to determine whether these e-mails raise additional
    quality assurance concerns and whether they might be relevant to the
    licensing process, and (2) reworking the technical documents created by
    USGS personnel to ensure that the science underlying the conclusions on
    water infiltration are correct and supportable in the license application.
    The Acting Director of OCRWM has stated that DOE will not submit a license
    application until these efforts are complete. Consequently, given the
    early planning stage of these efforts, it is unknown how long this will
    delay the submission of a license application.

    As part of the licensing process, DOE is required to publicly disclose all
    documents relevant to the licensing application, including e-mails, by
    posting them on DOE's public Web site, which is accessible through the
    NRC-sponsored, Internet-based Licensing Support Network (LSN). To satisfy
    schedule requirements, DOE must certify that relevant documents have been
    posted to the network and made available for public review 6 months before
    the submission of the license application. In preparation for submitting
    the license application by December 2004, in June of that year, DOE
    submitted almost 700,000 e-mails to the LSN that had been reviewed by
    their original authors and determined to be relevant to the licensing
    process. They were part of a group of approximately 6 million archived
    e-mails authored by individuals still associated with the project.
    However, in August 2004, NRC's Atomic Safety and Licensing Board ruled
    that DOE had not met its regulatory obligation to make all relevant
    documentary material available. Specifically, DOE had not reviewed a group
    of approximately 4 million archived e-mails authored by individuals no
    longer affiliated with the project to determine whether the e-mails were
    relevant to the licensing process. As part of its effort to address the
    board's ruling, BSC began a review of e-mails authored by employees who
    were not currently working at the project. During this review, the
    contractor discovered and brought forward e-mails between USGS scientists
    working on water infiltration models that raised questions of the
    potential falsification of technical information in order to sidestep
    quality assurance requirements.

    Following the discovery of the e-mails, DOE conducted a search to
    determine if there were similar e-mails in the approximately 1 million
    e-mails previously determined relevant for licensing. However, the DOE
    Inspector General reported in November 200516 that there was no evidence
    that the project requirements for identifying and addressing conditions
    adverse to quality, such as those contained in the USGS e-mails, were
    considered during the initial review of e-mails. Further, among the
    approximately 10 million e-mails that had already been reviewed for the
    licensing process, they found additional e-mails that identified possible
    conditions adverse to quality that had not been identified by project
    personnel as requiring further review. The DOE Inspector General
    recommended, among other things, that DOE (1) expand the review of
    archived e-mails to include both those deemed relevant and those deemed
    not relevant to the licensing process, and ensure that conditions adverse
    to quality are appropriately identified, investigated, reported, and
    resolved; and (2) ensure that current and future e-mails are reviewed for
    possible conditions adverse to quality and that such conditions are
    appropriately addressed under the Corrective Action Program (CAP) system.
    DOE accepted the Inspector General's recommendations. Specifically, DOE
    agreed to develop a corrective action plan to expand the review of
    archived e-mails to ensure that conditions adverse to quality are
    appropriately identified and processed under the CAP system. In addition
    to this review, the DOE Inspector General opened a criminal investigation
    into the USGS e-mails in March 2005. As of December 2005, the
    investigation was still in progress.

    According to NRC on-site representatives, completing these e-mail reviews
    will be challenging because DOE now has to screen millions of e-mails to
    ensure that records were not falsified. Further, many of these e-mails
    were written by employees who no longer work at the project or may be
    deceased, making it difficult to learn their true meaning and context.
    Moreover, if additional e-mails are found that raise quality assurance
    concerns, DOE may have to initiate further review, inspections, or rework
    to address the newfound problems. NRC officials stated that it takes the
    issue of potentially falsified documents by USGS employees very seriously,
    wants a full understanding of the situation regarding the USGS e-mails,
    and will conduct follow-up in this area. Because NRC wants DOE to submit a
    high-quality license application, it has encouraged DOE to take the time
    and actions necessary to fully and adequately resolve these and other
    quality assurance issues.

    Immediately following the discovery of the USGS e-mails, DOE undertook a
    scientific investigation into the technical documents created by USGS
    personnel. In October 2005, DOE began developing an action plan for
    reviewing, validating, augmenting, and replacing USGS work products that
    had come under scrutiny. Although the plan is not yet complete, the Acting
    Director told us that the license application would not be submitted until
    the USGS work is replaced and there is confidence that all requirements
    have been met. In an effort to ensure that the scientific work underlying
    water infiltration modeling is accurate, DOE is working to corroborate the
    original work by engaging multiple agencies and organizations to rework
    the models. For example, DOE has (1) had its lead project contractor work
    with the Idaho National Laboratories to extensively review the software
    and data used in the original science work, (2) engaged Sandia National
    Laboratories to rework the model and calculations using different software
    than was used originally, and (3) also asked USGS to rework the models.
    Consequently, when this additional rework is completed, DOE will have four
    sets of analysis (including the original scientific work) with which they
    can evaluate, compare, and corroborate results. DOE will then pick one set
    of scientific analysis for inclusion in the license application, and work
    to explain and defend its choice.

    Ongoing Design and Requirements Management Issues Could Delay DOE's
    Submission of the License Application

    In October 2005, DOE announced significant changes to the design of the
    Yucca Mountain repository to simplify the project and improve its safety
    and operation. However, these changes will also require additional design
    and engineering work that will add uncertainty about the timing of the
    submission of a license application. DOE had been considering a design
    where radioactive waste would be shipped to the Yucca Mountain site,
    removed from its shipping container, placed and sealed in a special
    disposal container, and finally moved into the underground repository. As
    a result, DOE contemplated handling the waste up to four separate times.
    In late 2003, DOE engineers began identifying potential safety problems
    with this approach. First, possible fissures or holes in the cladding
    surrounding the spent nuclear fuel accidentally caused during the handling
    of the waste could cause air to mix with the fuel and oxidize.
    Consequently, this radioactive oxidized material could then leak and be
    dispersed into the air. Second, DOE engineers determined that the original
    facility design would not be able to adequately control the levels of
    radioactivity in the buildings where the waste would be repackaged before
    being moved in the repository. To address these problems, DOE researched a
    series of options, including only accepting radioactive waste that had
    already decayed to the point where oxidization would not be problematic,
    and testing the waste shipments for oxidization and treating them at
    another site before they arrived at the repository. In addition, DOE also
    considered changing the design by filling the processing buildings with
    inert gas to prevent oxidization and revising the electrical and
    ventilation systems. According to a DOE official, these options were
    impractical or added complexity to the design.

    However, in October 2005, DOE proposed a new design that relies on uniform
    canisters that would be filled and sealed before being shipped,
    eliminating the need for direct handling of the waste prior to being
    placed in the repository. As a result, DOE will not have to construct
    several extremely large buildings costing millions of dollars for handling
    radioactive waste. DOE believes this change will improve the safety,
    operation, and long-term performance of the repository. However, this
    change will also pose a challenge to the project because of the widespread
    implications and the unknown time and effort required to implement it. For
    example, to implement the new design, DOE will need to, among other
    things,

    o get approval from the Energy Systems Acquisition Advisory Board17 for a
    new project plan, which, among other things, includes details on the
    conceptual design, cost estimates, risk management efforts, and
    acquisition strategies;

    o plan, design, and produce standardized canisters for the transportation
    of waste;

    o coordinate this new approach with commercial nuclear power plants, NRC,
    and government organizations that plan on shipping waste to the project;
    and

    o revise procurement and contracting plans to support the new design.

    Finally, DOE will need to perform the detailed design and engineering work
    required to implement the new design, and create new technical documents
    to support the license application. However, before it can present its new
    plans and perform this design and engineering work, DOE officials have
    stated that it will need to resolve long-standing quality assurance
    problems involving requirements management. Requirements management is the
    process that ensures the broad plans and regulatory requirements affecting
    the project are tracked and incorporated into specific engineering
    details. According to DOE's root cause analyses, low-level documents were
    appropriately updated and revised to reflect high-level design changes
    through fiscal year 1995. However, from 1995 through 2002, many of these
    design documents were not adequately maintained and updated to reflect
    current designs and requirements. Further, a document that is a major
    component of the project's requirements management process was revised in
    July 2002, but has never been finalized or approved. Instead, the project
    envisioned a transition to a new requirements management system after the
    planned submission of the license application in December 2004. However,
    for various reasons, the license application was not submitted at that
    time, and the transition to a new requirements management system was never
    implemented. As a result, the document refers to the out-of-date NRC
    regulations contained in 10 CFR part 60, and not the regulations in 10 CFR
    part 63 that were finalized in October 2002.

    The scope and cause of requirements management problems have been
    identified in multiple DOE and NRC reviews.

    o Multiple condition reports issued in 2004 and 2005 have identified
    problems with requirements management. Due to these condition reports and
    NRC concerns that repetitive deficiencies and the failure to implement
    timely corrective actions could have direct implications on the quality of
    the planned license application, NRC performed a review of Corrective
    Action Program documents related to the requirements management program in
    the late summer of 2005. NRC determined that these reports identified
    approximately 35 deficiencies related to requirements management. Because
    the requirements management documents are not current and the new
    requirements management system has not been implemented, NRC concluded
    that there does not appear to be a requirements management mechanism in
    place. Further, based on the number of reports and other issues identified
    by DOE audits, NRC concluded that the project's Corrective Action Program
    was not effective in, among other things, eliminating the repeated
    identification of deficiencies relating to requirements management or
    initiating the actions to identify and appropriately address the root
    cause of these problems.

    o In September 2005, DOE began reviewing the root causes associated with
    CR-6278, a condition report identifying problems with requirements
    management. As part of the review, DOE personnel analyzed 135 condition
    reports and other events and allegations. Among other things, this review
    found that DOE expectations for requirements management were diluted and
    eventually neglected, that DOE reduced funding for requirements management
    due to reductions in its annual budget, and that these and other events
    caused the requirements management process to become "completely
    dysfunctional" from July 2002 to the time of the review in the fall of
    2005. The analysis identified the root causes of these conditions as DOE's
    failure to fund, maintain, and rigidly apply a requirements management
    system.

    o In November 2005, a team of DOE personnel concluded an investigation
    into an employee's concerns regarding requirements management. The team
    substantiated all of the concerns they investigated and found instances of
    failures and breakdowns in the requirements management process. For
    example, among other things, the team found that no procedure was
    developed to describe how requirements management was to occur; some
    existing requirements management procedures were not implemented; and
    project management was aware of these conditions but corrective actions
    were deferred because the planned requirements management system was
    expected to address the problem. As a contributing factor, the team also
    observed that the project's lead contractor had not implemented a
    "traditional systems engineering approach" as it did not have, among other
    things, typical engineering management plans or a separate systems
    engineering organization responsible for requirements management. As a
    result of the investigation, the team initiated 14 condition reports, 13
    of which identified quality-related problems.

    To address these problems, on December 19, 2005, DOE issued a stop-work
    order on design and engineering for the surface facility and certain other
    technical work. DOE stated that the root cause analysis for CR 6278 and
    the investigation into employee concerns revealed that the project has not
    maintained or properly implemented its requirements management system,
    resulting in inadequacies in the design control process. This stop-work
    order will be in effect until, among other things, the project's lead
    contractor improves the requirements management system; validates that
    processes exist and are being followed; and requirements are appropriately
    traced to implementing mechanisms and products. Further, DOE will
    establish a team to take other actions necessary to prevent inadequacies
    in requirements management and other management systems from recurring.

    An example of the potential risks of a breakdown with requirements
    management was noted during a BSC audit on the design process in March
    2005. NRC on-site representatives observing this audit reported that the
    audit team noted problems with inconsistencies between the design
    documents of the planned fuel-handing facility that would be receiving,
    preparing, and packaging the waste before it is placed in the repository.
    The original set of requirements specified that no water from a fire
    protection system was to be used in the fuel-handling areas of the
    facility because under certain scenarios, water used for fire suppression
    could facilitate an accidental nuclear reaction, a condition known as
    criticality. Later, as the project began to review the design of the
    fuel-handling facility, the design was changed to allow the use of water
    sprinklers in the fuel-handling areas of the facility to suppress possible
    fires. NRC noted that personnel working on the design knew of the
    inconsistencies between older and newer design documents, but no formal
    tracking mechanism had been provided to ensure that those issues were
    rectified. According to an NRC on-site representative in December 2005,
    this was an example of a concern with requirements management, and that
    repetitive and uncorrected issues associated with the requirements
    management process could have direct implications on the quality of the
    license application.

    While the project may be able to resolve these inconsistencies through an
    informal process, the lack of a formal design control and requirements
    management process increases the risk that not all such problems will be
    addressed. These requirements management problems are potentially
    significant because if the high-level engineering needs of the project are
    not accurately or completely reflected in the detailed design, then the
    quality of the license application may be compromised and cause delays in
    the license application review process. For example, according to a 1989
    speech prepared by NRC's Office of General Counsel stressing the
    importance of quality assurance, a West Coast nuclear power plant
    experienced similar quality assurance problems with requirements
    management. After a license was issued by NRC, power plant personnel
    discovered that the wrong diagrams were used to develop design
    requirements. As a result of this and other quality assurance weaknesses
    identified by NRC, the license was suspended and the power plant was
    required to initiate an independent program to verify the correctness of
    the design. Further, NRC reopened hearings on the issue of the adequacy of
    the power plant's quality assurance program related to the plant's design.

    DOE Faces Challenges in Managing the Transition, Complexity, and
    Continuity of Its `New Path Forward'

    In October 2005, DOE announced a "new path forward" that would create a
    new project schedule and financial plan to address the completion of
    scientific and engineering work in support of a license application.
    However, DOE faces challenges to successfully implementing the new path,
    in terms of managing the transition, program and organizational
    complexities, and the continuity of management. According to DOE managers
    involved with planning the new path forward, the organizational transition
    could take several months to complete. It is too early to determine
    whether DOE's new effort will resolve quality assurance issues and move
    the project forward to the submission of a license application.

    Accountability for quality and results, which was identified as a
    significant transition issue in the Initiatives, will likely pose a
    challenge for managing the transition to the new path forward. The
    Initiatives sought to clarify roles and responsibilities within and
    between DOE and contractor organizations to ensure clear accountability
    for results and quality during the transition from OCRWM's organization,
    processes, procedures, and skills supporting scientific studies to those
    supporting the activities necessary to license a repository. As the
    project realigns organizations, processes, procedures, and skills to
    support the new path forward, it will also be faced with the challenge of
    ensuring that accountability is not undermined during the transition. For
    instance, according one DOE manager, transitioning project work to a lead
    laboratory under a direct contract with DOE could pose a significant
    challenge for quality assurance because the laboratories are currently
    working under BSC quality assurance procedures and will now have to
    develop their own procedures.

    Implicitly recognizing the importance of accountability issues, elements
    of the new path forward seek to address issues that can negatively affect
    quality assurance and project management in general. For instance, the new
    path includes plans for developing and transmitting requirements to USGS
    for the certification of scientific work. In addition, a senior project
    official told us that the lead laboratory would provide a single point of
    accountability that will enhance the quality of the science work. The
    Acting Director indicated that OCRWM's management structure may have to be
    reorganized to have a single manager clearly accountable for each of the
    new path's major tasks in science, engineering, and licensing. Moreover,
    the project is developing new performance indicators to allow the project
    to assess important activities under the new path forward. Outside of the
    new path, as the result of a September 2005 DOE Inspector General report18
    on accountability problems with managing contract incentives, OCRWM agreed
    to develop a comprehensive corrective action plan to provide clearer and
    more objective performance standards in the BSC contract.

    Program complexity and other project characteristics are also likely to
    pose challenges to managing quality assurance. Based on its experience
    with licensing and regulating nuclear power plants, NRC observed in the
    mid-1980s that the Yucca Mountain project's characteristics, such as a
    large and complicated program, increased the likelihood of major
    quality-related problems. Although the new path is intended to simplify
    design, licensing, and construction, the project remains a complicated
    program that seeks to both restore confidence in its scientific studies
    and pursue new design and engineering activities. As a result, the project
    has to manage quality assurance issues simultaneously in both areas.
    Moreover, the project involves a complicated organizational structure. The
    project will continue contracting work with BSC, USGS, and the Sandia
    National Laboratory, which involves working with organizations in various
    locations. In our 1988 report, we noted that the geographic distance
    between the various organizations may hamper OCRWM's quality assurance
    communication and oversight objectives.

    The project also faces management challenges related to ensuring
    management continuity at the project, since DOE has experienced turnover
    in 9 of 17 key management positions since 2001. To ensure the right
    managers move the project forward to licensing, the project has a
    recruitment effort for replacing key departing managers. In the past year,
    the project has lost key managers through the departures of the director
    of Project Management and Engineering, the director of the License
    Application and Strategy, the director of Quality Assurance, and the
    contractor's general manager. According to NRC on-site representatives in
    August and October 2005, management turnover is a concern for NRC because
    it would like to see continuity of qualified managers rather than a series
    of acting managers. Recruiting replacement managers can impact project
    continuity, and newly acting managers may not take full rein of project
    tasks. However, the Acting Director told us that the recruitment process
    is an opportunity to improve project managers and staff, but recruiting
    the right people is challenging for various reasons-for example,
    government salaries are less than those in industry, and employment
    clauses restrict subsequent employment in related industries.

    Finally, since new directors sometimes give new direction to the project,
    a critical issue for sustaining the current new path forward is continuity
    with OCRWM's director. This position was occupied by three individuals
    between late 1999 and early 2005. The last OCRWM director assumed the
    position in April 2002, started the Management Improvement Initiatives in
    2002, and left the position in February 2005. The current Acting Director
    began functioning in his position in the summer of 2005, and initiated the
    new path forward in October 2005. DOE is currently awaiting congressional
    confirmation of a nominee to take the director position. However, the
    Acting Director told us he expects that the new path forward will be
    sustained because it has been endorsed by the Secretary of Energy.

    Conclusions

    DOE's Yucca Mountain project has been wrestling with quality assurance
    problems for a long time. Now, after more than 20 years of project work,
    DOE is again faced with substantial quality assurance and other challenges
    to submit a fully defensible license application to NRC. Unless these
    challenges are effectively addressed, further delays on the project are
    likely. Furthermore, even as DOE faces new quality assurance challenges,
    it cannot be certain it has resolved past problems, largely because the
    department has not been well served by management tools-specifically, its
    performance indicators and trend evaluation reports-that have not
    effectively identified and tracked progress on significant and recurring
    problems. First, the management tools have provided limited coverage of
    the areas of concern identified in the Management Improvement Initiatives
    and thus have not enabled DOE managers to effectively monitor progress in
    these important areas. Second, the tools have often not reflected the full
    extent or significance of problems because their scope has been limited
    and not based on projectwide analysis. Third, the trend evaluation reports
    have, at times, not accurately characterized problems because reliable and
    complete data and appropriate performance benchmarks were not available at
    the time of analysis. Fourth, frequent changes in performance indicators
    and the way analysis is done have made it difficult to accurately identify
    trends over time. Fifth, the tools' rating categories have sometimes been
    misleading as to the significance of problems because the ratings tend to
    be skewed by the fact that corrective actions were already being taken,
    without considering their effectiveness or considering the significance of
    the problem on its own terms. These shortcomings with the tools limit
    project managers' ability to direct and oversee such a large and complex
    undertaking as constructing an underground repository for nuclear wastes.
    Further complicating DOE's ability to manage the project are the vacancies
    in key managerial positions for the quality assurance program and
    elsewhere on the project. The tools become even more important for new
    managers who need to quickly understand project management issues.

    Recommendations for Executive Action

    To improve the effectiveness of DOE's efforts to monitor performance in
    key areas at the Yucca Mountain project, including quality assurance, we
    recommend that the Secretary of Energy direct the Director, Office of
    Civilian Radioactive Waste Management, to take the following five actions
    to strengthen the project's management tools:

    o Reassess the coverage that the management tools provide for the areas of
    concern identified in the Management Improvement Initiatives and ensure
    that performance in these important areas is effectively monitored,
    especially in light of the more recent condition reports and associated
    cause analyses, trend reports, and other reviews indicating continuing
    problems.

    o Base future management tools, such as the trend evaluation reports, on
    projectwide analysis of problems, unless there are compelling reasons for
    a lesser scope.

    o Establish quality guidelines for trend evaluation reports to ensure
    sound analysis when reporting problems for management's attention. Such
    guidelines should address, among other things, having reliable and
    complete data and appropriate benchmarks.

    o To the extent practicable, make analyses and indicators of performance
    consistent over time so that trends or progress can be accurately
    identified and, where changes to analyses or indicators are made for
    compelling reasons, provide a clear history of the changes and their
    impact on measuring progress.

    o Focus the management tools' rating categories on the significance of the
    monitored condition, not on a judgment of the need for management action.

    Agency Comments

    We provided DOE and NRC with draft copies of this report for their review
    and comment. In oral comments, DOE agreed with our recommendations and
    provided technical and editorial comments that we have incorporated in the
    report, as appropriate. We also incorporated, as appropriate, NRC's oral
    editorial comments, which primarily served to clarify its role.

    As agreed with your office, unless you publicly announce the contents of
    this report earlier, we plan no further distribution until 30 days from
    the report date. At that time, we will send copies to interested
    congressional committees and Members of Congress, the Secretary of Energy,
    and the Chairman of the Nuclear Regulatory Commission. We will also make
    copies available to others upon request. In addition, the report will be
    available at no charge on the GAO Web site at www.gao.gov .

    If you or your staff have any questions about this report, please contact
    me at (202) 512-3841 or wellsj@gao.gov . Contact points for our Offices of
    Congressional Relations and Public Affairs may be found on the last page
    of this report. GAO staff who made major contributions to this report are
    listed in appendix III.

    Sincerely yours,

    Jim Wells Director, Natural Resources   and Environment

    Objectives, Scope, and Methodology Appendix I

    The objectives of this review were to determine (1) the history of the
    Yucca Mountain project's quality assurance problems since the project's
    start in the 1980s, (2) the Department of Energy's (DOE) tracking of
    quality problems and progress implementing quality assurance requirements
    since our April 2004 report, and (3) challenges that DOE faces as it
    continues to address quality assurance issues within the project. In
    addition, we were asked to provide information about implementation of the
    project's Employee Concerns Program and the types of concerns raised in
    recent years through the program.

    To determine the history of the project's quality assurance problems, we
    reviewed our prior reports and those of DOE's Office of the Inspector
    General concerning the Yucca Mountain project. We also reviewed internal
    DOE evaluations and audit reports written about the quality assurance
    program and Nuclear Regulatory Commission (NRC) reports and NRC-prepared
    summaries of NRC and DOE quarterly management meetings, technical exchange
    meetings, and quality assurance meetings dating to early 2004. In
    addition, we reviewed letters and communications between DOE and NRC
    regarding quality assurance from the NRC Web archives from the late 1980s.
    Furthermore, we reviewed plans for the Regulatory Integration Team (RIT)
    and subsequent correspondence between Bechtel/SAIC Company, LLC (BSC),
    DOE's management contractor for the Yucca Mountain project, and DOE.
    Moreover, we discussed quality assurance issues with officials of DOE's
    Office of Civilian Radioactive Waste Management (OCRWM), including the
    Acting Director and Deputy Director, at DOE headquarters in Washington,
    D.C., and at its field office in Las Vegas. In addition, we interviewed
    representatives of Navarro Quality Services, a DOE subcontractor, as well
    as BSC, and NRC officials in the agency's field office in Las Vegas,
    Nevada, and at its headquarters in Rockville, Maryland.

    To determine DOE's tracking of quality problems and progress implementing
    quality assurance requirements since our April 2004 report, we interviewed
    OCRWM, BSC, and NRC officials about the status of these efforts since the
    issuance of our prior report. We also reviewed DOE's Management
    Improvement Initiatives (2002), DOE's Management Improvement Initiatives
    Transition Approach (2003), and our 2004 report to understand the history
    of the improvement efforts. To understand DOE's management tools to
    monitor problems and progress, we reviewed the available performance
    indicators panels from April 2004 through August 2005, when it was last
    produced; the documentation on the individual indicators applied to August
    2005 data; and the quarterly trend reports from the fourth quarter of
    fiscal year 2003 through the fourth quarter of fiscal year 2005. We also
    reviewed information from condition reports and examined documentation on
    DOE's Quality Assurance Requirements and Description (issued in August
    2004), BSC's Trend Evaluation and Reporting, and DOE's Procedure:
    Condition Reporting and Resolution (issued in November 2005).

    To determine challenges that DOE faces as it continues to address quality
    assurance issues within the project, we reviewed information from
    condition reports, NRC on-site representative reports, DOE Inspector
    General reports, and an OCRWM Office of Concerns Program's investigative
    report on past quality assurance problems and DOE's efforts to address
    them. We obtained information on turnover in key management positions at
    DOE and BSC since 2000. In addition, we discussed with DOE and NRC
    officials DOE's difficulties in addressing recurring quality assurance
    problems and the quality assurance implications of the Yucca Mountain
    project moving from the site characterization phase to design and
    licensing. Also, to better understand issues and challenges, we attended
    quarterly meetings held between DOE and NRC in Rockville in September 2005
    and Las Vegas in December 2005.

    To identify recent employee concerns related to quality assurance, such as
    falsification of records and a safety-conscious work environment, as well
    as to identify the actions taken to address those concerns, we reviewed
    all concerns received by the OCRWM and BSC Employee Concerns Programs from
    January through November 2005. For the OCRWM program, we reviewed all
    employee concerns files to identify concerns related to quality assurance.
    For the BSC program, we first read summary descriptions of each concerns
    file, and reviewed the concerns files for only those we identified as
    related to quality assurance. We then conducted a content analysis of all
    concerns files that we reviewed. Next, our three team members reached
    consensus about the correct classification of a concern as a quality
    assurance problem, such as potential falsification of records. Finally,
    through a second review of concerns files, we verified our recorded
    information for those concerns that seemed to be important illustrations
    of problems. In addition, we also spot-checked a sample of OCRWM and BSC
    concerns received in 2005 to verify the accuracy of their placement in
    various concerns categories. We found that the concerns were generally
    categorized accurately.

    We performed our work from July 2005 through January 2006 in accordance
    with generally accepted government auditing standards.

    Yucca Mountain Project Employee Concerns Programs Appendix II

    NRC expects licensees to establish a safety conscious work
    environment-that is, one in which (1) employees are encouraged to raise
    concerns either to their own management or to NRC without fear of
    retaliation and (2) employees' concerns are resolved in a timely and
    appropriate manner according to their importance. NRC encourages but does
    not require licensees to establish employee concerns programs to help
    achieve such a work environment, and both DOE and BSC have established
    such programs.1 DOE's Employee Concerns Program is currently operated
    under the requirements of DOE Order 442.1A, but the department, in
    anticipation of becoming a licensee, is in the process of establishing the
    program to meet NRC expectations.

    DOE and contractor employees at the Yucca Mountain project may raise
    concerns about quality, safety, or other work environment issues-such as
    harassment, intimidation, retaliation, and discrimination-through various
    means. Employees are encouraged to resolve concerns at the lowest possible
    level in the organization, in the following order:

    o Use normal supervisory channels, such as by raising an issue to a
    manager for resolution.

    o Initiate a condition report through the Corrective Action Program-a
    process in which any employee can formally identify a problem on the
    project, such as with policies, procedures, or the work environment, and
    have the issue investigated and, if necessary, fixed through corrective
    actions.

    o Submit a concern via e-mail, telephone, or in person to one of the
    project's two Employee Concerns Programs-a BSC program for BSC employees
    and other subcontractors and another run by DOE for either DOE or BSC
    employees.

    o Contact NRC directly.

    The DOE and BSC concerns programs are intended to supplement rather than
    replace the resolution of problems through managers or the Corrective
    Action Program.

    DOE and BSC Employee Concerns Programs have each established a
    communication network to allow employees to register concerns. These
    networks include brochures and regular newsletters on the programs and
    numerous links to the program on the project's intranet, where employees
    can obtain concerns forms. Both the DOE and BSC concerns programs of the
    Yucca Mountain project have four main steps:

    1.Employees notify the concerns program staff about issues that they feel
    should be corrected, such as safety or health issues; harassment,
    intimidation, retaliation, or discrimination; concerns raised through the
    Corrective Action Program; and quality assurance problems.

    2.The concerns program staff document and handle the concern in accordance
    with the requirements of DOE Order 442.1A.

    3.The concerns program notifies the employees of the results of the
    investigation and notifies management of any deficiencies.

    4.Project management develops corrective actions for deficiencies, and the
    program validates that the concerns have been effectively addressed by the
    actions.

    Under DOE Order 442.1A, concerns may be addressed through an investigation
    by the concerns program staff, an independent investigation, a referral, a
    transfer, or a dismissal of the concern. Employees can request or waive
    confidentiality. If a concern is submitted anonymously, interpreting the
    main issues and problems is left up to the concerns program staff, and
    action on the concern may be limited if the submitted information does not
    clearly or sufficiently define the concern.

    The concerns program may conduct its own investigation of the concern.
    Alternatively, it may refer the concern to another project organization
    for investigation or resolution. After the results of the investigation or
    resolution are reported to the concerns program within a specified period,
    the concerns program accepts the results or requires additional actions.
    In other cases, concerns may be transferred to another organization with
    the appropriate subject matter responsibility or expertise, such as the
    Office of Human Relations, Office of General Counsel, or Office of the
    Inspector General.

    After investigating a concern, the concerns programs determines whether
    the concern is substantiated, partially substantiated, unsubstantiated, or
    indeterminate. If a concern is substantiated or partially substantiated,
    the investigation results are presented to the responsible senior
    managers. A concern is considered indeterminate when evidence is
    insufficient to substantiate a concern or allow for a conclusion to be
    drawn. Some concerns can be resolved through a noninvestigative
    resolution, a method to address concerns promptly when minimal effort is
    required for resolution. Some resolutions involve the development of
    management corrective action plans that are tracked until they are closed.
    In addition, for deficiencies that identify systemic problems, the
    concerns programs may file a condition report through the Corrective
    Action Program. Moreover, DOE and contractor employees are required to
    report certain conditions or alleged conditions to DOE's Office of the
    Inspector General under DOE Order 221.1, which covers waste, fraud, and
    abuse. The concerns program handles some employee concerns in this way.

    From January through November 2005, DOE's concerns program opened 139
    employee concerns for investigation, and the BSC concerns program opened
    112 concerns for investigation.2 DOE's concerns program places concerns
    into 14 categories, while the BSC program uses 20 categories.3 For both
    DOE and BSC, the category receiving by far the most concerns for calendar
    year 2005 was management: "management/mismanagement" for DOE and
    "management practices" for BSC. According to DOE, management concerns
    generally involved conditions related to management behavior, policy
    practice, budget allocation, or use of resources. According to the manager
    of BSC's program, about half of the concerns in the management practices
    category involve hiring and human relations issues and the other half
    involve organizational policies and other issues. The "quality" category
    accounts for a relatively small portion of total concerns-18 percent of
    concerns for the DOE program and 4 percent for the BSC program. Tables 3
    and 4 show the concerns received by the DOE and BSC programs for January
    through November 2005.

    Table 3: Employee Concerns Opened for Investigation under DOE's Employee
    Concerns Program by Category of Concern, January through November 2005

                          Concern category                    Percentage of total
                                                                         concerns
    Management/mismanagement                                                42
    Workplace violence                                                       0
    Harassment, intimidation, retaliation, and                               6
    discrimination
    Reprisal                                                                 0
    Chilling effect                                                          5
    Security                                                                 0
    Health                                                                   0
    Safety                                                                   4
    Environment                                                              0
    Fraud, waste and abuse                                                   4
    Human resources                                                         12
    Equal Employment Opportunity                                             2
    Quality                                                                 18
    Other                                                                    8
    Total                                                                  100

    Source: DOE.

    Note: Percentages may not add to 100 because of rounding.

    Table 4: Employee Concerns Opened for Investigation under BSC's Employee
    Concerns Program by Category of Concern, January through November 2005

                  Concern category                   Percentage of total concerns
    Management practices                                                    48
    Industrial                                                               1
    Health                                                                   4
    Fraud                                                                    3
    Fitness for duty                                                         1
    Ethics                                                                   5
    Cyber                                                                    0
    Access authorization                                                     0
    Environmental                                                            1
    Employee relations                                                       5
    Intimidation                                                             1
    Harassment                                                               0
    Discrimination                                                           4
    Chilling effect                                                          4
    Abuse                                                                    4
    Training                                                                 1
    Safety-conscious work environment                                        3
    Retaliation                                                              4
    Quality                                                                  4
    Other                                                                    6
    Total                                                                  100

    Source: BSC.

    Note: Percentages may not add to 100 because of rounding.

    The Employee Concerns Programs, which are designed to provide an
    alternative to raising issues through the Corrective Action Program and
    issuing condition reports, have been playing an active and sometimes key
    role in identifying and addressing quality assurance problems, as can be
    seen in the following examples:

    o As part of an effort to identify e-mails relevant to the licensing
    process and that therefore should be included in the Licensing Support
    Network, BSC employees in late 2004 discovered e-mails suggesting
    potential falsification of technical records. The e-mails were submitted
    to the Employee Concerns Program in March 2005 and were eventually
    reported to the DOE Inspector General for investigation. The quality
    assurance issues raised by the e-mails have resulted in a substantial
    effort by DOE to restore confidence in the quality of technical documents
    that will support its license application to construct the repository.

    o In mid-2005, the DOE concerns program referred to the project's senior
    management an employee's allegation that the project's schedule was taking
    priority over quality in the review of technical documents. In this
    instance, the Office of Concerns Program Manager negotiated with senior
    management to address the time and resource needs for ensuring quality
    assurance, rather than simply communicating to the organization that
    quality should take priority over the schedule.

    o As the result of an employee's concerns referred to DOE by NRC in mid-
    2005, the Employee Concerns Program initiated an extensive investigation
    of issues related to requirements management. That investigation
    substantiated the employee's concerns and led to the issuance of 14
    condition reports for problem resolution. Signifying the importance of
    this issue, DOE discussed problems with requirements management with NRC
    at their quarterly meeting in December 2005.

    The Employee Concerns Programs' role in identifying and addressing quality
    assurance and other issues is dependent upon employees' willingness to
    submit concerns, but the employees' willingness has sometimes been in
    doubt. A late 2004 DOE survey of project employees indicated, for example,
    that less than two-thirds of employees were confident that submitted
    concerns would be thoroughly investigated and appropriately resolved. DOE
    recognizes the need to improve employee trust and willingness to use the
    concerns program, and both the DOE and BSC program are engaged in outreach
    efforts. However, employees' willingness to submit concerns may be
    affected by factors outside the programs' control. According to a DOE
    manager, the project's recent and pending workforce reductions may account
    for a decreasing number of concerns submitted to the DOE program in late
    2005. Based on OCRWM Employee Concerns Program data, the program averaged
    about 13 concerns a month from January through November 2005. However, the
    number of monthly concerns dropped to 5 in October and 3 in November 2005.

    During our review of concerns opened for investigation from January 2004
    through November 2005, we did not identify any concerns alleging problems
    similar to the falsification of technical records suggested by the USGS
    e-mails. Although we found records of an early 2004 concern about an
    instance of inappropriate management of a technical document, this
    instance was resolved and did not appear to be an intentional or
    systematic effort to falsify records. The manager of the BSC program told
    us of a concern raised about another set of e-mails, but this concern was
    not about record falsification. The manager of the DOE program told us
    that she had not seen any reportable allegations of falsification of
    technical records since she took her position in July 2004.

    GAO Contact and Staff Acknowledgments Appendix III

    Jim Wells, (202) 512-3841 or wellsj@gao.gov

    In addition to the contact named above, Raymond Smith (Assistant
    Director), Casey Brown, John Delicath, James Espinoza, and Terry Hanford
    made key contributions to this report.

    (360611)

    GAO's Mission

    The Government Accountability Office, the audit, evaluation and
    investigative arm of Congress, exists to support Congress in meeting its
    constitutional responsibilities and to help improve the performance and
    accountability of the federal government for the American people. GAO
    examines the use of public funds; evaluates federal programs and policies;
    and provides analyses, recommendations, and other assistance to help
    Congress make informed oversight, policy, and funding decisions. GAO's
    commitment to good government is reflected in its core values of
    accountability, integrity, and reliability.

    Obtaining Copies of GAO Reports and Testimony

    The fastest and easiest way to obtain copies of GAO documents at no cost
    is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts newly
    released reports, testimony, and correspondence on its Web site. To have
    GAO e-mail you a list of newly posted products every afternoon, go to
    www.gao.gov and select "Subscribe to Updates."

    Order by Mail or Phone

    The first copy of each printed report is free. Additional copies are $2
    each. A check or money order should be made out to the Superintendent of
    Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
    copies mailed to a single address are discounted 25 percent. Orders should
    be sent to:

    U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
    D.C. 20548

    To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
    512-6061

    To Report Fraud, Waste, and Abuse in Federal Programs

    Contact:

    Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
    Automated answering system: (800) 424-5454 or (202) 512-7470

    Congressional Relations

    Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S.
    Government Accountability Office, 441 G Street NW, Room 7125 Washington,
    D.C. 20548

    Public Affairs

    Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 U.S.
    Government Accountability Office, 441 G Street NW, Room 7149 Washington,
    D.C. 20548

    www.gao.gov/cgi-bin/getrpt? GAO-06-313 .

    To view the full product, including the scope

    and methodology, click on the link above.

    For more information, contact Jim Wells at (202) 512-3841 or
    wellsj@gao.gov.

    Highlights of GAO-06-313 , a report to the Chairman, Subcommittee on the
    Federal Workforce and Agency Organization, Committee on Government Reform,
    House of Representatives

    March 2006

    YUCCA MOUNTAIN

    Quality Assurance at DOE's Planned Nuclear Waste Repository Needs
    Increased Management Attention

    The Department of Energy (DOE) is working to obtain a license from the
    Nuclear Regulatory Commission (NRC) to construct a nuclear waste
    repository at Yucca Mountain in Nevada. The project, which began in the
    1980s, has been beset by delays. In a 2004 report, GAO raised concerns
    that persistent quality assurance problems could further delay the
    project. Then, in 2005, DOE announced the discovery of employee e-mails
    suggesting quality assurance problems, including possible falsification of
    records. Quality assurance, which establishes requirements for work to be
    performed under controlled conditions that ensure quality, is critical to
    making sure the project meets standards for protecting public health and
    the environment.

    GAO was asked to examine (1) the history of the project's quality
    assurance problems, (2) DOE's tracking of these problems and efforts to
    address them since GAO's 2004 report, and (3) challenges facing DOE as it
    continues to address quality assurance issues within the project.

    What GAO Recommends

    GAO recommends five actions DOE can take to improve the project's
    management tools and identify and address quality assurance and other
    problems.

    In oral comments, DOE agreed with GAO's recommendations.

    DOE has had a long history of quality assurance problems at the Yucca
    Mountain project. In the 1980s and 1990s, DOE had problems assuring NRC
    that it had developed adequate plans and procedures related to quality
    assurance. More recently, as it prepares to submit a license application
    for the repository to NRC, DOE has been relying on costly and
    time-consuming rework to resolve lingering quality assurance problems
    uncovered during audits and after-the-fact evaluations.

    DOE announced, in 2004, that it was making a commitment to continuous
    quality assurance improvement and that its efforts would be tracked by
    performance indicators that would enable it to assess progress and direct
    management attention as needed. However, GAO found that the project's
    performance indicators and other key management tools were not effective
    for this purpose. For example, the management tools did not target
    existing areas of concern and did not track progress in addressing them.
    The tools also had weaknesses in detecting and highlighting significant
    problems for management attention.

    DOE continues to face quality assurance and other challenges. First, DOE
    is engaged in extensive efforts to restore confidence in scientific
    documents because of the quality assurance problems suggested in the
    discovered e-mails between project employees, and it has about 14 million
    more project e-mails to review. Second, DOE faces quality assurance
    challenges in resolving design control problems associated with its
    requirements management process-the process for ensuring that high-level
    plans and regulatory requirements are incorporated into specific
    engineering details. Problems with the process led to the December 2005
    suspension of certain project work. Third, DOE continues to be challenged
    to manage a complex program and organization. Significant personnel and
    project changes initiated in October 2005 create the potential for
    confusion over roles and responsibilities-a situation DOE found to
    contribute to quality assurance problems during an earlier transition.

    View of Yucca Mountain and the Exploratory Tunnel for the Repository
    *** End of document. ***