National Laboratories: Are Their R&D Activities Related to Commercial
Product Development (Letter Report, 11/25/94, GAO/PEMD-95-2).

In response to congressional interest in how the national laboratories
of the Energy Department (DOE) can best be focused to help solve the
problems faced by the nation during the 1990s, this report presents an
inventory of the human and capital resources housed in the national
laboratories that will provide baseline data for future GAO reports on
DOE laboratory policy issues. This report addresses congressional
interest in the current balance of the research efforts in 10
laboratories' research programs.  The 10 laboratories are as follows:
Argonne, Brookhaven, Idaho Engineering, Lawrence Livermore, Lawrence
Berkeley, Los Alamos, Oak Ridge, Pacific Northwest, Sandia, and the
Solar Energy Research Institute.  GAO also examines the extent to which
these national laboratories are now engaged in basic and applied
research or in research linked to commercial product development.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  PEMD-95-2
     TITLE:  National Laboratories: Are Their R&D Activities Related to 
             Commercial Product Development
      DATE:  11/25/94
   SUBJECT:  Technology transfer
             Laboratories
             Research and development
             Cooperative agreements
             Research and development facilities
             Energy research
             Information dissemination operations
             Technical assistance
             Research program management

             
**************************************************************************
* This file contains an ASCII representation of the text of a GAO        *
* report.  Delineations within the text indicating chapter titles,       *
* headings, and bullets are preserved.  Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are           *
* identified by double and single lines.  The numbers on the right end   *
* of these lines indicate the position of each of the subsections in the *
* document outline.  These numbers do NOT correspond with the page       *
* numbers of the printed product.                                        *
*                                                                        *
* No attempt has been made to display graphic images, although figure    *
* captions are reproduced. Tables are included, but may not resemble     *
* those in the printed version.                                          *
*                                                                        *
* A printed copy of this report may be obtained from the GAO Document    *
* Distribution Facility by calling (202) 512-6000, by faxing your        *
* request to (301) 258-4066, or by writing to P.O. Box 6015,             *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time.                                    *
**************************************************************************


Cover
================================================================ COVER


Report to Congressional Requesters

November 1994

NATIONAL LABORATORIES - ARE THEIR
R&D ACTIVITIES RELATED TO
COMMERCIAL PRODUCT DEVELOPMENT? 

GAO/PEMD-95-2

National Laboratories' R&D Activities


Abbreviations
=============================================================== ABBREV

  AMTEX - American Textile Partnership
  AT&T - American Telephone and Telegraph
  CRADA - Cooperative research and development agreement
  DOE - Department of Energy
  FTE - Full-time-equivalent
  OSTP - Office of Science and Technology Policy
  PNGV - Partnership for a New Generation of Vehicles
  R&D - Research and development

Letter
=============================================================== LETTER


B-256574

November 25, 1994

The Honorable Marilyn Lloyd
Chairman, Subcommittee on Energy

The Honorable Tim Valentine
Chairman, Subcommittee on Technology,
 Environment, and Aviation

The Honorable Tom Lewis
Ranking Minority Member, Subcommittee
 on Technology, Environment, and Aviation
Committee on Science, Space, and Technology
House of Representatives

You asked us to consider how the national laboratories of the U.S. 
Department of Energy (DOE) can best be focused to help solve the
problems our nation faces during the current decade.  As a beginning,
we have developed an inventory of the human and capital resources
housed in the national laboratories that will provide baseline data
for future reports on a number of DOE laboratory policy issues.  This
report addresses the Committee's interest in the current balance of
the research effort in the 10 laboratories' research programs.  It
examines the extent to which the national laboratories are engaged
now in basic and applied research or in research related to
commercial product development. 


   BACKGROUND
------------------------------------------------------------ Letter :1

The Department of Energy's multiprogram laboratories have had
missions that are national in scope since their inception during
World War II.  The original laboratories--Lawrence Berkeley (Calif.),
Los Alamos (N.  Mex.), and Oak Ridge (Tenn.)--were established as
government-owned, contractor-operated institutions to apply the
productive capability of private industry to the development of
atomic weapons.\1 The weapons-development mission continued during
the cold war, and six additional laboratories--Argonne (Ill.),
Brookhaven (N.Y.), Sandia (N.  Mex.  and Calif.), Idaho Engineering
(Idaho), Lawrence Livermore (Calif.) and Pacific Northwest
(Wash.)--were created between 1946 and 1965 to foster civilian
applications of nuclear technology.\2 A 10th laboratory, the Solar
Energy Research Institute, was designated a national laboratory in
1991 to expand federal energy research and development (R&D)
capability in alternative energy sources, and it was renamed the
National Renewable Energy Laboratory (Colo.).\3 As a group, the 10
laboratories are known as the national laboratories. 

As the laboratories' experience and research capability evolved,
mission emphases shifted among them.  Sandia, Los Alamos, and
Lawrence Livermore acquired primary responsibility for nuclear
weapons research and development and the largest share of the
laboratories' funds.  Responsibility for research in the
environmental and biological, energy, and national security areas was
distributed among all 10 laboratories to varying degrees.\4 However,
the Congress and DOE are reassessing this mission configuration. 

Since 1980, the Congress has had an active interest, expressed in a
series of laws, in seeing that more of the national laboratories'
outputs be put to commercial uses.\5

Changing needs for defense technology resulting from the end of the
cold war and concern with maintaining U.S industry's competitiveness
in global markets have led several members of Congress to open a
public debate and propose new legislation that addresses the national
laboratories' missions, structure, and cooperation with industry.\6
Among the alternatives being considered in the public debate are
reducing all the laboratories' budgets, consolidating or closing some
of them, and redirecting their weapons development mission toward
commercial product-related R&D in such areas as technology
development for environmental restoration, energy, and
high-performance computing. 

Underlying these discussions are questions about the type of R&D
activities the national laboratories are performing now, the nature
and scope of their outputs, and their potential for assisting
industry in bringing technology to the marketplace.  This report is
an effort to inform the debate by providing an empirical base for
these questions, as a starting point for addressing the broader
issues.  It examines whether the balance of laboratories' effort is
in basic and applied research or research related to commercial
product development, the distribution of the laboratories' research
outputs, and their potential for commercial application.  Findings
were based on a cross-section of the laboratories' R&D activities for
the period 1989-92.  However, the objectives for most of the programs
in the study population were initiated before the national
laboratories' legislative mandate for technology transfer in the
National Competitiveness Technology Transfer Act took effect in late
1989.  In most fields of R&D, more than 4 years are required for
outputs to evolve after objectives have been established.  Therefore,
the commercial product-related effort we found is to be considered a
baseline against which future activities and outputs can be measured. 


--------------------
\1 Ernest Orlando Lawrence founded the Radiation Laboratory for basic
scientific research on the University of California at Berkeley
campus in 1931.  It was funded under government contract in 1942. 
The laboratory was renamed Lawrence Radiation Laboratory after his
death in 1958 and later called Lawrence Berkeley. 

\2 The Atomic Energy Act of 1946 delegated responsibility for nuclear
weapons research, development, and manufacture as well as civilian
uses of nuclear energy to a civilian agency, the Atomic Energy
Commission.  Historically, civilian agencies have held this
responsibility in the United States and most other countries. 

\3 The National Renewable Energy Laboratory was established in 1977
as the Solar Energy Research Institute.  One of DOE's
program-directed laboratories, its R&D activities focus on developing
competitive renewable energy and related technologies for the nation
and on facilitating their commercialization. 

\4 A brief description of the 10 national laboratories' missions is
provided in appendix I. 

\5 See Technology Transfer:  Implementation Status of the Federal
Technology Transfer Act of 1986 (GAO/RCED-89-154, May 30, 1989),
Implementation of the Technology Transfer Act:  A Preliminary
Assessment (GAO/T-PEMD-90-4, May 3, 1990), and Diffusing Innovations: 
Implementing the Technology Transfer Act of 1986 (GAO/PEMD-91-23, May
29, 1991) for a discussion of congressional interest in technology
transfer from the national laboratories, as expressed in the
Stevenson-Wydler Technology Innovation Act of 1980 (P.L.  96-480) and
the Federal Technology Transfer Act of 1986 (P.L.  99-502).  The
implementation of Executive Order No.  12591, "Facilitating Access to
Science and Technology," which ordered executive departments and
agencies to facilitate collaboration between federal laboratories and
other public and private sector organizations, also is considered in
Diffusing Innovations.

Note that this legislation and our reports concern all laboratories
owned, leased, or otherwise used by a federal agency.  The 10
national laboratories are a subset of this population, which covers
at least 10 executive branch departments and 297 laboratories.  See
also the National Competitiveness Technology Transfer Act (P.L. 
101-189), which in 1989 authorized DOE to establish technology
transfer as a mission of government-owned, contractor-operated
laboratories, such as the national laboratories, and to approve
formation of cooperative research and development agreements (CRADAS)
between these laboratories and industry. 

\6 See U.S.  Congress, "Department of Energy Laboratory Technology
Act of 1993," H.R.1432, sections 2, 4, 5, and 9. 


   METHODOLOGY
------------------------------------------------------------ Letter :2

We began our work by developing a comprehensive description of
current research activities in the 10 laboratories.  We chose to
survey the laboratories directly because we could find no
sufficiently comprehensive existing documentation.  We collected our
data through a survey of the 10 laboratories' research programs and
the facilities and equipment that support them.  The survey scope
consisted of all major research programs and facilities with costs of
at least $10 million, as well as special nominations by the
laboratories themselves of other less costly programs and
facilities.\7

These two criteria were designed to ensure that all large subprograms
and smaller subprograms that were important to the laboratories'
missions would be included in our sample.  This allowed us to
describe the laboratories' major research efforts.  However, findings
based on these criteria should not be considered representative of a
laboratory's entire research effort since the proportion of programs
budgeted at less than $10 million can vary from one laboratory to
another. 

DOE's Budget and Reporting System categories provided a common
classification scheme for the laboratories' 12 research programs,
which permitted cross-laboratory comparisons of program
characteristics.  Research program and subprogram names are shown in
table 1.  The data we collected on these programs covered fiscal
years 1989-92. 

We conducted pilot tests of the survey methodology and data
collection instruments at Brookhaven and the National Renewable
Energy Laboratory.  We then revised the instrument and administered
one version to the remaining eight laboratories.\8 After we processed
the survey responses, we asked each laboratory to confirm by letter
that our list of research programs and



                           Table 1
           
            Classification for National Laboratory
                      Research Programs

Program             Subprogram category\a
------------------  ----------------------------------------
Energy Research     Magnetic fusion (AT)
                    High-energy physics (KA)
                    Superconducting supercollider (KS)
                    Nuclear physics (KB)
                    Basic energy sciences (KC)
                    Biological and environmental (KP)

Conservation and    Electric energy systems (AK)
Renewable Energy    Geothermal (AM)
                    Solar energy (EB)
                    Building and community systems (EC)
                    Industrial Energy Conservation (ED)
                    Transportation (EE)
                    State and local programs (EF)

Environment,        Environment, safety, and health (HA)
Safety and Health

Nuclear Energy      Nuclear energy R&D (AF)
                    Uranium enrichment (CD)

Defense Programs    Weapons activities (GB)
                    Verification and control technologies
                    (GC)
                    Nuclear safeguards and security (GD)
                    Production and surveillance support to
                    the nuclear weapons complex (NM)

New Production      New production reactors (NP)
Reactors

Environmental       Environmental restoration and waste
Restoration and     management--defense (EW, EM)
Waste Management    Environmental restoration and waste
                    management--nondefense (EX)

Fossil Energy       Coal (AA)
                    Petroleum (AC)
                    Strategic petroleum reserve (SA)

Civilian            Nuclear waste fund (DB)
Radioactive Waste
Management

Policy Planning     Policy, analysis, and systems studies
and Analysis        (PE)

Intelligence        Intelligence (NT)

Work for others     Work for others (WFO)
------------------------------------------------------------
\a DOE's Budget and Reporting System subprogram category codes are in
parentheses.  For purposes of this project, we have used the code
"WFO" to identify work for others programs. 

Source:  National laboratories' institutional plans for fiscal years
1991-96. 

The national laboratories engage in a wide range of defense and
nondefense R&D-related activities.  These range from generating
hypotheses and testing fundamental science principles to assisting a
potential user in adapting laboratory outputs to a production or
service delivery system.  To analyze the extent to which the
laboratories are engaged in basic and applied research or research
related to commercial product development, we divided their
activities into five categories:  basic research, applied research,
development, technology transfer, and technical assistance. 

Basic research is research undertaken primarily to gain fuller
knowledge or understanding of a subject and to contribute to the
knowledge base in the field of investigation.  Applied research is
research directed toward the practical use of knowledge or
understanding of a subject to meet a recognized need.  Development is
research directed toward the production of useful materials, devices,
systems, or methods, including the design and development of
prototypes or processes.  Development has some type of product as the
output goal, but may conclude with a prototype rather than a usable
good.  Additional time, research, and testing are usually required to
convert the prototype to a weapon or commercially viable product. 

Because the national laboratories perform R&D only through the
development stage, additional mechanisms and arrangements are
required to achieve application of the laboratories' outputs in the
public or private sector.  These activities are technology transfer
and technical assistance.  Technology transfer is the process that
fosters the use of devices, processes, "know-how," or scientific and
technical information produced in a national laboratory by
universities, private industry, or government agencies.  It includes
making potential users aware of the laboratories' research outputs,
assisting in their selection or use, and collaborating with
representatives of private industry and public or nonprofit
institutions to ensure that some of the laboratories' outputs will
have commercial or public applications.  Technical assistance applies
the laboratory's expertise to practical problems but does not involve
the use of a laboratory's outputs.  It is any form of assistance,
other than financial, to a state or local government or a business,
including publications, workshops, conferences, studies, or telephone
consultation. 

Development, technical assistance, and technology transfer are the
three national laboratory research activities related to commercial
product development.  All five categories, already used by the
laboratories but specially grouped for our analysis, constitute a
natural framework that, together with DOE's program classification
scheme, allowed us to look at R&D-related activity across all 10
laboratories, using expenditures as a measure of activity. 

Recognizing that the laboratories do not maintain records of their
R&D expenditures in terms of our five categories, we asked managers
of the subprograms in our study population to estimate, for each
subprogram they managed, the proportion of funds expended in each of
the five areas.  Our analysis of R&D activity is therefore presented
as percentages, not actual dollar values.  To provide a context for
considering our findings, we present in table 2 the fiscal year 1992
budgets for subprograms in the study population that were included in
our analysis and the laboratories' total budgets in fiscal year 1992. 



                           Table 2
           
            National Laboratory Budgets for Fiscal
                          Year 1992

                                           R&D         Total
National laboratory                   budget\a      budget\b
----------------------------------  ----------  ------------
Argonnne                                $357.8        $577.8
Lawrence Berkeley                        193.3         270.6
Oak Ridge                                346.6         726.9
Pacific Northwest                        340.4         417.0
Idaho National Engineering\c              98.7         931.5
Lawrence Livermore                       727.6        1395.5
Los Alamos                               710.9        1239.6
Sandia                                   722.1        1389.6
Brookhaven                               253.1         472.6
National Renewable Energy                113.4         131.2
============================================================
Total                                 $3,863.9      $7,552.3
------------------------------------------------------------
\a In millions of dollars; includes salaries and wages, overhead,
expendables, capital equipment, and other factors for a laboratory's
subprograms in the study population.  Construction costs are not
included.  Dollar values are the total of a laboratory's research
subprograms' budgets reported by program managers. 

\b In millions of dollars; includes salaries and wages, expendables,
overhead, capital equipment, general purpose equipment, construction,
and all other factors for all of a laboratory's research,
educational, administrative, and other activities.  Dollar values are
the estimated budget authorization reported in the laboratory 5-year
institutional plan. 

\c Idaho National Engineering Laboratory conducts energy, defense,
environmental, and nuclear reactor research; provides scientific
training in nuclear reactors and waste management; and designs,
constructs, and operates defense production facilities.  Idaho
laboratory representatives determined that much of their work in the
environmental, nuclear energy, and defense areas did not meet
criteria for inclusion in the inventory.  In addition, the total
laboratory budget estimate in the fiscal year 1992 institutional plan
included $250 million for construction, which was not included in the
R&D budget for the study population. 

We also examined the laboratories' outputs.  As output measures, we
selected products of laboratory R&D that were clearly identifiable to
our respondents and for which they were likely to maintain records. 
Since our study objective was to examine the balance among laboratory
activities rather than their impact, we focused on outputs of R&D
activity that occurred within the laboratories rather than their
efforts at job creation or increased sales.  Because of great
variation in the size, scope, field of investigation and funding
level of the subprograms in the study population, both within and
among laboratories, we presented our findings as simple tabulations,
rather than as standardized units.  Use of a single measure for
standardizing the outputs, such as dollar of funding per output,
would have failed to account for variations among the subprograms on
other dimensions.  Moreover, because of the institutional complexity
this variation represents, we interpreted our output findings very
conservatively, treating them as measures of activity rather than
indicators of performance.  We looked at the outputs in two broad
categories:  (1) publications and reports and (2) outputs related to
commercial product development.  The outputs attributed to each
category are described in the Principal Findings section. 

Finally, we looked at three other indicators--the formation of
cooperative R&D agreements, R&D effort devoted to critical
technologies, and program managers' assessment of their on-going
research--to gauge the laboratories' potential for commercial product
development. 

We conducted our review in accordance with generally accepted
government auditing standards.\9


--------------------
\7 A research program in the survey population is one of several
broad areas of research activity taking place within a laboratory
that had a total annual budget equal to or exceeding $10 million, was
funded in fiscal year 1992, and was planned to continue in fiscal
year 1993.  A program that did not meet these criteria could be
nominated by the laboratory for inclusion in the survey on the basis
of its uniqueness or contribution to science and technology
development.  Each laboratory could nominate up to 10 programs in
this category.  A facility in the survey population is an entity that
houses and comprises the equipment used in conducting R&D.  A
facility could be a building or defined structure, some area within a
structure, or a defined area not confined to a structure (for
example, a testing area).  An acquisition cost of $10 million or
nomination by the laboratory for its uniqueness, world-class quality,
synergistic effect when combined with other facilities, or
contribution to a given research program or project qualified a
facility for inclusion in the survey.  Each laboratory could nominate
up to 15 facilities that did not meet the $10-million acquisition
cost criterion. 

\8 The national laboratory inventory data collection instrument has
two parts--part I concerns the laboratory's research programs; part
II requests information about a laboratory's major research
facilities.  This report contains data for eight of the 10
laboratories from part I of the revised data collection instrument,
which is reproduced in appendix II.  Data reported for Brookhaven and
the National Renewable Energy Laboratory were collected with the
pilot-test versions of the data collection instrument, which were
somewhat different from the instrument in appendix II. 

\9 See appendix III for a more detailed discussion of survey
methodology. 


   RESULTS IN BRIEF
------------------------------------------------------------ Letter :3

The national laboratories devoted slightly more than half (52.4
percent) of their R&D funds to research related to commercial product
development during fiscal year 1992.  This includes 30.9 percent for
development, 14.4 percent for technical assistance, and 7 percent for
technology transfer.  However, most of the 10 laboratories'
development work (56.7 percent) was devoted to defense (which may
have more limited market opportunities) rather than nondefense
research.  Less than half (44.6 percent) of the laboratories' effort
was spent on basic and applied research:  17.4 percent on basic
research and 27.2 percent on applied research. 

The 10 laboratories produced many more publications and reports
(21,593) than they did outputs related to commercial product
development (2,510) in fiscal year 1992.  We expected this finding
because publications and reports are the primary mechanism for
disseminating the results of all types of R&D-related activities. 
Further, we found that the defense program supported most of the
commercial product-related outputs, and that Sandia, Los Alamos, and
Lawrence Livermore--which are known as weapons laboratories--produced
most of these outputs.  However, because we asked the laboratories to
classify these outputs--prototype devices and materials, algorithms,
software, and other commercial products or processes that have an
identified commercial use--at a time that is still several years away
from market entry, whether or to what degree they will actually
achieve commercial application is unknown. 

With regard to the laboratories' potential for commercial product
development, we found that such potential exists, based on the three
indicators we examined.  Activity is increasing in the formation of
cooperative R&D agreements between the laboratories and industry. 
These agreements increased from 17 in fiscal year 1989 to 196 in
fiscal year 1992.  Of course, they ensure only that the laboratories
and industry will collaborate on R&D a commercial innovation may or
may not be produced.  With respect to the research emphasis selected,
about three-fourths (74.1 percent) of the 10 laboratories' R&D
expenditures were focused on those technologies the National Critical
Technologies Panel had identified as vital to national needs.\10 Here
again, the potential for commercial product development exists, but
the actual outcome will not be known for several years.  Finally,
over half (57.5 percent) of the managers of programs with commercial
product potential expected clear evidence of that potential to emerge
within 5 years or less from fiscal year 1992. 


--------------------
\10 The Congress established the National Critical Technologies Panel
through the Fiscal Year 1990 Defense Authorization Act (P.L. 
101-189), an amendment to the National Science and Technology Policy,
Organization, and Priorities Act of 1976.  Congress asked the Panel
to identify up to 30 areas of technological development they consider
essential for the nation's long-term security and economic
prosperity.  The Panel reports to the Congress and the President on
the critical technologies biennially through the year 2000. 


   PRINCIPAL FINDINGS
------------------------------------------------------------ Letter :4


      BALANCE AMONG R&D-RELATED
      ACTIVITIES
---------------------------------------------------------- Letter :4.1

To examine the balance of the national laboratories' current
R&D-related activities, we analyzed the distribution of laboratory
expenditures for R&D within and among laboratories and research
programs.  For the 10 laboratories overall, R&D-related activity was
almost evenly divided between basic and applied research on the one
hand, and research related to commercial product development on the
other.  Approximately 8 percent more of the effort was devoted to R&D
activities related to commercial product development, as shown in
figure 1.  More applied research than basic research was conducted: 
27.2 percent versus 17.4 percent.  Among research activities related
to commercial product development, most (30.9 percent) was
development, but more activity was devoted to technical assistance
(14.4 percent) than technology transfer (7 percent).  Thus,
R&D-related activity directly targeted on potential commercial
applications of the laboratories' outputs currently constitutes the
smallest proportion of the laboratories' R&D-related effort.  Despite
its small size, however, this level of effort exceeds the
laboratories' minimum statutory requirement for technology transfer
activity.\11

   Figure 1:  National Laboratory
   Mean Percent Expenditures for
   R&D-Related Activities\a

   (See figure in printed
   edition.)

\a Fiscal year 1992. 

\b Subprogram expenditures for activities other than basic and
applied research or research related to commercial product
development, such as training graduate students and postdoctoral
fellows or safety procedures. 

\c Development, technical assistance, and technology transfer. 

These overall percentages, however, mask major differences among the
laboratories with regard to R&D funding distribution.  (See table
IV.1 in appendix IV.) Four laboratories--Argonne, Lawrence Berkeley,
Oak Ridge, and Brookhaven--spent 25 percent or more of their research
funds on basic research.  These laboratories account for over half
(59.3 percent) of the total national laboratory research budget that
is spent on basic research.  (See table IV.2.) Los Alamos spent 19.4
percent of its R&D funds to support its mission to perform "basic
research in selected disciplines that help maintain an outstanding
science and technology base." Only about 10 percent or less of the
laboratory research budget was spent on basic research at the other
laboratories.  The energy research program accounted for the greatest
proportion of funds spent on basic research, both within and among
research program areas.  (See tables IV.3 and IV.4.)

As table IV.1 shows, four laboratories--Oak Ridge, Pacific Northwest,
Lawrence Livermore, and Los Alamos--spent 29 percent or more of their
research funds on applied research.  Among the 10 laboratories,
Lawrence Livermore and Los Alamos accounted for almost half (47.9
percent) of applied research expenditures.  (See table IV.2.) Most
applied research was supported by programs in the areas of defense,
energy research, and work for others.  (See table IV.4.)

As noted earlier, most of the laboratories' development work, the
most product-oriented of R&D activities, was devoted to defense,
rather than nondefense, research.  Almost three-quarters (71.5
percent) of all the laboratories' development research was conducted
at Lawrence Livermore, Los Alamos, and Sandia.  (See table IV.2.)\12
In turn, the largest share of development research was performed in
the defense and nuclear energy programs.  (See tables IV.3 and IV.4.)
Therefore, while it is true that across the 10 laboratories, a
greater proportion of research funding was devoted to activities more
closely related to commercial product development than to basic and
applied research, most of these funds currently support defense
research.  To determine whether this research will have commercial
opportunities for use, we examined the national laboratories'


--------------------
\11 The Stevenson-Wydler Act requires that each federal agency with
one laboratory or more make available at least one-half of 1 percent
of its R&D budget for technology transfer activities and that
laboratories having 200 or more scientific, engineering, and
technical full-time-equivalent (FTE) staff assign at least one FTE to
an Office of Research and Technology Applications, which has formal
responsibility for the laboratory's technology transfer activities. 

\12 Pacific Northwest and the National Renewable Energy Laboratory
each spend almost one-third of their research funds on development. 
However, these laboratories are funded at a substantially lower level
than the weapons laboratories. 


      NATIONAL LABORATORIES' R&D
      OUTPUTS
---------------------------------------------------------- Letter :4.2

A second measure of the type of effort in which the national
laboratories are engaged--as between basic and applied research or
research related to commercial product development--is output.  The
laboratories produce two major types of outputs:  (1) publications
and reports, and (2) outputs related to commercial product
development.\13 Table 3 shows that, across a 4-year period, most of
the laboratories' outputs were publications and reports.  This
finding was expected because reports and publications are the primary
mechanisms for diffusion of R&D findings, and they are prepared at
all stages of the R&D process.  Reports, conference papers, and
published articles, which can be produced more quickly than books and
book chapters, substantially outnumber the latter. 

As we discussed above, a slightly higher percentage of the
laboratories' expenditures was devoted to R&D activities related to
commercial product development than to basic and applied research;
nevertheless, few of their outputs were commercial
product-related.\14 Prototype devices and materials, algorithms, and
software are the largest number of outputs in this group.  These
outputs tend to arise from the development stage of the R&D process,
which often occurs several years before production of a marketable or
usable good.  Not all outputs of the development stage will, of
course, achieve commercial application. 

Most of the prototype devices and materials, algorithms, and
software, as indicated in tables V.1 and V.2 in appendix V, were
produced at the weapons laboratories, and most were funded by DOE's
defense program.  Other outputs laboratory managers identified as
commercial products or commercial processes also tend to arise from
the development stage.  Although they will require a substantial
additional investment before they are ready to market, these products
or processes will more likely result in actual commercial
applications because a potential commercial use has already been
identified.  Most of these outputs were produced by Los Alamos,
Sandia, and Pacific Northwest, and the defense program supports most
of the research that has led to these outputs.  The point here is
that although defense-funded R&D has produced more outputs that could
lead to commercial products, whether these outputs will achieve
commercial application is still unknown. 



                                     Table 3
                     
                     National Laboratories' Research Program
                         Outputs for Fiscal Years 1989-92


              All    Any    All    Any    All    Any    All    Any    All    Any
                4      4      4      4      4      4      4      4      4      4
            years  years  years  years  years  years  years  years  years  years
Outputs        \a     \b     \a     \b     \a     \b     \a     \b     \a     \b
----------  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----
Publicatio
 ns and
 reports
Books          88     94    120    121    181    181    120    120    509    516
Articles    6,153  6,471  6,330  6,387  5,612  5,652  5,959  5,970  24,05  24,48
                                                                        4      0
Book          355    371    269    271    321    322    280    280  1,225  1,244
 chapters\c
Reports     6,245  6,802  5,416  5,542  4,761  4,851  4,497  4,533  20,91  21,72
                                                                        9      8
Conference  7,237  7,855  6,812  6,931  6,498  6,572  6,421  6,449  26,96  27,80
 papers                                                                 8      7
================================================================================
Subtotal    20,07  21,59  18,94  19,25  17,37  17,57  17,27  17,35  73,67  75,77
                8      3      7      2      3      8      7      2      5      5
Outputs
 related
 to
 commercia
 l product
 developme
 nt
Prototypes    501    566    485    490    497    500    492    492  1,975  2,048
Algorithms    732    778    713    715    632    634    552    552  2,629  2,679
Software      508    554    417    422    350    381    314    315  1,589  1,669
Patents\c     159    237    139    199    145    203    148    166    591    805
Licenses      210    303    128    149     42     60     39     47    419    559
Commercial     28     36     17     24     19     26     17     21     81    107
 products
Commercial     22     29     13     20     11     18     14     18     60     86
 processes
Spin-off        6      7      2      3      8     11     11     12     27     33
 company\c
================================================================================
Subtotal    2,166  2,510  1,914  2,022  1,704  1,833  1,587  1,623  7,371  7,988
Other
New           153    159    142    146    137    139     98    100    530    544
 programs\c
Invention     320    617    336    618    247    481    232    341  1,135  2,057
 disclosur
 es\c
Other\d       510    523    472    483    412    421    404    414  1,798  1,841
================================================================================
Subtotal      983  1,299    950  1,247    796  1,041    734    855  3,463  4,442
--------------------------------------------------------------------------------
\a These are outputs for subprograms that were in operation every
year during fiscal years 1989-92. 

\b These are outputs for subprograms that were initiated in any year
during fiscal years 1989-92. 

\c Responses were not collected from Brookhaven National Laboratory
and National Renewable Energy Laboratory. 

\d Research subprogram outputs other than those listed above, such as
technical abstracts, workshops for laboratory users, and an
electronic bulletin board service. 

Patent applications may be submitted for inventions throughout the
entire R&D process, but a license is usually acquired only when a
decision to market a technology has been made.  The number of
licenses awarded, therefore, is a stronger measure of output activity
related to commercial product development than the number of patents. 
A trend in the data indicative of the laboratories' production of
outputs related to commercial product development is the increase in
the number of licenses awarded during fiscal years 1989 through 1992. 
(See table 3.) In fiscal year 1992, Sandia and Pacific Northwest
awarded the most licenses, and most licensed outputs were supported
by defense program research.  (See tables V.1 and V.2.)

We expected to find that most commercial product-related outputs were
supported by research programs that spent most of their R&D funds for
development.  However, the R&D expenditures of those programs that
supported the most outputs related to commercial product development
covered the range of R&D activities.  We found that in fiscal year
1992, four research programs--energy research, conservation and
renewable energy, defense, and work for others--supported most of the
commercial product-related outputs of all types and that, over 4
years, commercial product-related output production had been
increasing each year in three of the programs, as shown in figure
V.1.  We also found that in fiscal year 1992, the largest proportion
of expenditures in the defense and conservation and renewable energy
programs was for development. 

As expected, the defense and conservation and renewable energy
programs supported more of the outputs specifically designated as
commercial products and processes than any of the 10 other research
programs.  However, in looking more closely at these four programs,
we found some interesting differences.  Work for others, which
supports more commercial product and process-type outputs than eight
other programs, devoted a slightly higher proportion of R&D
expenditures to applied research than to development.  But in energy
research, which supports more commercial product- and process-related
outputs than nine other programs, the largest proportion of
expenditures was for basic research.  (See table V.3.)


--------------------
\13 A third set of outputs, which are not related to these
categories, is designated as "other."

\14 Prototype devices and materials, algorithms, software, patents,
licenses, commercial products, commercial processes, and spin-off
companies are defined as laboratory outputs related to commercial
product development.  The research program managers have identified a
potential commercial use for commercial products and commercial
processes, but these outputs have achieved at most a precompetitive
stage of development.  We view patents, licenses, and spin-off
companies as evidence of intent to pursue a marketing strategy for a
research output. 


      LABORATORIES' POTENTIAL FOR
      COMMERCIAL PRODUCT
      DEVELOPMENT
---------------------------------------------------------- Letter :4.3

We looked at three indicators of the national laboratories' potential
for commercial product development:  (1) formation of cooperative
research and development agreements; (2) proportion of R&D
expenditures in critical technology areas; and (3) research program
managers' judgments about their programs' outputs.  Of these three,
the most frequently used indicator of the national laboratories'
potential for commercial product development is the formation of
CRADAs.\15 Here we found a major increase in activity.  The national
laboratories reported that from fiscal year 1989 through 1992, they
entered into 196 CRADAs.  Among programs in the study population in
operation all 4 years, the number of new CRADAs formed increased from
17 in fiscal 1989 to 130 in fiscal 1992. 

Sandia and Oak Ridge laboratories were most active in entering into
CRADAs.  (See table VI.1.  in appendix VI.) Most were formed for
research sponsored by programs in the defense and conservation and
renewable energy areas.  (See table VI.2.) The greatest increase in
CRADA formation occurred at Sandia, where 74 CRADAs were in effect in
fiscal year 1992.  Fifty-three of the CRADAs effective in fiscal year
1992 were sponsored by the defense program technology transfer
initiative at Sandia.  This subprogram was initiated in June 1990 to
identify opportunities for commercializing technologies produced by
DOE-funded defense research activities in such areas as advanced
manufacturing and precision engineering, materials and processes,
advanced microelectronics and photonics, and computer architecture
and applications. 

Although the national laboratories do not yet have a legislative
mandate or mission for research in the critical technologies, their
research program managers reported that 74.1 percent of R&D
expenditures are devoted to work in critical technology areas.  This
research was distributed over the 22 areas identified by the National
Critical Technologies Panel, with the greatest concentration in
energy technologies (13.6 percent); pollution minimization,
remediation, and waste management (8.8 percent); computer simulation
and modeling (6.7 percent); and materials synthesis and processing
(6.2 percent).  (See table VI.3.)

Work in these critical technology areas was distributed broadly among
the laboratories and research programs.  Five laboratories--Argonne,
Lawrence Berkeley, Oak Ridge, Idaho, and Lawrence Livermore--devoted
approximately 20-30 percent of their research funds to energy
technologies.  (See table VI.3.) Pacific Northwest expended the
greatest proportion of R&D funds (41.3 percent) on pollution
minimization technologies.  Idaho and Lawrence Livermore were most
active in computer simulation and modeling.  Oak Ridge and Los Alamos
devoted the greatest percentage of effort to materials synthesis and
processing.  As a group, the laboratories devoted approximately
three-fourths of their R&D expenditures to research in critical
technology areas, but Sandia and Los Alamos expended only about half
of their resources on critical technologies research.  All of the
research programs sponsored research in critical technologies to some
degree, with the least effort expended by environment, safety, and
health.  (See table VI.4.)

Finally, laboratory research program managers' judgments about their
research programs' potential for commercial product development were
optimistic.  Among the subset of all national laboratory programs
with a potential for commercial product development, almost 58
percent of the program managers expected that development to occur
within 5 years of fiscal 1992.  (See figure VI.1.) An additional 27.6
percent reported that their program has the potential for commercial
product development within 5-10 years. 


--------------------
\15 A CRADA is a contractual provision created to foster technology
transfer from federal laboratories to the private sector.  Agreements
can be formed with businesses as well as nonprofit organizations and
state and local government agencies.  The National Competitiveness
Technology Transfer Act of 1989 authorized government-owned,
contractor-operated laboratories, such as the national laboratories,
to enter into CRADAs.

CRADA formation represents commercial product potential because it
establishes the process uniquely for conducting research related to
commercial product development.  The conclusion of the period of
performance does not guarantee that the research will have been
completed or that a market-ready product will have been developed. 


   CONCLUSIONS
------------------------------------------------------------ Letter :5

As of 1992, the national laboratories spent slightly more than half
of their R&D funds on research related to commercial product
development.  However, most of this R&D was performed at the weapons
laboratories and was supported by the defense and nuclear energy
programs.  Analysis of the outputs produced by the national
laboratories indicated that defense-funded research produced more
outputs--prototype devices and materials, algorithms, software, and
other products and processes that have an identified commercial
application--that are precursors to marketable goods, but at this
point, whether they will achieve commercial application is not known. 

Moreover, three indicators of the laboratories' potential for
commercial product development--CRADA formation, critical technology
research, and program managers' expectations for commercial
potential--showed that some activity was occurring.  CRADA formation
was increasing, but these arrangements ensure only that collaboration
between the laboratories and industry will occur, not that a
commercial product will be generated.  Almost three-fourths of the
laboratories' effort was devoted to research in critical technology
areas, but achievement of commercial application will not be known
for several years.  Over half of the managers of research subprograms
that have commercial product potential expected innovations to arise
within 5 years, but these expectations must be considered "best
educated guesses." While we can conclude, therefore, that the
national laboratories' were engaged in slightly more research related
to commercial product development than basic and applied research, it
is too early to determine whether this activity will produce
technologies with commercial uses. 


   AGENCY COMMENTS
------------------------------------------------------------ Letter :6

We requested comments on a draft report and received a response from
DOE and the 10 national laboratories.  DOE questioned the definitions
and categories we defined to analyze the laboratories' R&D-related
activities and our finding that the laboratories perform slightly
more research related to commercial product development than basic
and applied research.  DOE also thought that this study should have
examined additional institutional factors, including the R&D
activities of other agencies, and should have used data maintained by
DOE headquarters rather than surveyed the laboratories for data. 

We note that the definitions for R&D-related activities we employed
are derived from a Congressional Budget Office study of the federal
R&D enterprise, our study of the Technology Transfer Act of 1986, and
expert opinion.  We also disagree with DOE's proposed broader scope
for this study because it exceeds our study objective and would have
required additional data collection and analyses that are beyond the
study scope.  Furthermore, our exploration of data available at DOE
headquarters found that it was not adequate to satisfy our
information needs. 

Eight laboratories agreed with the report's objective, analyses, and
conclusions.  However, one of this group, Lawrence Berkeley, thought
that the relationship of commercial product development to the
broader needs of industry and the nation should have been addressed
in the study.  Two of the laboratories raised issues about study
methodology.  Idaho believed that a greater proportion of the budget
for its subprograms should have been included in the study sample. 
Oak Ridge questioned the effect of the study's sampling methodology
on output findings for the laboratory and the definition of the
category called outputs related to commercial product development. 

Lawrence Berkeley said that we had overlooked an important issue. 
The laboratory thought that the study should have included an
examination of the relationship of the national laboratories' role in
commercial product development to the broader needs of industry and
the nation.  We agree that this issue is important to address as part
of the public debate about the laboratories' missions and structure. 
However, we disagree that it should have been examined in this
report, which focuses on establishing an empirical baseline of
national laboratories' activities. 

DOE's Idaho Operations Office responded for Idaho National
Engineering Laboratory.  The Idaho Operations Office said that the
budget figure reported for Idaho subprograms included in the study
sample should have been higher.  We did not agree to revise Idaho's
budget figure, because to do so would have violated the study
methodology used to sample programs at other laboratories. 

Oak Ridge took the position that most of its commercial
product-related outputs were produced by subprograms that were not
selected in the study sample because they were funded at less than
$10 million.  The laboratory expressed concern that the subprograms
we sampled produced only 7 percent of its commercial product-related
outputs while representing 73 percent of its overall budget.  Oak
Ridge based this position on summary output data for the entire
laboratory and sampled subprograms that laboratory representatives
had tabulated.  Again, we could not include the output data for Oak
Ridge's unsampled programs in our analyses without violating the
sampling methodology.  We also had some questions about the large
number of outputs the Oak Ridge analysis ascribed to unsampled
programs. 

Oak Ridge also thought that our definitions for these outputs equated
the laboratories' development work with commercial product
development.  We disagree.  The definitions we used make it clear
that the laboratories were not expected to produce commercial
products.  Our conclusion reiterates that the laboratories' outputs
related to commercial product development are "precursors to
marketable goods" and that "whether they will achieve commercial
application is not known."

We provide a more detailed discussion of all these comments and our
response in appendixes VII through XVII. 

As agreed with your offices, we plan no further distribution of this
report until 30 days from its date of issue, unless you publicly
announce its contents earlier.  We will then send copies to
interested parties, and we will also make copies available to others
upon request. 

If you have any questions or would like additional information,
please call me at (202) 512-3092.  Other major contributors to this
report are listed in appendix XVIII. 

Kwai-Cheung Chan
Director of Program Evaluation in
 Physical Systems Areas


DESCRIPTION OF THE NATIONAL
LABORATORIES
=========================================================== Appendix I

The descriptions of the national laboratories are adapted from the
5-year institutional plans that the laboratories update and issue
annually and from U.S.  Department of Energy, Multiprogram
Laboratories, 1979 to 1988, A Decade of Change (Washington, D.C.: 
Apr.  1990). 

ARGONNE NATIONAL LABORATORY

Argonne was established in 1946.  The University of Chicago operates
the laboratory, which develops and operates national facilities for
use by university, industry, and national laboratory groups; performs
basic research, technology-directed research and technology
evaluations; and conducts technology transfer through cooperative
research, and development agreements, sponsored research, staff
exchanges, and licensing of intellectual property or through the
formation of new firms by the laboratory's Arch Development
Corporation. 

The laboratory's basic research effort includes experimental and
theoretical research on fundamental problems in the physical, life,
and environmental sciences to advance scientific understanding and
support energy technology development.  Argonne's technology-directed
research includes conceptualization, design, and testing of advanced
fission reactors and other technologies for power applications in
both the civilian and defense sectors and investigations of
strategies for overcoming materials, chemical, and electrochemical
barriers to the development of these technologies.  Argonne also
supports DOE and, where appropriate, other federal agencies in
characterizing and evaluating nationally important projects and
technology options in terms of their environmental cost or other
implications. 

LAWRENCE BERKELEY LABORATORY

Lawrence Berkeley, founded in 1931 as the Radiation Laboratory by
Ernest Orlando Lawrence of the University of California at Berkeley,
was one of the original national laboratories.  It was funded under
government contract in 1942.  The University of California, which
operates the laboratory, renamed it the Lawrence Radiation Laboratory
after his death in 1958, and later called it Lawrence Berkeley.  The
laboratory conducts a wide range of interdisciplinary research with
core competencies in biosciences and biotechnology; particle and
photon beams; advanced detector systems; characterization and
synthesis of materials; chemical dynamics, catalysis, and surface
sciences; advanced techniques for energy supply and energy
efficiency; and environmental assessment and remediation.  It
performs research in the energy, physical, and life sciences;
develops and operates national experimental facilities; fosters
industry's interactions with the laboratory's research programs; and
offers scientific and engineering education programs. 

The laboratory's work in the energy sciences includes applied
science, such as the energy efficiency of buildings; chemical
sciences, such as the structure and reactivity of transient species;
earth sciences, including geophysical imaging methods, isotopic
geochemistry and physicochemical process investigation; and materials
sciences, such as advanced ceramic, metallic, and polymeric materials
for electronic, magnetic, catalytic, and structural applications. 
Accelerator and fusion research, nuclear science, and physics are
pursued in the general science area.  Lawrence Berkeley's work in the
life sciences includes cellular and molecular biology, chemical
biodynamics, and research medicine and radiation biophysics.  This
work is supported by the laboratory's scientific and technical
resources in the areas of engineering, information and computing
sciences, and occupational health. 

OAK RIDGE NATIONAL LABORATORY

Oak Ridge was one of the original national laboratories.  Now
operated by Martin Marietta Energy Systems, Oak Ridge was established
in 1943.  The laboratory's R&D activities are focused on basic and
applied research, technology development, and other technological
challenges in areas that include energy production and conservation
technologies; experimental and theoretical research in physical,
chemical, materials, computational, biomedical, earth, environmental,
and social sciences; the design, building, and operation of unique
research facilities for the benefit of university, industrial, and
other federal agency and national laboratory researchers; and the
development of environmental protection and waste management
technologies.  Oak Ridge also performs technology transfer and offers
educational services from the preschool through the postdoctoral
level. 

PACIFIC NORTHWEST LABORATORY

Pacific Northwest was established in 1965.  Battelle Memorial
Institute now operates the laboratory, which performs scientific
research and rapid technology development and deployment to meet
national needs.  Laboratory efforts include molecular science,
hazardous waste characterization, global environmental studies,
subsurface science, biological systems, technical support for
environmental policies and procedures, federal infrastructure
modernization, national security technology, energy-efficient
methods, advanced analytical methods, materials research, magnetic
fusion research, civilian nuclear waste management, technical support
for nuclear power plant operation, space exploration technology,
fossil fuel technology, renewable energy sources, energy policy
analysis, and surveillance and oversight of operations at its Hanford
site. 

IDAHO NATIONAL ENGINEERING
LABORATORY

This laboratory was established in 1949.  Three contractors operated
the laboratory during the time period of our study:  Westinghouse
Idaho Nuclear Co., Rockwell-INEL, and EG&G Idaho.  The laboratory's
areas of primary emphasis are nuclear reactor technology R&D, defense
production-related support, waste management and environmental
restoration analysis, advanced energy production technology
development, and research and development on energy and environmental
issues, including performance testing of industry-developed electric
vehicles, small hydropower and geothermal power production, and
fossil energy research.  Idaho also offers educational activities and
performs technology transfer. 

LAWRENCE LIVERMORE NATIONAL
LABORATORY

Lawrence Livermore was established in 1952.  The University of
California operates the laboratory, which serves as a national
resource in science and engineering, focused on national security,
energy, environment, biomedicine, economic competitiveness, and
science and mathematics education, with a special responsibility for
nuclear weapons.  National security has traditionally been a special
focus of the laboratory's research and development effort.  Lawrence
Livermore's major areas of activity have included research,
development, and testing for all phases of the nuclear weapons life
cycle; strategic defense research; arms control and treaty
verification technology; inertial confinement fusion; atomic vapor
laser isotope separation; magnetic fusion; other energy research;
research in biological, ecological, atmospheric, and geophysical
sciences; charged-particle beam and free-electron laser research;
advanced laser and optical technology applications; technology
transfer; and science education.  The laboratory also participates in
human genome research as part of a nationally directed initiative. 

LOS ALAMOS NATIONAL LABORATORY

Los Alamos, one of the original national laboratories, was
established in 1943 and is operated by the University of California. 
Ensuring the nation's deterrence capability through nuclear weapons
technology is the laboratory's primary focus.  Los Alamos' major R&D
activities include research, design, development, engineering, and
testing of nuclear warheads; maintenance and enhancement of the
weapons technology base and warhead stockpile management; research,
development, and testing support for advanced nuclear directed-energy
concepts; nuclear materials R&D for the nuclear weapons program;
nonnuclear strategic defense R&D activities; advanced conventional
munitions development and simulation; verification and safeguards
R&D vulnerability, lethality, effects, and countermeasures research;
advanced defense technologies; intelligence activities involving
hardware analysis and technology security; weapons and energy
technology systems studies; and R&D in nonnuclear energy and
technology areas.  The laboratory's basic research activities in
defense and energy areas include atomic and molecular physics,
bioscience, chemistry, computational science and applied mathematics,
geoscience, space science, astrophysics, materials science, nuclear
and particle physics, plasma physics, fluids, and particle beams. 
Los Alamos also performs technology transfer and offers science and
engineering education programs. 

SANDIA NATIONAL LABORATORIES

Sandia was established in 1949 under an agreement with AT&T to
operate the laboratory for the government as a public service on a
nonprofit basis.  AT&T stepped out of this role in 1993.  A contract
was recently awarded to Martin Marietta Corporation to operate the
laboratories.  Sandia's major areas of effort are nuclear weapons,
arms control and treaty verification, environmental restoration and
waste management, energy supply and conservation, advanced
conventional military technologies, and other programs in the
national interest.  The laboratories' R&D activities in these areas
include research, development, and engineering associated with
advancing nuclear explosives to integrated, functional weapons for
Department of Defense weapon delivery systems; other defense
programs, including development of verification and control
technologies to support arms reduction and concepts and systems for
the safeguarding and security of nuclear materials; research,
development, and engineering for hazardous waste removal,
minimization, and remediation; and nonnuclear energy research in
energy efficiency, recovery techniques, conversion technologies,
alternative energy sources, characterization of environmental change
phenomena, environmental restoration technologies, and basic energy
sciences.  Sandia also conducts technology transfer and offers
mathematics and science education opportunities. 

BROOKHAVEN NATIONAL LABORATORY

Brookhaven was established in 1947 by a group of nine universities to
facilitate their mutual access to large-scale research facilities,
particularly in nuclear science.  The laboratory is operated by
Associated Universities, a corporation governed by a board of
trustees representing the original nine universities as well as other
universities, research institutions, and industrial organizations. 
Brookhaven's primary role is to conceive, design, build, and operate
large-scale, complex facilities for scientific research and to
conduct basic and applied research in energy-related physical, life,
and environmental sciences.  When feasible, Brookhaven makes its
laboratory facilities available to state and federal agencies,
universities, and private industry.  The laboratory's major areas of
R&D are high-energy and nuclear physics; basic energy sciences
emphasizing research on biological, chemical, and physical phenomena
underlying energy-related transfer, conversion, and storage systems;
life sciences, nuclear medicine, and medical applications of nuclear
techniques; and a broad span of applied programs that draw on the
laboratory's unique capabilities.  Brookhaven makes all useful
results and knowledge obtained from its research activities available
to private industry.  Brookhaven also performs technology transfer
and offers science and engineering education programs. 

NATIONAL RENEWABLE ENERGY
LABORATORY

The former Solar Energy Research Institute was designated a DOE
national laboratory in 1991 and renamed the National Renewable Energy
Laboratory.  The focus of the laboratory's effort is on developing
competitive renewable energy and related technologies and
facilitating their commercialization.  The laboratory's R&D
activities include basic and applied research, exploratory and
advanced development and other activities in renewable energy and
related technologies; analytic studies and technology evaluations;
and collaborative R&D with universities and industry.  The laboratory
also manages subcontracted R&D on behalf of DOE and serves as a
source of scientific and technical information on renewable energy. 




(See figure in printed edition.)Appendix II
NATIONAL LABORATORY INVENTORY
RESEARCH PROGRAM DATA COLLECTION
INSTRUMENT
=========================================================== Appendix I



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)


SURVEY METHODOLOGY
========================================================= Appendix III

SURVEY RESPONSE

The research program survey population was enumerated by applying
selection criteria to each laboratory's research programs.  After
processing the surveys, we sent the laboratories a letter requesting
confirmation that our list of research programs and subprograms was
complete.  In response to our letter, the laboratories confirmed a
total of 252 research subprograms.  The laboratories returned a total
of 247 data collection instruments, for a survey response rate of 98
percent. 

DATA QUALITY ISSUES

The data contained in this report are results of analyses of national
laboratory program managers' responses to questions 5, 6, 7, 9, and
10 in part I of the national laboratory inventory.  These responses
represent program managers' judgments or self-reports about question
elements, as follows.  We made no attempt to validate these responses
through independent sources. 


      QUESTION 5
----------------------------------------------------- Appendix III:0.1

Responses are research program managers' best estimates of the
proportion of the total program budget expended for each R&D-related
activity.  Although they had our definitions for key R&D-related
activities listed in the data collection instrument, their responses
also may reflect their own understanding of terms such as basic
research, applied research, or technical assistance. 


      QUESTION 6
----------------------------------------------------- Appendix III:0.2

Responses are research program managers' best estimates of the
proportion of the total program budget expended for research in
critical technology areas.  The response categories in the question
are the critical technologies identified by the National Critical
Technologies Panel.  Some overlap may exist among these categories
because they were not identified for research measurement purposes. 
The Panel's critical technology categories were used in this question
to determine the congruence between research already being conducted
at the national laboratories and the research needs articulated by a
congressionally mandated body. 

A few responses submitted for this question summed to more than 100
percent.  These responses were prorated to include them in the
calculation of mean percent expenditures for R&D in critical
technologies. 


      QUESTION 7
----------------------------------------------------- Appendix III:0.3

Responses are research program managers' reports of research program
outputs.  The responses concerning commercial products and commercial
processes are judgments made about research outputs that have reached
only the precompetitive stage of the R&D process. 


      QUESTION 9
----------------------------------------------------- Appendix III:0.4

Responses are research program managers' reports about CRADAs in
effect through the end of fiscal year 1992. 


      QUESTION 10
----------------------------------------------------- Appendix III:0.5

Responses are research program managers' judgments about potential
industrial application or commercial product development for outputs
of their research program over a 20-year planning horizon.  The size
of research subprograms in the study population varied; thus,
managers were considering outputs of one or more research activities
in making their assessments. 


BALANCE AMONG THE NATIONAL
LABORATORIES' R&D ACTIVITIES
========================================================== Appendix IV

Data concerning the distribution of the national laboratories'
expenditures among R&D-related activities, by laboratory and by
program, are presented below. 



                                    Table IV.1
                     
                        Mean Percent Expenditures for R&D-
                     Related Activities Within Laboratories\a

R&D-related                                                                  All
activity       ANL   LBL  ORNL   PNL  INEL  LLNL  LANL   SNL   BNL  NREL    labs
------------  ----  ----  ----  ----  ----  ----  ----  ----  ----  ----  ------
Basic         29.9  39.7  25.1   9.8   2.2   7.6  19.4   4.9  53.6   6.9    17.4
 research
Applied       22.5  15.9  29.1  29.7  20.6  36.2  33.1  23.5   7.5  24.3    27.2
 research
Development   17.4   7.4  17.9  31.8  25.5  43.6  25.9  47.7  13.9  31.9    30.9
Technical
 assistance
 to
Government    10.8   4.3   5.9  21.9  15.8   6.2   8.6  11.9   7.8   4.5     9.7
 agencies
Universities   3.5  11.0   6.2   1.0   1.6   0.6   1.4   0.9   5.7   3.3     2.5
Private        1.6   3.5   5.0   1.2   1.4   0.8   1.7   2.0   2.9   7.9     2.2
 firms or
 industrial
 organizatio
 ns
Technology
 transfer to
Government     4.0  10.4   4.5   2.4   7.0   1.9   2.5   3.1   1.6   1.7     3.2
 agencies
Private        6.2   1.7   6.1   2.1   4.6   2.1   3.3   3.9   0.9  18.1     3.8
 firms or
 industrial
 organizatio
 ns
Other\b        4.1   6.1   0.2     0  21.3   1.0   4.1   2.1   6.2   1.5     3.0
================================================================================
Total          100   100   100   100   100   100   100   100   100   100     100
--------------------------------------------------------------------------------
Legend

ANL = Argonne National Laboratory
LBL = Lawrence Berkeley Laboratory
ORNL = Oak Ridge National Laboratory
PNL = Pacific Northwest Laboratory
INEL = Idaho National Engineering Laboratory
LLNL = Lawrence Livermore National Laboratory
LANL = Los Alamos National Laboratory
SNL = Sandia National Laboratories
BNL = Brookhaven National Laboratory
NREL = National Renewable Energy Laboratory

\a Fiscal year 1992. 

\b Subprogram expenditures for activities other than those listed
above, such as training graduate students and postdoctoral fellows or
safety procedures. 



                                    Table IV.2
                     
                        Mean Percent Expenditures for R&D-
                     Related Activities Among Laboratories\a

R&D-
related
activity   ANL   LBL  ORNL   PNL  INEL  LLNL  LANL   SNL   BNL    NREL
--------  ----  ----  ----  ----  ----  ----  ----  ----  ----  ------  --------
Basic     14.9  10.9  13.1   5.0   0.3   8.3  20.7   5.3  20.4     1.2  100
 research
Applied    7.2   2.8   9.7   9.6   2.0  25.3  22.6  16.3   1.8     2.7  100
 research
Developm   5.0   1.1   5.2   9.1   2.2  26.8  15.6  29.1   3.0     3.1  100
 ent
Technica
 l
 assista
 nce to
Governme   9.7   2.1   5.6  20.0   4.2  12.1  16.5  23.2   5.3     1.4  100
 nt
 agencie
 s
Universi  12.1  20.9  22.2   3.3   1.6   4.3  10.2   6.9  14.8     3.9  100
 ties
Private    6.4   7.6  20.8   4.8   1.7   7.3  14.7  17.1   9.0    10.7  100
 firms
 or
 industr
 ial
 organiz
 ations
Technolo
 gy
 transfe
 r to
Governme  10.8  15.4  12.6   6.7   5.6  11.4  14.5  18.2   3.3     1.6  100
 nt
 agencie
 s
Private   14.2   2.2  14.5   4.8   3.1  10.4  16.0  19.3   1.6    14.0  100
 firms
 or
 industr
 ial
 organiz
 ations
Other\b   11.9   9.6   0.7   0.0  18.1   6.4  25.2  13.1  13.6     1.5  100
================================================================================
All        8.7   4.8   9.1   8.8   2.6  19.0  18.6  18.9   6.6     3.0  100
 activity
--------------------------------------------------------------------------------
Legend

ANL = Argonne National Laboratory
LBL = Lawrence Berkeley Laboratory
ORNL = Oak Ridge National Laboratory
PNL = Pacific Northwest Laboratory
INEL = Idaho National Engineering Laboratory
LLNL = Lawrence Livermore National Laboratory
LANL = Los Alamos National Laboratory
SNL = Sandia National Laboratories
BNL = Brookhaven National Laboratory
NREL = National Renewable Energy Laboratory

\a Fiscal year 1992. 

\b Subprogram expenditures for activities other than those listed
above, such as training graduate students and postdoctoral fellows or
safety procedures. 



                                    Table IV.3
                     
                        Mean Percent Expenditures for R&D-
                       Related Activities Within Programs\a

R&D-
relat
ed                                                                           All
activ                                                                    program
ity     ER   CE   ES&H   NE   DP  NPR   ERWM   FE   CRWM  PPA  INT  WFO        s
-----  ---  ---  -----  ---  ---  ---  -----  ---  -----  ---  ---  ---  -------
Basic  53.  4.4      0  7.6  7.2    0    0.3  14.    8.2  3.3  2.0  11.     17.4
 rese    1                                      1                     5
 arch
Appli  15.  25.    8.8  29.  32.  7.6   32.4  31.   52.8  43.  50.  26.     27.2
 ed      2    9           4    5                5           3    0    1
 rese
 arch
Devel  10.  28.    9.8  51.  40.  27.   37.3  15.   17.3  6.7  15.  23.     30.9
 opme    1    6           7    9    7           8                0    4
 nt
Techn
 ical
 assi
 stan
 ce
 to
Gover  4.0  5.9   71.8  1.6  9.7    0   10.1  10.   11.6  46.  25.  25.      9.7
 nment                                          3           7    0    1
 agen
 cies
Unive  6.5  4.0    2.7  0.6  0.7    0    1.2  2.9    0.4    0    0  2.8      2.5
 rsit
 ies
Priva  2.8  11.    0.1  0.6  0.9    0    0.7  5.8      0    0    0  2.2      2.2
 te           6
 firm
 s or
 indu
 stri
 al
 orga
 niza
 tion
 s
Techn
 ology
 tran
 sfer
 to
Gover  4.1  3.1    2.4  2.0  2.8    0    3.3  7.7    1.7    0  8.0  4.3      3.2
 nment
 agen
 cies
Priva  2.5  15.    0.7  6.5  2.5    0    2.8  7.1      0    0    0  4.4      3.8
 te           7
 firm
 s or
 indu
 stri
 al
 orga
 niza
 tion
 s
Other  1.7  0.7    3.7    0  2.8  64.   11.8  4.9    8.0    0    0  0.4      3.0
 \b                                 7
================================================================================
Total  100  100    100  100  100  100    100  100    100  100  100  100      100
--------------------------------------------------------------------------------
Legend

ER = Energy Research
CE = Conservation and Renewable Energy
ES&H = Environment, Safety and Health
NE = Nuclear Energy
DP = Defense Programs
NPR = New Production Reactors
ERWM = Environmental Restoration and Waste Management
FE = Fossil Energy
CRWM = Civilian Radioactive Waste Management
PPA = Policy Planning and Analysis
INT = Intelligence
WFO = Work for Others

\a Fiscal year 1992. 

\b Subprogram expenditures for activities other than those listed
above, such as training graduate students and postdoctoral fellows or
safety procedures. 



                                    Table IV.4
                     
                        Mean Percent Expenditures for R&D-
                       Related Activities Among Programs\a

R&D-
related                                                                     Tota
activity   ER   CE   ES&H   NE   DP  NPR   ERWM   FE   CRWM  PPA  INT  WFO     l
--------  ---  ---  -----  ---  ---  ---  -----  ---  -----  ---  ---  ---  ----
Basic     68.  1.7      0  3.0  17.    0    0.1  0.2    0.6    0    0  7.7   100
 research   7                     8
Applied   12.  6.4    0.1  7.3  51.  0.2    7.9  0.3    2.4  0.1  0.1  11.   100
 research   6                     3                                      2
Developm  7.3  6.2    0.1  11.  56.  0.6    8.0  0.1    0.7    0    0  8.9   100
 ent                         3    7
Technica
 l
 assista
 nce to
Governme  9.4  4.1    3.0  1.1  43.    0    6.9  0.3    1.5  0.3  0.1  30.   100
 nt                               1                                      3
 agencie
 s
Universi  57.  10.    0.4  1.6  12.    0    3.2  0.3    0.2    0    0  12.   100
 ties       9    8                5                                      8
Private   29.  36.      0  1.9  17.    0    2.3  0.7      0    0    0  12.   100
 firms      1    3                6                                      0
 or
 industr
 ial
 organiz
 ations
Technolo
 gy
 transfe
 r to
Governme  28.  6.4    0.3  4.1  36.    0    6.9  0.6    0.6    0  0.1  15.   100
 nt         4                     9                                      5
 agencie
 s
Private   14.  27.    0.1  11.  27.    0    4.8  0.5      0    0    0  13.   100
 firms      5    8           5    5                                      4
 or
 industr
 ial
 organiz
 ations
Other\b   13.  1.6    0.5    0  39.  14.   25.9  0.4    3.3    0    0  1.4   100
            0                     2    6
================================================================================
All       22.  6.8    0.4  6.8  42.  0.7    6.6  0.3    1.2  0.1    0  11.   100
 activity   5                     9                                      7
--------------------------------------------------------------------------------
Legend

ER = Energy Research
CE = Conservation and Renewable Energy
ES&H = Environment, Safety and Health
NE = Nuclear Energy
DP = Defense Programs
NPR = New Production Reactors
ERWM = Environmental Restoration and Waste Management
FE = Fossil Energy
CRWM = Civilian Radioactive Waste Management
PPA = Policy Planning and Analysis
INT = Intelligence
WFO = Work for Others

\a Fiscal year 1992. 

\b Subprogram expenditures for activities other than those listed
above, such as training graduate students and postdoctoral fellows or
safety procedures. 


NATIONAL LABORATORIES' R&D OUTPUTS
=========================================================== Appendix V

Data concerning outputs of the national laboratories' R&D-related
activities, by laboratory and by program, are presented below. 



                                    Table V.1
                     
                     National Laboratories' Research Program
                             Outputs by Laboratory\a

Outputs        ANL   LBL  ORNL   PNL  INEL  LLNL  LANL   SNL   BNL  NREL   Total
------------  ----  ----  ----  ----  ----  ----  ----  ----  ----  ----  ------
Publications
 and reports
Books            8     2    16     1     3    17    14    10    14     9      94
Articles       738  1,63   955   210    79   461  1,13   534   634    92   6,471
                       4                             4
Book            71    35   154    11     5    51    25    19    \b    \b     371
 chapters
Reports        540   620  1,20   170   335  1,40   828  1,00   515   180   6,802
                             0                 5           9
Conference     662  1,02  1,43   369   245   747  1,61   968   534   247   7,855
 papers                7     7                       9
================================================================================
Subtotal      2,01  3,31  3,76   761   667  2,68  3,62  2,54  1,69   528  21,593
                 9     8     2                 1     0     0     7
Outputs         28    14    27    31    28    68    75   233    50    12     566
 related to
 commercial
 product
 development
 Prototypes
Algorithms      11     6     6    10     1    21    52   646    17     8     778
Software        36    11     7     3   120    57    96   193    23     8     554
Patents         14    12    12    38    14    37    33    77   \\b    \b     237
Licenses         2     3     4    83    16     8     5   145    35     2     303
Commercial       3     2     1     7     0     1     7     8     5     2      36
 products
Commercial       3     1     2     6     0     1     3    12     1     0      29
 processes
Spin-off         1     0     0     1     1     0     2     2    \b    \b       7
 company
================================================================================
Subtotal        98    49    59   179   180   193   273  1,31   131    32   2,510
                                                           6
Other
New program     20     8     3     0     2     6    29    91    \b    \b     159
Invention       43    32    54   122    54    89    11   212    \b    \b     617
 disclosures
Other\c        141     0     4     0     1    96    15     0   101   165     523
================================================================================
Subtotal       204    40    61   122    57   191    55   303   101   165   1,299
================================================================================
Total         2,32  3,40  3,88  1,06   904  3,06  3,94  4,15  1,92   725  25,402
                 1     7     2     2           5     8     9     9
--------------------------------------------------------------------------------
Legend

ANL = Argonne National Laboratory
LBL = Lawrence Berkeley Laboratory
ORNL = Oak Ridge National Laboratory
PNL = Pacific Northwest Laboratory
INEL = Idaho National Engineering Laboratory
LLNL = Lawrence Livermore National Laboratory
LANL = Los Alamos National Laboratory
SNL = Sandia National Laboratories
BNL = Brookhaven National Laboratory
NREL = National Renewable Energy Laboratory

\a Fiscal year 1992. 

\b Responses were not collected from Brookhaven National Laboratory
and National Renewable Energy Laboratory. 

\c Research subprogram outputs other than those listed above, such as
technical abstracts, workshops for laboratory users, and



                                    Table V.2
                     
                     National Laboratories' Research Program
                               Outputs by Program\a

Outputs\                                                                    Tota
b          ER   CE   ES&H   NE   DP  NPR   ERWM   FE   CRWM  PPA  INT  WFO     l
--------  ---  ---  -----  ---  ---  ---  -----  ---  -----  ---  ---  ---  ----
Publicat
 ions
 and
 reports
Books      46   12      0    3   23    0      1    0      0    0    0    8    93
Articles  3,9  475      5   33  1,0    0     45   12     32    2    0  666  6,26
           51                    40                                            1
Book      257   35      0    2   39    0      4    5      1    0    0   17   360
 chapter
 s\c
Reports   1,6  582     20  393  2,4   80    296   50     84   16    0  1,0  6,63
           92                    07                                     12     2
Conferen  3,4  663      6  219  2,0    9    215   44     70    2    0  773  7,48
 ce        63                    22                                            6
 papers
================================================================================
Subtotal  9,4  1,7     31  650  5,5   89    561  111    187   20    0  2,4  20,8
           09   67               31                                     76    32
Outputs
 related
 to
 commerc
 ial
 product
 develop
 ment
Prototyp   85   39      0   20  309    0     18    1      0    0    0   63   535
 es
Algorith   43   16      0   10  671    0      6    1      2    0    0   19   768
 ms
Software   49   17      0   29  233    0     17    3      0    3    0  200   551
Patents\   52   18      0    5  111    0      6    0      0    0    0    7   199
 c
Licenses   41   22      0    0  152    0      1    0      0    0    0    4   220
Commerci    3    7      0    1   13    0      1    0      0    0    0    4    29
 al
 product
 s
Commerci    3    4      0    1   13    0      1    0      0    0    0    1    23
 al
 process
 es
Spin-       3    0      0    0    3    0      0    0      0    0    0    0     6
 off
 company
 \c
================================================================================
Subtotal  279  123      0   66  1,5    0     50    5      2    3    0  298  2,33
                                 05                                            1

Other
--------------------------------------------------------------------------------
New        25   12      0    2   97    0      5    2      0    0    0   16   159
 program
 \c
Inventio   80   58      0   28  270    0     40    0      0    0    0   19   495
 n
 disclos
 ures\c
Other\d   139  168     90    0  108    0     11    6      0    0    0    1   523
================================================================================
Subtotal  244  238     90   30  475    0     56    8      0    0    0   36  1,17
                                                                               7
================================================================================
Total     9,9  2,1    121  746  7,5   89    667  124    189   23    0  2,8  24,3
           32   28               11                                     10    40
--------------------------------------------------------------------------------
Legend

ER = Energy Research
CE = Conservation and Renewable Energy
ES&H = Environment, Safety and Health
NE = Nuclear Energy
DP = Defense Programs
NPR = New Production Reactors
ERWM = Environmental Restoration and Waste Management
FE = Fossil Energy
CRWM = Civilian Radioactive Waste Management
PPA = Policy Planning and Analysis
INT = Intelligence
WFO = Work for Others

\a Fiscal year 1992. 

\b Pacific Northwest Laboratory provided information about all
outputs for the laboratory as a whole that are not included in the
data presented here. 

\c Responses were not collected from Brookhaven National Laboratory
and National Renewable Energy Laboratory. 

\d Research subprogram outputs other than those listed above, such as
technical abstracts,



                          Table V.3
           
            Proportion of Funds Spent by Programs
            With More Than 100 Outputs Related to
               Commercial Product Development\a

R&D-related activity                    ER    CE    DP   WFO
------------------------------------  ----  ----  ----  ----
Basic research                        53.1   4.4   7.2  11.5
Applied research                      15.2  25.9  32.5  26.1
Development                           10.1  28.6  40.9  23.4
Technical assistance to
Government agencies                    4.0   5.9   9.7  25.1
Universities                           6.5   4.0   0.7   2.8
Private firms or industrial            2.8  11.6   0.9   2.2
 organizations
Technology transfer to
Government agencies                    4.1   3.1   2.8   4.3
Private firms or industrial            2.5  15.7   2.5   4.4
 organizations
Other\b                                1.7   0.7   2.8   0.4
============================================================
Total                                  100   100   100   100
------------------------------------------------------------
Legend

ER = Energy Research
CE = Conservation and Renewable Energy
DP = Defense Programs
WFO = Work for Others

\a Fiscal year 1992. 

\b Subprogram expenditures for activities other than those listed
above, such as training graduate students and postdoctoral fellows or
safety procedures. 

   Figure V.1:  Trends in Outputs
   Related to Commercial Product
   Development\a

   (See figure in printed
   edition.)

\a The number of outputs is shown only for research subprograms in
the study population that were in operation all 4 years (fiscal years
1989-92). 


NATIONAL LABORATORIES' POTENTIAL
FOR COMMERCIAL PRODUCT DEVELOPMENT
========================================================== Appendix VI

Data concerning the formation of cooperative research and development
agreements, expenditures for R&D in critical technologies, and the
views of national laboratory program managers on their programs'
potential for commercial product development are presented below. 



                                    Table VI.1
                     
                         CRADAs in Effect at the National
                                  Laboratories\a


                                   All   Any   All   Any   All   Any   All   Any
                                     4     4     4     4     4     4     4     4
                                  year  year  year  year  year  year  year  year
National laboratory\b              s\c   s\d   s\c   s\d   s\c   s\d   s\c   s\d
--------------------------------  ----  ----  ----  ----  ----  ----  ----  ----
Argonne                              7     7     2     2     0     0     0     0
Brookhaven                           4     4     0     0     0     0     0     0
Lawrence Berkeley                    4     4     0     0     0     0     0     0
Oak Ridge                           55    55    38    38    20    20     9     9
Pacific Northwest                    4     9     0     0     0     0     0     0
Idaho                               12    12     9     9     4     4     1     1
Lawrence Livermore                  13    13     3     3     1     1     1     1
Los Alamos                          18    18     8     8     6     6     6     6
Sandia                              13  74\e     4    11     0     0     0     0
================================================================================
Total                              130   196    64    71    31    31    17    17
--------------------------------------------------------------------------------
\a Fiscal years 1989-92. 

\b Responses on CRADA formation were not collected from National
Renewable Energy Laboratory. 

\c These are CRADAs formed for research sponsored by subprograms that
were in operation every year during fiscal years 1989-92. 

\d These are CRADAs formed for research sponsored by subprograms that
were initiated in any year during fiscal years 1989-92. 

\e Most of the CRADAs formed in fiscal year 1992 were sponsored by
the DOE defense program technology transfer initiative at Sandia. 
This subprogram was initiated in June 1990 to identify opportunities
for commercializing technologies produced by DOE-funded defense
research activities. 



                                    Table VI.2
                     
                       CRADAs in Effect Within the National
                        Laboratories by Research Program\a


                                   All   Any   All   Any   All   Any   All   Any
                                     4     4     4     4     4     4     4     4
                                  year  year  year  year  year  year  year  year
Research program\b                 s\c   s\d   s\c   s\d   s\c   s\d   s\c   s\d
--------------------------------  ----  ----  ----  ----  ----  ----  ----  ----
Energy Research                     41    41    13    13     4     4     1     1
Conservation and Renewable          52    52    41    41    26    26    15    15
 Energy
Environment, Safety and Health       0     0     0     0     0     0     0     0
Nuclear Energy                       0     0     1     1     0     0     0     0
Defense Programs                    26  79\e     7    13     1     1     1     1
New Production Reactors              0     0     0     0     0     0     0     0
Environmental Restoration and        1    14     1     2     0     0     0     0
 Waste Management
Fossil Energy                        3     3     0     0     0     0     0     0
Civilian Radioactive Waste           0     0     0     0     0     0     0     0
 Management
Policy Planning and Analysis         0     0     0     0     0     0     0     0
Intelligence                         0     0     0     0     0     0     0     0
Work for Others                      7     7     1     1     0     0     0     0
================================================================================
Total                              130   196    64    71    31    31    17    17
--------------------------------------------------------------------------------
\a Fiscal years 1989-92. 

\b Responses were not collected from National Renewable Energy
Laboratory. 

\c These are CRADAs formed for research sponsored by subprograms that
were in operation every year during fiscal years 1989-92. 

\d These are CRADAs formed for research sponsored by subprograms that
were initiated in any year during fiscal years 1989-92. 

\e Most of the CRADAs formed in fiscal year 1992 were sponsored by
the DOE defense program technology transfer initiative at Sandia. 
This subprogram was initiated in June 1990 to identify opportunities
for commercializing technologies produced by DOE-funded defense
research activities. 



                                    Table VI.3
                     
                      Mean Percent Expenditures for Critical
                        Technologies Within Laboratories\a

Critical                                                                     All
technology\b        ANL    LBL   ORNL    PNL   INEL   LLNL   LANL    SNL    labs
----------------  -----  -----  -----  -----  -----  -----  -----  -----  ------
Material            4.3    5.5   11.8    4.3    4.6    3.6   13.0    1.9     6.2
 synthesis and
 processing
Electronic and      1.1    4.4    1.7    0.2      0    1.9    1.4    2.1     1.7
 photonic
 materials
Ceramics            2.2    2.2    9.8    0.5    0.8    0.7    0.7    0.4     1.8
Composites          0.2    0.4    4.0    0.6    6.9    0.4    0.9    0.2     1.0
High-               3.0    1.5    8.9      0    1.9    1.6    1.2    1.4     2.2
 performance
 metals and
 alloys
Flexible            0.2      0    1.6    0.7    1.3      0    0.6    0.9     0.6
 computer
 integrated
 manufacturing
Intelligent         0.2      0    0.3    1.1    3.7    0.3    1.1    0.8     0.7
 processing
 equipment
Micro-and           0.2    0.4    1.5    0.1    0.5    0.4    1.2    0.7     0.7
 nanofabrication
System                0    0.2    0.6    5.0    0.8    0.8    0.7    1.4     1.2
 management
 technologies
Software            2.2    0.7    0.4    2.4   12.0    6.1    3.5    5.6     4.0
Microelectronics    0.6    1.1    0.5    2.3    0.3    2.1    2.3   11.5     3.7
 and
 optoelectronics
High-               1.9    0.5    3.3    0.2    3.4    5.0    2.0    1.2     2.4
 performance
 computing and
 networking
High-definition     0.4    1.0    0.2    1.1    1.2    0.3    0.5    0.1     0.4
 imaging and
 displays
Sensors and         0.8    0.2    1.5    3.5    7.1    5.1    4.2    9.0     4.6
 signal
 processing
Data storage and    0.1    0.2      0    0.8      0    0.4    1.1    1.9     0.8
 peripherals
Computer            5.7    1.4    5.0    5.4   14.6   12.7    4.5    5.0     6.7
 simulation and
 modeling
Applied             0.2    8.6    4.6    0.7    1.4    2.4    1.6      0     1.9
 molecular
 biology
Medical             0.8    4.7    0.6    0.3    0.1    0.3    0.4      0     0.6
 technology
Aeronautics           0      0    0.3    1.0    0.5      0      0      0     0.2
Surface             1.9      0    1.2      0    2.2      0      0    0.4     0.5
 transportation
 technologies
Energy             32.0   24.2   19.7    7.6   24.8   18.7    4.5    3.5    13.6
 technologies
Pollution          18.8    9.1    1.9   41.3    7.5    3.8    3.8    1.9     8.8
 minimization,
 remediation,
 and waste
 management
Other\c             5.3    0.1    6.6   15.3    4.3   32.9    2.1    0.4    10.2
Research funds     18.0   33.3   14.1    5.6      0    0.5   48.5   49.9    25.9
 not expended on
 critical
 technologies
================================================================================
Total               100    100    100    100    100    100    100    100     100
--------------------------------------------------------------------------------
LEGEND

ANL = Argonne National Laboratory
LBL = Lawrence Berkeley Laboratory
ORNL = Oak Ridge National Laboratory
PNL = Pacific Northwest Laboratory
INEL = Idaho National Engineering Laboratory
LLNL = Lawrence Livermore National Laboratory
LANL = Los Alamos National Laboratory
SNL = Sandia National Laboratories

\a Fiscal year 1992. 

\b Responses were not collected from Brookhaven National and National
Renewable Energy Laboratory. 

\c Subprogram expenditures for activities other than those listed
above, such as robotics, special nuclear materials, environmental



                                    Table VI.4
                     
                      Mean Percent Expenditures for Critical
                          Technologies Within Programs\a

Critic
al                                                                           All
techno            ES&H\                                                   progra
logy\b   ER   CE      c   NE   DP  NPR   ERWM   FE   CRWM  PPA  INT  WFO      ms
------  ---  ---  -----  ---  ---  ---  -----  ---  -----  ---  ---  ---  ------
Materi  7.0  9.6      0  6.1  7.3    0    1.6  8.5    4.4    0    0  3.1     6.2
 al
 synth
 esis
 and
 proce
 ssing
Electr  3.1  0.6      0  0.7  1.9    0      0  5.3      0    0    0  0.5     1.7
 onic
 and
 photo
 nic
 mater
 ials
Cerami  3.5  12.      0  2.4  0.5  3.1    0.1  9.8      0    0    0  0.6     1.8
 cs            5
Compos  2.3  2.7      0  0.4  0.3    0    0.5  3.9      0    0    0  1.7     1.0
 ites
High-   4.0  2.5      0  6.2  1.2  4.6    0.2  3.8    1.8    0    0  1.4     2.2
 perfo
 rmanc
 e
 metal
 s and
 alloy
 s
Flexib  0.4    0      0  0.3  0.6    0      0    0      0    0    0  1.8     0.6
 le
 compu
 ter
 integ
 rated
 manuf
 actur
 ing
Intell  0.5  0.3      0  0.3  0.8    0    0.1    0      0    0    0  1.6     0.7
 igent
 proce
 ssing
 equip
 ment
Micro-  0.6  0.4      0    0  1.0    0      0  0.6      0    0    0  0.7     0.7
 and
 nanof
 abric
 ation
System  0.1  0.3      0  0.7  0.9    0    3.4    0    7.1    0    0  2.9     1.2
 manag
 ement
 techn
 ologi
 es
Softwa  2.8  0.9      0  2.3  5.3    0    0.3  7.6    5.6  5.0    0  5.6     4.0
 re
Microe  1.0  0.6      0  0.7  6.7    0      0    0      0    0    0  2.4     3.7
 lectr
 onics
 and
 optoe
 lectr
 onics
High-   7.1    0      0  2.0  1.2    0      0    0    0.3    0    0  1.6     2.4
 perfo
 rmanc
 e
 compu
 ting
 and
 netwo
 rking
High-   0.4  0.1      0  0.3  0.4  4.6      0    0      0    0    0  1.0     0.4
 defin
 ition
 imagi
 ng
 and
 displ
 ays
Sensor  0.9  0.7    5.0  3.6  7.4    0    2.9  2.4      0    0  10.  3.3     4.6
 s and                                                            0
 signa
 l
 proce
 ssing
Data    0.7    0      0    0  1.2    0    1.1    0      0    0    0    0     0.8
 stora
 ge
 and
 perip
 heral
 s
Comput  6.5  2.1   10.0  4.0  7.4  4.6    3.9  8.5   12.2  5.0    0  8.4     6.7
 er
 simul
 ation
 and
 model
 ing
Applie  8.8  0.3      0    0  0.1    0    0.6    0      0    0    0    0     1.9
 d
 molec
 ular
 biolo
 gy
Medica  2.2    0      0    0    0    0      0    0      0    0    0  1.0     0.6
 l
 techn
 ology
Aerona    0    0      0    0    0    0      0    0      0    0    0  1.3     0.2
 utics
Surfac    0  4.2      0    0  0.2    0    1.2    0      0    0    0  1.0     0.5
 e
 trans
 porta
 tion
 techn
 ologi
 es
Energy  12.  39.      0  68.  3.6  83.    1.0  41.   25.8  90.    0  12.    13.6
 techn    6    7           6         1           3           0         5
 ologi
 es
Pollut  2.3  1.7   10.0  1.3  2.5    0   77.6  0.3    9.5    0    0  11.     8.8
 ion                                                                   1
 minim
 izati
 on,
 remed
 iatio
 n,
 and
 waste
 manag
 ement
Other\  10.  0.4      0    0  14.    0    0.5  8.1    3.6    0  90.  9.3    10.2
 d        5                     5                                 0
Resear  22.  20.   75.0    0  34.    0    4.8    0   29.6    0    0  27.    25.9
 ch       7    3                9                                      1
 funds
 not
 expen
 ded
 on
 criti
 cal
 techn
 ologi
 es
================================================================================
Total   100  100    100  100  100  100    100  100    100  100  100  100     100
--------------------------------------------------------------------------------
Legend

ER = Energy Research
CE = Conservation and Renewable Energy
ES&H = Environment, Safety and Health
NE = Nuclear Energy
DP = Defense Programs
NPR = New Production Reactors
ERWM = Environmental Restoration and Waste Management
FE = Fossil Energy
CRWM = Civilian Radioactive Waste Management
PPA = Policy Planning and Analysis
INT = Intelligence
WFO = Work for Others

\a Fiscal year 1992. 

\b Responses were not collected from Brookhaven National and National
Renewable Energy Laboratory. 

\c Only one laboratory provided complete information for the ES&H
program.  ES&H activity at this laboratory provides support for
environmental protection at DOE sites in several areas, ensuring safe
facility management practices, developing and recommending radiation
and chemical protection policies and practices, and evaluating the
health of DOE personnel and the public.  R&D to improve dosimetry and
measurement techniques also is undertaken. 

\d Subprogram expenditures for activities other than those listed
above, such as robotics, special nuclear materials, environmental
R&D, and detector technology. 

   Figure VI.1:  Program Managers'
   Views of Programs' Commercial
   Product Potential\a

   (See figure in printed
   edition.)

\a Total exceeds 100 owing to rounding. 




(See figure in printed edition.)Appendix VII
COMMENTS FROM THE DEPARTMENT OF
ENERGY
========================================================== Appendix VI

See comment 1. 

See comment 2. 



(See figure in printed edition.)

See comment 3. 

See comment 4. 

See comment 5. 

See comment 6. 

See comment 7. 

See comment 8. 

See comment 9. 



(See figure in printed edition.)


The following are GAO's comments on the September 14, 1994, letter
from DOE. 

GAO COMMENTS

1.  The definitions for basic research, applied research, and
development that our study employs are derived from a Congressional
Budget Office study of the federal R&D enterprise.  The definition of
technology transfer is the one used in our study of the Technology
Transfer Act of 1986 and the definition of technical assistance is
based on expert opinion.  Our analysis examined the laboratories'
effort in each type of activity separately, and grouped, in two major
categories, in order to address the study objective:  to provide an
empirical base for examining the extent to which the laboratories are
engaged in basic and applied research or research related to
commercial product development.  Figure 1 and tables IV.1-IV.4 allow
the reader to view our findings in both the two major categories and
as separate R&D-related activities.  The finding for each major
category presented in figure 1 is the sum of the findings for the
corresponding separate R&D-related activities presented in the last
column of table IV.1. 

DOE disagrees with the category we established for "research related
to commercial product development"--that is, that development,
technical assistance, and technology transfer are all laboratory
activities related to commercial product development--but does not
question our definitions or findings for each separate activity.  We
agree that DOE may decline to accept our definition for research
related to commercial product development, but we do not agree that
our finding for the sum of the three separate activities is
erroneous.  This finding is based on laboratory research managers'
estimates of the distribution of their subprograms' expenditures that
were collected, verified, and analyzed according to generally
accepted government auditing standards.  We consider these estimates,
made by research managers who are closely involved with the R&D, more
accurate than estimates that may be obtained by other methods. 

2.  The analyses we produced were intended to establish baseline data
for addressing empirical questions underlying the public debate,
rather than to serve as a comprehensive analysis of the laboratories'
roles.  To address the study objective, we focused on the 10
laboratories as a set of institutions, on comparing the distribution
of expenditures for five types of R&D-related activities both within
and among the 10 laboratories, on the nature and scope of their
outputs, and on their potential for working with industry to bring
commercial products to market.  Given this approach, with the
exception of expenditures for critical technologies and collaboration
with industry, which we do examine, the other factors DOE suggests
for analysis are beyond the scope of this study.  However, we
anticipate that our study might stimulate another party to undertake
the type of institutional, comparative analysis that DOE suggests. 

3.  We agree with DOE that the laboratories collaborate in R&D with
industry partners who then perform the additional testing and
research activities required for commercial application.  We also
agree that "it typically should take years from the conclusion of a
CRADA and the transfer of a technology to a partner, to the
commercialization of a product." The explicit definitions of terms
and the discussion of CRADAs in the report make this clear.  (See pp. 
6, 12, and 15.) However, we disagree that the report attributes
commercial product development work to the national laboratories. 

4.  We state in the section on Methodology that we began our work
with a survey of the laboratories' R&D activities because we could
find no sufficiently comprehensive (emphasis added) existing
documentation.  To confirm that we had not overlooked an important
information source when we designed and implemented our data
collection strategy, we made inquiries about DOE's institutional plan
and research and development databases.  We found that DOE
headquarters maintains only the institutional plan database and that
it includes only one of the data items, research program budget, that
we used in our report.  This budget information was available for
fiscal years 1989-91 when we implemented our survey but would not
have been useful for our analyses because it is not compiled at the
same level of detail as our data. 

We also found that the research and development database is not one
of DOE's databases.  It is being developed by the Critical
Technologies Institute for the Office of Science and Technology
Policy (OSTP) in the Executive Office of the President.  When it is
complete, it will have five data items analogous to our data. 
However, this database was not available when we developed our
national laboratory inventory and is not now available to users other
than OSTP.  Forty-one of the items in our report are not included in
either the institutional plan or research and development databases. 

The Laboratory Management Division in DOE's Office of Energy Research
maintains the institutional plan database.  It has research program
budget data for fiscal years 1979 to the present at the program level
for 9 of the 10 national laboratories, and it has subprogram budget
data for selected programs, such as energy research, defense
programs, civilian radioactive waste management, and work for others. 
Because they are incomplete at the subprogram level, these data would
not have been useful for our R&D-related activities and critical
technologies analyses, which required budget data for all subprograms
in our sample.  Further, none of the budget data for the National
Renewable Energy Laboratory are included in the institutional plan
database.  These data must be obtained from NREL's hardcopy
institutional plan, which is available from the Office of Energy
Efficiency and Renewable Energy at headquarters. 

The Critical Technologies Institute's research and development
database will have information on laboratory expenditures for basic
research, applied research, development, and technology transfer for
research subprogram categories analogous, but not identical, to those
we used, and on CRADAs--for the national laboratories as well as for
the laboratories of several other federal agencies--when it is
available to organizations other than OSTP.  The Critical
Technologies Institute representative to whom we spoke could not
specify when the database will be available.  However, the research
and development database will not have information comparable to the
16 research subprogram outputs we collected from the laboratories nor
on the proportion of subprogram expenditures for the 22 critical
technologies and the proportion of expenditures for technical
assistance. 

We are also aware that abstracts of CRADA agreements can be obtained
through DOE headquarters from the Office of Scientific and Technical
Information, which is based in Oak Ridge, Tennessee.  However, we
also found these data to be incomplete.  In August 1992, we requested
these data through DOE's Office of Technology Utilization at
headquarters and received 147 abstracts for the nine laboratories
from which we collected CRADA information--49 fewer than the total
the laboratories reported to us.  Since the fiscal year was not then
complete, we assumed that all CRADA information had not yet been
reported to DOE or entered into the database. 

Our experience developing the survey frame, moreover, suggested that
the laboratories' institutional plan data needed modification to
address our study requirements and that the information available
from DOE was not consistent with information available from the
laboratories.  We used the list of research programs included in the
institutional plans as a preliminary frame for part I of the survey. 
Recognizing that the laboratories are dynamic institutions, we asked
each laboratory to confirm the list before survey implementation. 
Most of the laboratories made both deletions and additions to the
list to meet our survey selection criteria.  (See p.  4.) We used the
lists of facilities reported in the DOE report, Capsule Review of DOE
Research and Development Laboratories and Field Facilities, as a
preliminary frame for part II of the survey.  The laboratories made
deletions and additions to these lists as well and in two cases
almost completely replaced them.  Changes of this magnitude confirmed
the strategy of collecting data directly from the laboratories to
address our study's information requirements. 

5.  During the agency review of our draft report, two laboratories
provided us with additional CRADA information, bringing the total
number of CRADAs in effect among all programs in operation in any
year from fiscal year 1989 to 1992 to 196.  (See table VI.1.) This
total is the number of CRADAs in effect in fiscal year 1992, rather
than "now," to which DOE refers and which we assume is fiscal year
1994.  Moreover, we found a substantial increase in CRADA formation
in fiscal year 1992, sponsored by DOE's defense program technology
transfer initiative at Sandia.  (See tables VI.1 and VI.2.) It is
possible that the increase we found persisted and included more
laboratories, bringing the total to 1,000 in fiscal year 1994. 
However, such a change would not render our finding for fiscal year
1992 inaccurate. 

6.  Brookhaven brought it to our attention that the number of CRADAs
formed is limited by the amount of money allocated to a laboratory
and that this amount varies widely from laboratory to laboratory.  We
agree with Brookhaven that characterizing CRADA formation as the
"strongest" indicator of a laboratory's commercial product potential
is misleading for this reason, and we have modified our discussion of
CRADA findings. 

Scientific user facilities and personnel exchanges will be examined
in a separate study.  Licensing is described in the section on
Principal Findings of this report.  (See pp.  13-14.) CRADAs are
cost-shared cooperative agreements targeted to a commercial
innovation. 

7.  We treat laboratory outputs as measures of activity, not as
measures of impact or productivity.  (See pp.  7-8.)

8.  We found that the 10 laboratories produced many more publications
and reports (21,593) than they did outputs related to commercial
product development (2,510) in fiscal year 1992.  This is a statement
of fact, tabulated from reports to us by the laboratories' research
managers.  It describes the laboratories' activity.  It is not
intended as a criticism of the research enterprise. 

9.  The purpose of this report was to examine the balance of
R&D-related activity across the laboratories, rather than to examine
the magnitude of the R&D investment.  We used the proportion of funds
expended for each type of R&D-related activity as a measure of
activity, not as a measure of investment.  (See pp.  6-7.) An
examination of human resources and a comparison of DOE's national
laboratories to those of other agencies was beyond the scope of this
study, given its focus on laboratory R&D-related activity. 


COMMENTS FROM ARGONNE NATIONAL
LABORATORY
======================================================== Appendix VIII

A representative of Argonne, Internal Audit, called us on July 5,
1994, to report that the laboratory had no substantive comments on
the report draft. 




(See figure in printed edition.)Appendix IX
COMMENTS FROM BROOKHAVEN NATIONAL
LABORATORY
======================================================== Appendix VIII

Now p.  4.
See comment 1. 

Now p.  7.
See comment 2. 

Now table IV.1, p.  50.
See comment 3. 



(See figure in printed edition.)

Now p.  14.
See comment 4. 

See comment 5. 

Now p.  28. 


The following are GAO's comments on the June 27, 1994, letter from
Brookhaven National Laboratory. 

GAO COMMENTS

1.  We have added a statement to the report clarifying this
difference. 

2.  We have evaluated the data Brookhaven submitted and, after making
the appropriate changes, added it to the database.  These data have
been incorporated into the tables included in the report letter and
appendixes. 

3.  We agree with Brookhaven's evaluation of this response and have
made the change they requested to the database and report tables. 

4.  We agree with Brookhaven and have modified the discussion of
CRADA findings. 

5.  The information on CRADA formation Brookhaven submitted in the
pilot version of the data collection instrument has been added to the
database and the tables in appendix VI.  We also have added the
sentence Brookhaven suggests to appendix I. 




(See figure in printed edition.)Appendix X
COMMENTS FROM DOE'S IDAHO
OPERATIONS OFFICE
======================================================== Appendix VIII

See comment 1. 

See comment 2. 


The following are GAO's comments on the June 29, 1994, memorandum
from DOE's Idaho Operations Office. 

GAO COMMENTS


      GENERAL COMMENTS
---------------------------------------------------- Appendix VIII:0.1

DOE's Idaho Operations Office representative, who responded for
Idaho, observed that the value in the "R&D Budget" column of table 2
for Idaho should be $275 million, rather than $98.7 million and that
the Idaho Operations Office made this determination by applying DOE
headquarters' definitions for research programs to Idaho's research
programs.  The list of Idaho research programs to which the Idaho
Operations Office applied DOE headquarters' definitions is
unspecified.  We disagree with this determination, because it
violated the study methodology. 


      SPECIFIC COMMENTS
---------------------------------------------------- Appendix VIII:0.2

1.  The R&D budgets of the 10 national laboratories in table 2 were
not compared. 

2.  We coordinated data collection from the laboratories with DOE's
operations office representatives, but none of them participated in
any of the technical activities involving survey implementation. 
Therefore, the Idaho Operations Office representative may not have
been aware that GAO program selection criteria should have been
employed to assess the "R&D Budget" column value for Idaho in table 2
to be consistent with the methodology employed for the other nine
laboratories.  The use of DOE headquarters' definitions for research
programs to make this determination would result in a list of
subprograms that differs substantially from the one jointly developed
by GAO and Idaho. 

Subprograms included in the survey population were identified by
laboratory representatives who applied the selection criteria we
specified (see p.  4) to a preliminary subprogram list we compiled
from the institutional plans and sent to the laboratories.  This
approach was followed by Idaho's representatives, who identified 10
subprograms.  We reduced the number of Idaho subprograms to nine
during the editing and coding process.  The $98.7-million value in
the "R&D Budget" column is the total of nine research subprogram
budgets reported by Idaho program managers on part I of the national
laboratory inventory data collection instrument. 




(See figure in printed edition.)Appendix XI
COMMENTS FROM LAWRENCE BERKELEY
LABORATORY
======================================================== Appendix VIII

See comment 1. 

See comment 2. 

See comment 3. 



(See figure in printed edition.)


The following are GAO's comments on the June 17, 1994, letter from
Lawrence Berkeley Laboratory. 

GAO COMMENTS

Although Lawrence Berkeley agreed with the study's analytic framework
and with the need for studies of this type to inform congressional
policymakers, the laboratory raised an issue about the relationship
of the national laboratories' role in commercial product development
to the broader needs of industry or the nation, which was not
addressed in the report.  This omission warrants clarification. 


      GENERAL COMMENTS
---------------------------------------------------- Appendix VIII:0.3

The relationship of the national laboratories' role in commercial
product development to the broader needs of industry is an issue
being discussed in the public debate about the laboratories' missions
and structure, but one that falls outside of the study scope.  The
purpose of this study was to examine the extent to which the national
laboratories are engaged in basic and applied research or research
related to commercial product development.  Scientific and technical
infrastructure, which Lawrence Berkeley gives as an example of
industry need, while important to the considerations of laboratory
mission and structure that serve as the study's policy context, was
not addressed.  It was our expectation that the findings of this
study would serve as an empirical base for designing a study to
address this and other institutional issues. 


      SPECIFIC COMMENTS
---------------------------------------------------- Appendix VIII:0.4

1.  The statement "interest in the current balance of the research
effort in the 10 laboratories' research programs .  .  .  .  (and)
the extent to which the national laboratories are engaged now in
basic and applied research or in research related to commercial
product development.  .  .  ." implies that commercial product
development is an indirect consequence of laboratory R&D-related
activities, rather than a subset of the broader research and
development role of the national laboratories.  That is the meaning
of the phrase "research related to (emphasis added) commercial
product development," and the use of the phrase "outputs related to
(emphasis added) commercial product development" elsewhere in the
report.  The explicit definitions and discussions of CRADAs in the
report make it clear that the laboratories' involvement in commercial
product development is limited to collaboration with industry
partners in R&D-related activities that produce innovations with
market potential and that move these technologies beyond the
laboratories' walls.  These definitions assume that the industry
partner performs the subsequent research, testing, and marketing
activity that accomplish commercial application.  (See pp.  6, 12,
and 15.)

2.  This study was not designed as a broad assessment of the national
laboratories' roles, but to examine the balance of the laboratories'
R&D-related activities in two major areas:  basic and applied
research and research related to commercial product development.  We
looked at these activities with three types of measures, and our
conclusions interpret our findings for each type.  The conclusion
focuses on research related to commercial product development because
we found slightly more activity in this area.  We amplified this
conclusion with an interpretation of findings for the other two types
of measures.  A discussion of the noncommercial product output of
nuclear weapons research was not relevant. 

3.  We have added a discussion of these limitations to the
Methodology section.  (See pp.  7-8.)




(See figure in printed edition.)Appendix XII
COMMENTS FROM LAWRENCE LIVERMORE
NATIONAL LABORATORY
======================================================== Appendix VIII

Now page 26. 


The following are GAO's comments on the July 6, 1994, letter from
Lawrence Livermore National Laboratory. 

GAO COMMENTS

We have added the revised text describing Lawrence Livermore
Laboratory to appendix I. 




(See figure in printed edition.)Appendix XIII
COMMENTS FROM LOS ALAMOS NATIONAL
LABORATORY
======================================================== Appendix VIII


The following are GAO's comments on the June 23, 1994, letter from
Los Alamos National Laboratory. 

GAO COMMENTS

We have added a footnote to the Background section discussing the
legal division of Department of Defense and civilian responsibility
for nuclear weapons research and development.  We also expanded the
phrase on page 2 from "weapons development" to "nuclear weapons
research and development."




(See figure in printed edition.)Appendix XIV
COMMENTS FROM NATIONAL RENEWABLE
ENERGY LABORATORY
======================================================== Appendix VIII

See comment 1. 

Now p.  2.
See comment 2. 

Now p.  8.
See comment 3. 

Now p.  10.
See comment 4. 

Now p.  15.
See comment 5. 

Now p.  28.
See comment 6. 


The following are GAO's comments on the June 22, 1994, letter from
National Renewable Energy Laboratory. 

GAO COMMENTS

1.  One future report will provide a descriptive statistical analysis
of the technical and operating characteristics of the national
laboratories' major research facilities.  Other topics are yet to be
determined. 

2.  We have made this correction to the text. 

3.  Graphs and tables are presented in the section on Principal
Findings. 

4.  The aggregation in figure 1 is intentional.  The graph is
designed to illustrate the balance between the two major areas of
R&D-related activity we examined.  The last column of table IV.1,
labeled "All Labs," presents percentages for development, technical
assistance, and technology transfer for the 10 laboratories. 

5.  See table VI.3 in appendix VI.  Table VI.4 presents these
percentages by program area. 

6.  We have made this correction to the text. 




(See figure in printed edition.)Appendix XV
COMMENTS FROM OAK RIDGE NATIONAL
LABORATORY
======================================================== Appendix VIII


We did not receive Oak Ridge's written comments from DOE.  We did
discuss Oak Ridge's views with laboratory representatives by
telephone on June 22 and July 13 and 19, 1994, and we spoke with a
representative of DOE's Oak Ridge Operations Office on July 7, 1994. 
We also received new output data for Oak Ridge's subprograms by
facsimile from representatives of both organizations.  A summary of
their comments and our response follows. 

GAO COMMENTS

Oak Ridge raised two general issues.  One was the effect of the study
sampling methodology on findings for the laboratory's outputs related
to commercial product development.  Oak Ridge took the position that
most of the laboratory's outputs related to commercial product
development were produced by subprograms not selected in the study
sample and, consequently, expressed the concern that GAO's findings
for outputs related to commercial product development based on the
sampled subprograms may not be representative because of this
distribution of outputs among all laboratory subprograms.  Most of
these outputs, they explained, are produced by programs that fall
below the $10-million threshold for inclusion in the survey.  In
fact, according to tabulations they had performed, the sampled
programs, while representing 73 percent of the overall budget,
produce only 7 percent of the outputs in question. 

Secondly, Oak Ridge thought that the report's definitions and
analyses equate development work with commercial product development
and that the conclusion based on this definition is not supported by
the data.  We address these issues separately. 

First, Oak Ridge actually had identified two sources of potential
underreporting:  (1) data for outputs of sampled subprograms that
were not available at the time of the survey and (2) data for outputs
of unsampled programs.  We agreed that additional data for sampled
subprograms should be added to findings for Oak Ridge.  We requested
and received from Oak Ridge the new data for the sampled subprograms,
and we added them to our database and report tables. 

We did not add to our database and report tables the summary data Oak
Ridge tabulated as total outputs (including unsampled subprograms)
for the entire laboratory.  To have incorporated these data would
have violated the sampling methodology.  Moreover, without more
detailed information at the subprogram level, we could not judge to
what extent these totals represented outputs of research and
development programs.  This was a matter of some concern to us,
particularly in light of the large number of outputs Oak Ridge
ascribed to the unsampled programs. 

Second, we disagreed that the report definitions and analyses equate
development work with commercial product development.  Our
definitions, analyses, and conclusions make it clear that the
laboratories were not expected to produce commercial products.  We
defined development as having "some type of product as the output
goal (emphasis added)," but concluding "with a prototype rather than
a usable good." Further, we point out that "Additional time,
research, and testing are required to convert the prototype to a
weapon or commercially viable product." The definitions of outputs
related to commercial product development, including those for
precompetitive commercial products and processes, state that these
outputs tend (emphasis added) to arise from development work, but
that "they will require a substantial additional investment before
they are ready to market." The conclusion, moreover, reiterates that
these outputs are "precursors to marketable goods," and that, for
this reason, "it is too early to determine whether this activity will
produce technologies with commercial uses."

We also examined the assumption that R&D is a linear process, with
all commercial product-related outputs arising from development, and
found that our data did not support it.  We included this segment of
the analysis to emphasize the uncertainty associated with current
understanding of the operation of the R&D process, and the origin of
technologies with commercial potential.  The conclusion we reached
concerning the uncertain prospects of the laboratories' commercial
product-related outputs is an interpretation of this finding as well
as our definitions for outputs related to commercial product
development. 


COMMENTS FROM PACIFIC NORTHWEST
LABORATORY
========================================================= Appendix XVI

A representative of Pacific Northwest, called us on June 22, 1994, to
comment on the draft report by telephone.  A summary of the
laboratory's comments is included in our response, which follows. 

GAO COMMENTS

Pacific Northwest offered one general comment and several comments
and questions about specific items in the text.  We address the
general comment first and then the specific comments. 


      GENERAL COMMENT
----------------------------------------------------- Appendix XVI:0.1

Pacific Northwest suggested that a section be added to the report
describing the major commercial product-related initiatives the
national laboratories have undertaken since the end of fiscal year
1992.  Partnership for a New Generation of Vehicles (PNGV) and
American Textile Partnership (AMTEX), two consortia for R&D targeted
on commercial applications in which several laboratories are
participating, were mentioned as examples. 

We are aware that the laboratories have been active in technology
transfer activities of many types since the end of fiscal year 1992. 
This activity will be captured in any follow-up study that is
performed in the next few years to determine if progress has been
made since fiscal years 1989-92, the time period measured in this
report. 


      SPECIFIC COMMENTS
----------------------------------------------------- Appendix XVI:0.2

1.  Pacific Northwest thought that the word "primarily" in the
sentence beginning on draft line 10, page 4 (now line 12, p.  3),
should be deleted because it implies that the laboratories have only
one primary mission.  We have modified this sentence. 

2.  Pacific Northwest said that the output data in table 5 (now table
V.1) not reported for the laboratory are available and will be
submitted to us.  We received and reviewed the data, and we added it
to table V.1. 

3.  Pacific Northwest said that information on CRADA formation for
the laboratory as a whole was submitted to us during survey
implementation.  We confirmed that this information had been received
and added it to table VI.1. 




(See figure in printed edition.)Appendix XVII
COMMENTS FROM SANDIA NATIONAL
LABORATORIES
========================================================= Appendix XVI


The following are GAO's comments on the July 1, 1994, letter from
Sandia National Laboratories. 

GAO COMMENTS

Sandia agreed with the report's objective, methodology, and
conclusion, but made two general comments.  First, Sandia suggested
that the report include a description of the national laboratories'
expanded efforts in technology transfer during fiscal years 1993-94. 
Second, Sandia suggested that we review the substantial variation in
the percentage of laboratory funds not expended for critical
technologies reported for Lawrence Livermore, Los Alamos, and Sandia
in table VI.3.  Sandia expected this percentage to be very similar
for all three laboratories. 

We are aware that the national laboratories have been active in
technology transfer activities of many types during fiscal years
1993-94, including participation in large-scale R&D consortia such as
PNGV and AMTEX.  These activities will be captured in any follow-up
study that is performed during the next few years to determine if
progress has been made since fiscal years 1989-92, the time period
measured in this report. 

We reviewed all responses by Lawrence Livermore, Los Alamos, and
Sandia concerning percent of expenditures for critical technologies
and funds not expended for R&D in these areas.  We found that
Lawrence Livermore program managers allocated a percentage of funds
expended to the "other" category to a much greater extent than did
program managers at Sandia or Los Alamos.  We also found considerable
variation among all laboratories in the proportion of expenditures
allocated to this category.  R&D activities specified in the "other"
category included items such as robotics, special nuclear materials,
environmental R&D, and detector technology.  Allocations to this
category, and to the energy technologies category, accounted for most
of the difference in proportion of funds not expended for critical
technologies by Lawrence Livermore, Los Alamos, and Sandia. 


MAJOR CONTRIBUTORS TO THIS REPORT
======================================================= Appendix XVIII

PROGRAM EVALUATION AND METHODOLOGY
DIVISION

Robert E.  White, Assistant Director
Sara E.  Edmondson, Project Manager
Dale W.  Harrison, Computer Analyst
Venkareddy Chennareddy, Referencer
Richard R.  Scott, Project Adviser
Gerald L.  Dillingham, Project Adviser
Eric M.  Larson, Project Adviser
Nancy A.  Briggs, Project Adviser

DENVER REGIONAL OFFICE

Miguel A.  Lujan, Project Adviser

