Federally Funded R&D Centers: Issues Relating to the Management of
DOD-Sponsored Centers (Letter Report, 08/06/96, GAO/NSIAD-96-112).

GAO reviewed the Department of Defense's (DOD) management and use of
federally funded research and development centers (FFRDC), focusing on
issues that merit closer attention as Congress and DOD seek to resolve
certain concerns.

GAO found that; (1) overly broad FFRDC mission statements may allow
FFRDC to diversify into areas beyond their proper scope or to perform
inherently governmental functions; (2) DOD has tried to tighten FFRDC
roles, but it is still difficult to determine if given tasks meet FFRDC
criteria; (3) DOD controls to ensure FFRDC objectivity are sparse and
inadequate, since it only requires an approved conflict-of-interest
policy for FFRDC trustees, rather than establishing departmentwide
criteria; (4) DOD generally relies on FFRDC sponsors to prevent
conflicts of interest, but some FFRDC sponsors' internal controls are
not as rigorous as other sponsors' controls; (5) DOD has not exerted
strong control over FFRDC parent organizations' non-FFRDC work, but
recent DOD management plan revisions incorporate criteria to limit
parent corporations' diversification and control the non-FFRDC work they
perform; (6) DOD policy guidance and oversight of FFRDC have been
inadequate regarding justification of noncompetitive contract awards for
FFRDC operation, audit controls, contract fees, and employee
compensation; (7) a panel has been established to review and advise DOD
on the management of FFRDC; (8) DOD does not believe that using civil
servants or contractors would be effective alternatives to FFRDC; and
(9) it is too soon to assess DOD initiatives to address concerns about
FFRDC.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  NSIAD-96-112
     TITLE:  Federally Funded R&D Centers: Issues Relating to the 
             Management of DOD-Sponsored Centers
      DATE:  08/06/96
   SUBJECT:  Laboratories
             Research and development contracts
             Conflict of interest
             Cost effectiveness analysis
             Internal controls
             Competitive procurement
             Defense contracts
             Privatization
             Department of Defense contractors
             Research and development facilities

             
******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************


Cover
================================================================ COVER


Report to Congressional Committees

August 1996

FEDERALLY FUNDED R&D CENTERS -
ISSUES RELATING TO THE MANAGEMENT
OF DOD-SPONSORED CENTERS

GAO/NSIAD-96-112

Federally Funded R&D Centers

(705084)


Abbreviations
=============================================================== ABBREV

  AFMC - Air Force Materiel Command
  C\3 I - command, control, communication, and intelligence
  DCAA - Defense Contract Audit Agency
  DOD - Department of Defense
  DSB - Defense Science Board
  FFRDC - federally funded research and development centers
  IDA - Institute for Defense Analyses
  MTS - members of technical staff
  NASA - National Aeronautics and Space Administration
  OFPP - Office of Federal Procurement Policy
  OMB - Office of Management and Budget
  OSD - Office of the Secretary of Defense

Letter
=============================================================== LETTER


B-259033

August 6, 1996

The Honorable Curt Weldon
Chairman
The Honorable John M.  Spratt, Jr.
Ranking Minority Member
Subcommittee on Military Research
 and Development
Committee on National Security
House of Representatives

This report identifies issues that we believe merit closer attention
as Congress and the Department of Defense (DOD) seek to resolve
concerns relating to the management and use of DOD's federally funded
research and development centers (FFRDC).  We also discuss
information from our March 1996 testimony on recent DOD actions to
improve the management of FFRDCs.\1


--------------------
\1 Federally Funded R&D Centers:  Observations on DOD Actions to
Improve Management (GAO/T-NSIAD-96-117, Mar.  5, 1996). 


   BACKGROUND
------------------------------------------------------------ Letter :1

FFRDCs were first established during World War II to meet specialized
or unique research and development needs that could not be readily
satisfied by government personnel (due to limits on federal salaries
and hiring) or commercial contractors.  Additional and expanded
requirements for specialized services led to increases not only in
the size but also in the number of FFRDCs, which peaked at 74 in
1969.  Today, 8 agencies, including DOD, fund 39 FFRDCs that are
operated by universities, nonprofit organizations, or industrial
firms under long-term contracts.  Provisions of the Competition in
Contracting Act\2 authorize agencies to award these contracts
noncompetitively.  The Office of Federal Procurement Policy (OFPP)
within the Office of Management and Budget (OMB) establishes
governmentwide policy on the use and management of FFRDCs. 

The Director of Defense Research and Engineering is responsible for
developing overall policy for DOD's 11 FFRDCs.  The Director also
determines the funding level for each FFRDC based on the overall
congressional ceiling on FFRDC funding and FFRDC sponsors' funding
requirements.  Planned fiscal year 1996 funding for DOD's FFRDCs is
about $1.2 billion.  DOD categorizes each of its FFRDCs as either a
systems engineering and integration center, a studies and analyses
center, or a research and development laboratory. 

The military services and defense agencies sponsor individual centers
and award and administer 5-year contracts after reviewing the
continued need for the FFRDCs.  Unlike a commercial contractor, an
FFRDC accepts restrictions on its ability to manufacture products and
compete for other government or commercial business.  These
restrictions are intended to (1) limit the potential for conflicts of
interest when FFRDC staff have access to sensitive government or
contractor data and (2) allow the DOD sponsor to form and maintain a
special or strategic relationship with its FFRDC. 

As DOD downsizing continues, attention has turned to the
infrastructure supporting DOD's research and technology programs. 
DOD has, for example, initiated efforts to consolidate and downsize
military-operated laboratories to eliminate redundant facilities.  In
line with heightened attention accorded DOD's technology
infrastructure, Congress has renewed its long-standing concern over
DOD's use of FFRDCs and undertaken several initiatives to control the
centers more tightly.  The Department of Defense Appropriations Act,
1995, Public Law 103-335, prohibited the creation of new FFRDCs,
imposed more specific restrictions on their operations, and limited
the total funding DOD provides such centers.  For example, overall
funding for DOD's FFRDCs, in constant 1995 dollars, increased by
about 23 percent, from almost $1.4 billion in fiscal year 1985 to a
peak of approximately $1.7 billion in fiscal year 1990, after which
Congress began reducing DOD's FFRDC funding.  Since fiscal year 1990,
funding for DOD's FFRDCs has decreased by almost 26 percent to about
$1.3 billion in fiscal year 1995.  Figure 1 shows the obligations of
DOD's FFRDCs, in constant 1995 dollars, from fiscal years 1985 to
1995. 

   Figure 1:  Obligations for
   DOD's FFRDCs From Fiscal Years
   1985-95 (in constant 1995
   dollars)

   (See figure in printed
   edition.)

Source:  DOD. 


--------------------
\2 See 10 U.S.C.  2304 (b)(1)(C) and (c)(3)(B). 


   RESULTS IN BRIEF
------------------------------------------------------------ Letter :2

On the basis of our work on FFRDCs, as well as studies done by
others, we have identified four issues we believe merit attention as
Congress and DOD work to resolve concerns relating to FFRDCs.  These
issues are whether DOD (1) limits its FFRDCs to performing
appropriate work, (2) adequately safeguards the objectivity of its
FFRDCs, (3) oversees its FFRDCs effectively, and (4) adequately
considers cost-effective alternatives to using FFRDCs. 

First, the DOD Inspector General's office and others have raised
concerns that FFRDC mission statements are too broad and do not
clearly identify the specialized tasks that FFRDCs should perform. 
An overly broad charter may allow FFRDCs to diversify into areas
outside their proper scope.  For example, an FFRDC might assume work
that commercial contractors could perform as effectively under
competitive contracts.  On the other hand, there is some debate as to
whether FFRDCs may be performing inherently governmental functions
that should be performed by civil servants.  DOD now defines the role
of FFRDCs as performing work that is consistent with the FFRDC's
purpose, mission, capabilities, and core competencies and requires a
special relationship between the FFRDC and its sponsor. 

While DOD states that it is important to ensure that tasks assigned
to the FFRDC meet the core work criteria, we believe it will continue
to be difficult to determine whether a task meets these criteria.  We
found that FFRDC mission statements remain broad, and FFRDCs'
recently identified core competencies differ little from the previous
scope of work descriptions.  As we stated in our 1988 report, the
need for the special relationship is the key to determining whether a
task is appropriate for an FFRDC.\3 However, determining whether one
or more of the characteristics of the special relationship is
required for a task may be difficult, since determining the need for
an element of the special relationship requires potentially difficult
and subjective judgments. 

Second, questions have been raised by Congress and others about
whether DOD efforts to safeguard the objectivity\4 of its FFRDCs are
adequate.  Specifically, these questions have been raised because DOD
(1) has issued little specific FFRDC-wide guidance on safeguarding
objectivity or regulating the outside interests of Board of Trustee
members, relying instead on FFRDC sponsors to design appropriate
internal controls and (2) has not historically exerted strong control
over work FFRDCs' parent organizations perform through non-FFRDC
divisions.  The ability to give impartial, objective advice is a key
characteristic of DOD's special relationship with FFRDCs and is
fundamental to the value DOD attaches to their use.  DOD believes the
risk of abuse is low and generally relies on its sponsors to design
appropriate safeguards.  We found that some sponsors have implemented
more rigorous controls than others.  Further, we found that FFRDC
trustees had a broad network of affiliations with private industry,
universities, and government.  As the Department of Defense
Appropriations Act, 1992, Public Law 102-172, directed, DOD now
requires an approved conflict-of-interest policy for FFRDC trustees,
but rather than establishing departmentwide criteria, DOD relies on
sponsors to design appropriate internal controls. 

Likewise, DOD has not historically exerted strong control over work
FFRDCs' parent organizations perform through non-FFRDC divisions. 
The question of whether accepting work from organizations other than
its sponsor impairs an FFRDC's ability to provide objective advice
has long been discussed.  DOD has revised its FFRDC management plan
to incorporate criteria to limit diversification by FFRDC parent
corporations and control the non-FFRDC work they perform.  The
ultimate effectiveness of the criteria in alleviating concerns about
parent corporations' diversification will depend largely on the
quality and thoroughness of sponsor reviews of proposed non-FFRDC
work. 

Third, Congress and others have repeatedly raised questions about the
adequacy of DOD policy guidance and oversight.  Concerns have been
raised regarding whether DOD policy guidance ensures that sponsors
(1) adequately justify awarding noncompetitive contracts for the
operation of the FFRDCs, (2) adequately screen tasks assigned to
FFRDCs, (3) implement adequate audit controls for the FFRDCs, (4)
award reasonable contract fees, and (5) make certain that FFRDC
employees' compensation is reasonable.  For example, concerns remain
as to whether DOD sponsors adequately justify the continuing need for
the FFRDCs.  Under federal policy, an agency may continue sponsorship
of an FFRDC and award a noncompetitive contract for its operation
only after demonstrating a continuing need for the center.  We found
that recently completed reviews to justify the continuing need for
FFRDCs generally contained extensive analyses and discussion and
addressed the criteria required by DOD.  These reviews, however, did
not include formal market surveys to identify alternatives to using
FFRDCs.  Similarly, we and others have raised recurring questions
about how FFRDCs use their contract fees.  Our recent work at The
Aerospace Corporation and the MITRE Corporation found that these two
organizations used contract fees to pay costs for items such as
entertainment, personal expenses for company officers, and employee
benefits. 

While DOD has taken some steps to address the concerns regarding
oversight, a Defense Science Board (DSB) Task Force recently noted a
significant distrust of DOD's use and management of its FFRDCs and
recommended the establishment of an independent advisory panel to
address this distrust.\5 This panel was established in late 1995 with
the mission of reviewing and advising DOD on the management of its
FFRDCs.  During meetings earlier this year, we observed that the
panel members appeared to be approaching their task with the utmost
seriousness and asking tough questions of both DOD and FFRDC
officials. 

Fourth, concerns have long been expressed that DOD does not
adequately consider hiring civil servants or contracting with
commercial firms as alternatives to using FFRDCs.  DOD maintains that
in-house and commercial contractor alternatives to the centers are
not as effective.  While limits on hiring additional civil servants
make in-house alternatives difficult to pursue, we concluded in 1994
that managers should be required to analyze the relative cost of
performing a function both by contract and in-house when deciding to
contract for services.\6 A 1991 Air Force initiative, known as Coral
Convert, illustrates the potential benefits of such analyses,
projecting $17.9 million in annual savings by using civil servants to
replace FFRDC and service support contractor personnel.\7 Similarly,
the complexity of awarding competitive contracts to commercial firms
may create incentives to use FFRDCs without adequately examining
competitive alternatives.  We identified two alternatives--using
broad agency announcements\8 to obtain information on the
capabilities of non-FFRDCs, or using procedures for the award of task
order contracts for advisory and assistance services that were
authorized under the Federal Acquisition Streamlining Act of 1994
Public Law 103-355--which could alleviate some of the administrative
burdens of competitive contracting. 

In our March 1996 testimony, we reported that DOD had recently
provided an update on initiatives it was taking to (1) define FFRDC
core work appropriate for FFRDCs, (2) establish stringent criteria
for the noncore work FFRDCs' parent corporations accept, (3) develop
guidelines to ensure that management fees are based on need and
detailed justification, and (4) establish an independent advisory
panel as the DSB Task Force recommended.  We generally support the
direction DOD has taken with these initiatives to address
long-standing issues related to management of FFRDCs.  Because the
initiatives are just now being implemented, it is too early to tell
how effectively they will be implemented or how well they will
address long-standing concerns regarding use of FFRDCs. 


--------------------
\3 Competition:  Issues on Establishing and Using Federally Funded
Research and Development Centers (GAO/NSIAD-88-22, Mar.  7, 1988). 

\4 Objectivity in this context is generally understood to mean the
ability to provide technical and analytical support that is unbiased,
impartial, free from real or perceived conflicts of interest, and
solely in the public interest. 

\5 The Role of Federally Funded Research and Development Centers in
the Mission of the Department of Defense, Defense Science Board Task
Force, April 25, 1995. 

\6 Government Contractors:  Contracting Out Implications of
Streamlining Agency Operations (GAO/T-GGD-95-4, Oct.  5, 1994). 

\7 In 1991, the Air Force Materiel Command (AFMC) proposed hiring
civil servants to replace certain contractor personnel at its
Electronic Systems Center and Space and Missile Systems Center.  In a
limited test of this proposal, AFMC successfully recruited qualified
personnel for civil service positions, and officials reported that
cost savings were being achieved.  The test was not expanded,
however, due to federal workforce reductions imposed after the
National Performance Review report. 

\8 The Federal Acquisition Regulation 6.102 (d)(2)(i) provides that a
broad agency announcement is a general announcement of an agency's
research interest, including criteria for selecting proposals, and
soliciting the participation of all offerors capable of satisfying
the government's needs. 


   AGENCY COMMENTS
------------------------------------------------------------ Letter :3

DOD generally concurred with this report (see app.  V).  DOD also
provided some technical comments and suggested clarifications that
have been incorporated into the report as appropriate. 


   SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :4

Our review included the 11 FFRDCs DOD sponsors.  We did not include
DOD's university-affiliated research centers in our review, since
DOD's internal reviews of FFRDC issues did not include these centers
until May 1995, and the studies on which we relied did not include
information on the university-affiliated centers.\9

We based this review on our work related to FFRDCs as well as reports
and studies done by others.  We supplemented this information with
limited additional data collection to provide up-to-date information
on DOD activities.  We did not attempt to provide definitive answers
to policy questions or recommendations for resolving the issues.  We
obtained information from documents, reports, and interviews with
officials from the Office of the Director of Defense Research and
Engineering, Defense Contract Audit Agency, selected FFRDCs and their
sponsors, and OMB.  We also interviewed officials from Johns Hopkins
University's Applied Physics Laboratory--currently a
university-affiliated research center that had been an FFRDC until
the mid-1970s--and the Professional Services Council.\10 We reviewed
legislation and federal regulations pertaining to FFRDCs and reviewed
prior reports by DOD, the DSB Task Force, Senate Governmental Affairs
Committee, Congressional Research Service, Office of Technology
Assessment, and our office.  We conducted our review between October
1994 and May 1996 in accordance with generally accepted government
auditing standards. 


--------------------
\9 DOD's internal advisory group decided to include
university-affiliated research centers when reviewing FFRDCs due to
the similar manner in which the organizations function. 

\10 The Professional Services Council is a trade association
representing commercial professional services' firms. 


---------------------------------------------------------- Letter :4.1

Appendix I provides additional details on the issues discussed above
relating to the management of DOD-sponsored FFRDCs, and appendix II
lists the reports issued by us and others relating to DOD's FFRDCs. 
Appendix III includes general information on DOD's FFRDCs, and
appendix IV compares the FFRDCs' former scope of work descriptions
and their recently identified core competencies.  Appendix V contains
comments from DOD. 

We are providing copies of this report to the Secretary of Defense,
Administrator of OFPP, and other interested congressional committees
and subcommittees.  Copies will also be made available to others on
request. 

Please contact me at (202) 512-4587 if you or your staff have any
questions concerning this report.  Major contributors to this report
are listed in appendix VI. 

David E.  Cooper
Associate Director,
Defense Acquisitions Issues


ISSUES RELATING TO THE MANAGEMENT
OF DOD-SPONSORED FFRDCS
=========================================================== Appendix I

This appendix discusses issues that we believe merit attention as
Congress and the Department of Defense (DOD) work to resolve concerns
relating to federally funded research and development centers
(FFRDC), including the (1) missions and core functions of FFRDCs, (2)
safeguards for FFRDC objectivity, (3) DOD oversight of the centers,
and (4) alternatives to using the centers.  This appendix also
discusses information from our March 1996 testimony on recent DOD
actions to improve the management of FFRDCs. 


   THE MISSIONS AND CORE FUNCTIONS
   OF DOD'S FFRDCS
--------------------------------------------------------- Appendix I:1

The DOD Inspector General's office, among others, has raised concerns
that FFRDC mission statements are too broad and do not clearly
identify specialized tasks that the FFRDC can perform most
effectively.  The mission statement of the Lincoln Laboratory FFRDC,
for example, states that the Laboratory is to "carry out a program of
research and development pertinent to national defense.  .  .  ."
There has been some concern within Congress and among others that
FFRDCs may be diversifying into areas outside their proper scope. 
DOD, however, now defines the role of FFRDCs as performing work that
is consistent with the FFRDC's purpose, mission, capabilities, and
core competencies and requires a special relationship to exist
between the FFRDC and its sponsor.  Such work need not require unique
capabilities.  Using the special relationship to identify work
appropriate for the FFRDCs will require potentially difficult and
subjective judgments to determine whether a task is appropriate for
an FFRDC to perform.  Further, there is some debate that the FFRDCs
may perform inherently governmental functions that government
employees should perform. 


      ROLE OF FFRDCS IS DEFINED IN
      TERMS OF SPECIAL
      RELATIONSHIP RATHER THAN
      UNIQUE CAPABILITIES
------------------------------------------------------- Appendix I:1.1

As a robust private sector professional services industry grew to
meet the demand for technical services, it became apparent that
industry had the capability to perform some tasks assigned to FFRDCs. 
As early as 1962, the Bell Report noted criticism that nonprofit
systems engineering contractors had undertaken work traditionally
done by private firms.\1

A 1971 DOD report stated, "It is pointless to say that the [systems
engineering FFRDC] function could not be provided by another
instrumentality.  .  .  ." According to this report, private
contractors could also do much the same type of work as the studies
and analyses of FFRDCs.  The report pointed to the flexibility of
using the centers and their broad experience with sponsors' problems
as reasons for continuing their use.  In 1994, the DOD Inspector
General concluded that FFRDC mission statements did not identify
unique capabilities or expertise, resulting in FFRDCs being assigned
work without adequate justification. 

In a 1988 report, we pointed out that governmentwide policy did not
require that FFRDCs be limited to work that industry could not do;
FFRDCs could also undertake tasks they could perform more effectively
than industry.  FFRDCs are effective, we observed, partly because of
their special relationship with their sponsoring agency.  This
special relationship embodies elements of access and privilege as
well as constraints to limit their activities to those DOD deems
appropriate. 

DOD has recently elaborated on and refined the concept of the FFRDC
special relationship.  According to DOD, FFRDCs perform tasks that
require a special or strategic relationship to exist between the task
sponsor and the organization performing the task.  Table I.1 shows
DOD's description of the characteristics of this special
relationship.  According to the Defense Science Board (DSB) Task
Force, these characteristics allow an FFRDC to perform research,
development, and analytical tasks that are integral to the mission
and operation of the sponsor. 



                               Table I.1
                
                     Characteristics of the Special
                 Relationship Between an FFRDC and Its
                              DOD Sponsor

Characteristic                      Description
----------------------------------  ----------------------------------
Long-term continuity.               Uninterrupted, consistent support
                                    based on a continuing
                                    relationship.

Comprehensive knowledge of sponsor  Expertise on and institutional
needs and operations.               memory about issues of enduring
                                    concern to the sponsor.

Adaptability.                       Ability to respond to emerging
                                    needs of their sponsors.

Objective, high-quality, current    A highly educated and skilled
research.                           professional staff that can
                                    produce thorough, independent
                                    analyses to address complex
                                    technical and analytical problems
                                    and maintain currency in their
                                    fields of expertise.

Freedom from real or perceived      Independence of commercial,
conflicts of interest.              shareholder, political, and other
                                    associations and dedication to the
                                    public interest.

Broad access to sensitive           Lack of institutional interests
government and commercial           that could lead to misuse of
proprietary information.            information or cause contractor
                                    reluctance to provide such
                                    information.

Quick response capability.          Short-term assistance to help
                                    sponsors meet urgent and high-
                                    priority requirements.
----------------------------------------------------------------------

--------------------
\1 Complete citations for all references to specific reports and
studies are provided in app.  II. 


      DOD SEEKS TO MORE CLEARLY
      IDENTIFY WORK TO BE
      PERFORMED BY FFRDCS
------------------------------------------------------- Appendix I:1.2

In 1995, the DSB Task Force and an internal DOD advisory group
concluded that there is a continuing need for FFRDCs for certain core
work that requires the special relationship previously described. 
Giving such tasks to commercial contractors, DOD concluded, would
raise numerous concerns, including questions about potential
conflicts of interest.  Accordingly, DOD has defined an FFRDC's core
work effort as tasks that (1) are consistent with the FFRDC's
purpose, mission, capabilities, and core competencies and (2) require
the FFRDC's special relationship with its sponsor.  The DOD advisory
group estimated that this core work effort represents about 3.4
percent of DOD's research, development, and analytic effort.  The
Task Force and advisory group also found that FFRDCs performed some
noncore work that did not require a special relationship and
concluded that this work should be transitioned out of the FFRDCs and
acquired competitively. 

On the basis of these conclusions, DOD required each sponsor to
review its FFRDC's core competencies, identify and prioritize the
FFRDC's core work effort, and identify the noncore work that should
be transitioned out of the FFRDC.  We found, however, that the core
competencies the sponsors identified differed little from the scope
of work descriptions that were in place previously (see app.  IV). 
In several cases, the core competencies simply restated the functions
listed in an FFRDC's scope of work description.  In others, the core
competencies summarized the scope of work functions into more generic
categories.  The sponsors we spoke to told us they identified little,
if any, noncore work being performed at their FFRDCs. 

Even though DOD states that it is important to ensure that tasks
assigned to the FFRDC meet the core work criteria, we believe it will
continue to be difficult to determine whether a task meets these
criteria.  FFRDC mission statements remain broad, and core
competencies differ little from the previous scope of work
descriptions.  As we stated in our 1988 report, the need for the
special relationship is the key to determining whether work is
appropriate for an FFRDC.  However, determining whether one or more
of the characteristics of the special relationship is required for a
task may be difficult, since the need for an element of the special
relationship is normally relative rather than absolute.  For example,
we believe DOD would expect objectivity in any research effort, but
it may be difficult to demonstrate that a particular task requires
the special degree of objectivity an FFRDC is believed to provide. 

Uncertainty about whether an FFRDC's special relationship allows it
to perform a task more effectively than other organizations also
accompanies decisions to assign work to an FFRDC.  In our 1988
report, we stated that full and open competition between FFRDCs and
non-FFRDCs could provide DOD assurance that it has selected the most
effective source for the work.  However, the report also stated that
exposing FFRDCs to marketplace competition would fundamentally alter
the character of the special relationship. 


      FFRDC'S ROLE IN SUPPORTING
      GOVERNMENTAL FUNCTIONS
      SHOULD BE CLOSELY MONITORED
------------------------------------------------------- Appendix I:1.3

There is some debate that FFRDCs may perform some inherently
governmental functions.\2 We believe this concern arises in part as a
result of the difficulty in clearly distinguishing work that should
be performed by FFRDCs.  DOD officials believe that it is clear that
the FFRDCs are not performing inherently governmental functions but
are performing work that supports such functions.  Because of both
the nature of the functions FFRDCs perform and the nature of the
special relationship between an FFRDC and its sponsor, we believe
that DOD needs to devote continuing management attention to ensuring
that inherently governmental functions are not tasked to the FFRDCs. 

On the basis of a review of agency use of advisory contractors, we
reported in 1991 that defining inherently governmental functions was
difficult and subject to substantial judgment.  We concluded that
examining the nature of the relationship between the government and
an advisory contractor was more useful than developing a specific
list of functions that are inherently governmental.  Further, we
stated that government officials must retain final authority to
decide government policy as well as maintain an active role in the
decision-making process to allow officials to make independent
judgments regarding a contractor's policy recommendations.  OFPP's
1992 policy statement on inherently governmental functions endorses
this conclusion, stating that when service contractors are used,
government action must be "taken as the result of informed,
independent judgments made by Government officials who are ultimately
accountable to the President."

While DOD officials believe that the FFRDCs are not performing
inherently governmental functions, they clearly perform functions
that federal officials should monitor closely.  OFPP's policy
statement includes a list of functions for which agencies must devote
additional management attention regarding the contractor's manner of
performance.  OFPP does not classify these functions as inherently
governmental but requires additional management attention for them so
as to ensure that appropriate agency control is maintained.  FFRDCs
routinely perform several of these functions, including the
following: 

  -- services that relate to analyses, feasibility studies, and
     strategy options to be used by agency personnel in developing
     policy;

  -- services in support of acquisition planning;

  -- technical evaluation of contract proposals;

  -- assistance in developing statements of work; and

  -- work in situations that permit access to sensitive government or
     proprietary information. 

We believe that in addition to closely monitoring FFRDCs' performance
of such functions, DOD should closely monitor the assignment of work
to FFRDCs to ensure that contracting out for such services is
appropriate.  Although the special relationship between an FFRDC and
its sponsor is an important criterion for assigning work to the
FFRDCs, certain characteristics of the special relationship can raise
questions about whether functions performed by FFRDCs should be
performed in-house. 

Our 1991 report on whether service contractors are performing
inherently governmental functions proposed guidelines to help
agencies in determining whether contracting out for consulting
services would be appropriate.  The guidelines do not specify which
functions are governmental, but address, among other things, the
nature of the relationship between a government agency and a
prospective contractor as aspects for consideration in contracting
out for services.  For example, while these guidelines suggest that
government officials must set a definite time period for the use of a
contractor and use government employees if the need is for a long or
indefinite period, an FFRDC is expected to have a long-term,
continuing relationship with its sponsor.  Similarly, while the
guidelines suggest that the institutional memory about agency
programs must reside with the agency rather than the contractor, the
FFRDC is expected to maintain program institutional memory for the
sponsor.  In this regard, we believe the guidelines are appropriate
for use by DOD when it considers whether to contract out for FFRDCs'
services or perform these services in-house. 


--------------------
\2 According to the Office of Federal Procurement Policy (OFPP),
"inherently governmental functions" are functions so intimately
related to the public interest as to mandate performance by
government employees.  OFPP cites as examples such functions as
commanding military forces, hiring federal employees, or awarding
government contracts. 


   SAFEGUARDS FOR FFRDC
   OBJECTIVITY
--------------------------------------------------------- Appendix I:2

According to DOD, the ability to give impartial, objective advice is
a key characteristic of the special relationship and is fundamental
to the value DOD attaches to the use of FFRDCs.  DOD believes the
risk of abuse is low and generally relies on sponsors to identify and
report or avoid potential conflicts of interest and to design
appropriate internal controls.  Some sponsors have seen a need to
implement more rigorous controls than others.  Similarly, DOD has not
historically exerted strong control over work FFRDCs' parent
organizations performed through non-FFRDC divisions.  The question of
whether accepting work from organizations other than its sponsor
impairs an FFRDC's ability to provide objective advice has long been
discussed.  Finally, as Congress directed (P.L.  102-172), DOD has
implemented controls over FFRDC trustees' affiliations with defense
contractors.  DOD's approach to implementing these controls is to
rely on sponsors' action rather than to establish departmentwide
criteria. 


      CONTROLS TO ENSURE
      OBJECTIVITY VARY
------------------------------------------------------- Appendix I:2.1

Since objectivity is a key characteristic of FFRDCs, questions have
been raised about whether DOD efforts to safeguard the objectivity of
its FFRDCs are adequate.  According to the DSB Task Force's 1995
report, DOD-sponsored FFRDCs are valuable largely because they
provide objective, high-quality work.  The Task Force reported that
the special relationship between FFRDCs and their sponsors allows the
centers to offer unbiased advice, analyses, and evaluations that
focus on the public interest.  DOD sponsors also identified
objectivity as one of the more important aspects of the special
relationship.  Objectivity is particularly important because FFRDCs
help DOD develop and analyze policy and strategy options and assess
alternative technology directions.  In some cases, FFRDCs also
evaluate contractors' performance and participate in source
selections for weapon system procurements. 

DOD sponsors believe that the potential for bias in FFRDC advice is
small.  According to the DSB Task Force, nonprofit corporations or
universities operate DOD's FFRDCs and therefore are not influenced by
corporate or shareholder interests.  Further, DOD asserts that FFRDCs
operate under restrictions more stringent than applied to other
government contractors.  For example, to prevent an FFRDC's judgment
from being swayed by prospective profits, DOD prohibits the centers
from undertaking long-term hardware production.  DOD also prohibits
the centers from competing with commercial firms for government
service contracts.  These restrictions, DOD maintains, are sufficient
to allow FFRDCs to provide impartial advice.  Accordingly, DOD has
issued little specific guidance on safeguarding FFRDC objectivity. 

In the absence of specific DOD guidance, some sponsors have seen the
need to implement more rigorous controls to safeguard objectivity
than have other sponsors.  For example, DOD's sponsoring agreement
with the MITRE Corporation provides that MITRE will avoid any action
that will place its personnel in possible or actual conflict of
interest positions.  The Navy's agreements with The CNA Corporation
also refer to ensuring freedom from conflicts of interest but include
additional controls not present in the agreements between DOD and
MITRE.  The CNA Corporation's charter and bylaws require an annual
conflict of interest statement for each employee and annual
statements of outside affiliations and financial interests for
trustees, corporate officers, and senior managers.  Further, in its
Code of Ethics and Standards of Conduct, The CNA Corporation warrants
that it should avoid organizational conflicts of interest and agrees
to disclose any conflicts that it discovers to the contracting
officer.  Also, the Code states that if The CNA Corporation employees
do not report perceived or actual conflicts to the appropriate
supervisors, they may be subject to disciplinary action, which may
include termination of employment. 


      DOD IS SEEKING STRONGER
      OVERSIGHT AS FFRDCS
      DIVERSIFY
------------------------------------------------------- Appendix I:2.2

The question of whether accepting work from organizations other than
its sponsor impairs an FFRDC's ability to provide objective advice
has long been discussed.  As early as 1962, the Bell Report raised
this question but noted that no clear consensus had developed as to
whether concerns about diversification were well founded.  The report
recognized that studies and analyses FFRDCs could effectively serve
multiple clients but concluded that systems engineering organizations
were primarily of value when they served a single client.  During the
early 1970s, DOD encouraged its FFRDCs to diversify into nonsponsor
work.  According to a 1976 DOD report on the management of federal
contract research centers,\3 FFRDCs that did not diversify suffered
efficiency and morale problems as their organizations shrank in the
face of declining DOD research and development budgets.  Nonetheless,
this report recommended that the systems engineering FFRDCs limit
themselves to DOD work and adjust their workforces in line with
changes in the DOD budget.  Regarding the MITRE Corporation, the
report recommended that MITRE create a separate organization to carry
out its non-DOD work.  In 1994, concerns were raised within Congress
that the activities of non-FFRDC organizations resulted in "an
ambiguous legal, regulatory, organizational, and financial
situation," and DOD was directed to prepare a report on non-FFRDC
activities (Senate Report 103-282).  More recently, however, the DSB
Task Force concluded in its 1995 report that FFRDCs and their parent
companies should be allowed to accept work outside the core domain
only when doing so was in the best interest of the country; the Task
Force did not propose criteria for determining whether accepting
nonsponsor work was in the country's best interest. 

Acceptance of non-FFRDC work is now common at seven of the eight
parent organizations operating DOD's FFRDCs.  Except for the
Institute for Defense Analyses, each parent organization performs
some non-FFRDC work either within the FFRDC or through an affiliated
organization created to pursue non-FFRDC work.  Some affiliates are
quite small:  The CNA Corporation's Institute for Public Research
accounts for about 3 percent of the Corporation's total effort. 
Other affiliates are more significant:  prior to its division in
1996, the MITRE Corporation's two non-FFRDC affiliates accounted for
about 11 percent of MITRE's total effort, and RAND's five non-FFRDC
divisions account for about 32 percent of RAND's total effort.  The
Massachusetts Institute of Technology and Carnegie-Mellon
University--parent organizations of Lincoln Laboratory and the
Software Engineering Institute, respectively--each pursue a diverse
range of non-FFRDC activities. 

DOD has recently become more active in seeking to oversee work the
parent organizations perform through non-FFRDC divisions.  Sponsors
have historically had the opportunity to oversee nonsponsor work
performed within the FFRDC because the work is carried out under the
FFRDC contracts that sponsors administer.  This contract oversight
mechanism is not available for non-FFRDC divisions.  During 1995, for
example, the Air Force expressed great reluctance to support The
Aerospace Corporation's proposal to establish a non-FFRDC affiliate,
indicating its concerns about the perception of a conflict of
interest.  Similarly, the MITRE Corporation's Board of Trustees
decided to create a separate corporate division to perform non-FFRDC
work for government and commercial customers after DOD rejected a
MITRE proposal to establish an affiliated organization to handle
commercial and non-FFRDC work.  DOD generally concurred with the
direction of the Board's plan to divide the corporation and formally
expressed the Department's position that any cost recovery and
distribution of assets and liabilities be legally appropriate,
justified, and subject to audit by the Defense Contract Audit Agency
(DCAA).  This new corporation has a separate Board of Trustees and
its own corporate officers.  Further, no work is to be subcontracted
between the two entities in order to preclude the sharing of
employees involved in DOD work--and knowledge developed in the course
of DOD work--with the new corporation. 


--------------------
\3 FFRDCs were previously called federal contract research centers. 


      FFRDC SPONSORS OVERSEE
      AFFILIATIONS OF TRUSTEES
------------------------------------------------------- Appendix I:2.3

A long-standing concern has been that the personal financial
interests and affiliations of FFRDC trustees may affect the
objectivity of the advice they provide DOD.  In 1962, the Bell Report
noted that the boards of organizations such as FFRDCs were often
comprised of representatives of universities and commercial firms
involved in federal research.  Although acknowledging that it was in
the public interest to have such organizations "controlled by the
most responsible, mature, and knowledgeable [individuals] available
in the nation," the report pointed to the clear possibility that the
potential for conflicts of interest inherent in such interlocking
directorships could harm the public interest.  The report concluded
that FFRDC trustees should observe the spirit of recently adopted
government policy concerning government consultants and advisers. 
This policy required consultants and advisers to fully disclose their
private interests and abstain from rendering advice on matters that
would affect their private interests. 

FFRDC trustees have a broad network of affiliations.  Our 1995 report
on the affiliations of the 141 trustees who served on FFRDC boards
during fiscal year 1993 showed that these individuals had
affiliations with 447 private sector firms, 126 universities, and 91
federal government or military organizations.  Further, 51 of the
private sector firm affiliations and 77 of the university
affiliations were with organizations that were among the top 100
prime contractors to DOD.  For example, one individual served as a
trustee of two different DOD FFRDCs and was affiliated with four
private sector firms, including one of the nation's principal weapons
producers.  Another trustee was affiliated with two presidential
advisory commissions addressing military questions and with five
private sector firms, including a major defense aerospace company. 
Another was a member of the DSB and was affiliated with two
universities that operate FFRDCs for DOD and receive significant DOD
research funding. 

This issue of interlocking directorships received little attention in
the studies of FFRDC issues we reviewed that were published during
the 1970s and 1980s.  In 1991, Congress (P.L.  102-172) prohibited
members of FFRDC boards of trustees or directors from simultaneously
serving on the board of a for-profit DOD contractor unless the FFRDC
had a DOD-approved conflict of interest policy for board members. 
DOD assigned responsibility for reviewing and approving conflict of
interest policies for boards of trustees to FFRDC sponsors and
declined to establish DOD-wide criteria for approving such policies. 
In 1992, a congressional committee expressed doubt about the wisdom
of DOD's heavy reliance on sponsors to exercise oversight of FFRDCs
(House Report 102-627).  The DOD Inspector General's 1994 report
stated that each of the organizations that operates an FFRDC for DOD
had conflict of interest policies for board members that had been
approved by primary sponsors and that all board members had disclosed
their affiliations as these policies required.  Further, the report
found that contracting officers for 9 of the 10 FFRDCs, however, were
personally unaware of the affiliations of FFRDC trustees. 


   DOD OVERSIGHT OF ITS FFRDCS
--------------------------------------------------------- Appendix I:3

Questions have repeatedly been raised within Congress and among
others about the adequacy of DOD's policy guidance and oversight. 
The Professional Services Council, for example, has expressed concern
that sponsors do not adequately consider whether their FFRDCs are
still needed or whether work they plan to assign to their FFRDCs is
appropriate.  Concerns have also been raised by the DOD Inspector
General's office, about whether controls over FFRDC costs are
adequate, particularly concerning contract fees paid to FFRDCs and
compensation paid to their employees.  In each case, DOD has taken
some steps to address the concerns raised.  The DSB Task Force's 1995
report noted a significant distrust of DOD's use and management of
FFRDCs and recommended the establishment of an independent review
panel to address this distrust. 


      DOD EFFORTS TO DEMONSTRATE
      NEED FOR FFRDCS HAVE BEEN
      CRITICIZED
------------------------------------------------------- Appendix I:3.1

Concerns have been raised by DOD's Inspector General's office, among
others, that DOD does not adequately justify the continuing need for
an FFRDC, which determines whether to continue sponsorship and
subsequently award noncompetitive contracts to the organizations that
operate FFRDCs.  The Federal Acquisition Regulation at section
35.017-4 (a), requires a sponsor to review the continuing need for an
FFRDC before renewing its contract or agreement.  This comprehensive
review must be done at least every 5 years.  The Professional
Services Council asserts that private firms have a broad range of
research and analytical capabilities that would allow them to
undertake many tasks DOD assigns to FFRDCs.  Further, according to
the Professional Services Council, industry and DOD have successfully
used contract language to mitigate potential conflicts of interest. 
DOD continues to rely on FFRDCs, the Professional Services Council
maintains, because it has not considered the capabilities of
industry.  In a 1993 report, the DOD Inspector General found that the
sponsors did not use market surveys to identify alternatives to
FFRDCs during comprehensive reviews.  The Inspector General also
noted that sponsors did not publicize planned FFRDC contract renewals
to invite industry to express interest in operating the FFRDCs and
explain its capability to do so. 

DOD has taken steps to ensure that sponsors provide more thorough
documentation of their comprehensive reviews.  In response to
congressional concern, DOD issued a 1992 FFRDC management plan, which
directed sponsors to provide copies of review reports to the Director
of Defense Research and Engineering for evaluation and to update
reviews conducted before 1990.  DOD's 1994 updated FFRDC management
plan outlines the criteria that governmentwide policy requires
sponsors to address in comprehensive reviews.  Under this plan, the
Director of Defense Research and Engineering must concur with the
sponsor's determination to continue an FFRDC before its agreement is
renewed.  Sponsors that we spoke with believe that current
comprehensive review guidance is adequate, although it lacks detail
on how to assess the criteria for renewing an FFRDC's sponsoring
agreement.  We found that recently completed comprehensive reviews
generally contained extensive analyses and discussion and addressed
the required criteria but did not include formal market surveys. 


      APPROPRIATENESS OF WORK
      GIVEN TO FFRDCS HAS BEEN
      QUESTIONED
------------------------------------------------------- Appendix I:3.2

Concerns have also been expressed about whether sponsors adequately
screen tasks assigned to FFRDCs.  The Federal Acquisition Regulation
at section 35.017-3 requires sponsors to assign an FFRDC only tasks
that are consistent with its mission and scope of work.\4 The
Professional Services Council, however, maintains that FFRDCs often
receive work for which they are not uniquely qualified because
assigning a task to an FFRDC is more convenient than awarding a
separate competitive contract.  On the basis of a survey of 229 FFRDC
projects that are discussed in its 1994 report, the DOD Inspector
General also criticized task sponsors' justifications for requesting
FFRDC support.  Task sponsors cited unique FFRDC expertise to explain
why an FFRDC could perform a task more effectively than non-FFRDC
sources, but they generally did not define the characteristics that
made the FFRDC more effective.  Further, task sponsors generally did
not compare FFRDC costs with the cost of other sources.  Similarly,
the Inspector General of the Defense Information Systems Agency found
that the agency lacked adequate internal controls over requests for
support from the DOD-sponsored FFRDC operated by the MITRE
Corporation.  According to the Inspector General, justifications that
task sponsors provided included "boilerplate" wording with no
analysis to show why MITRE's FFRDC was the best alternative. 

DOD believes that critics expect unrealistically thorough
justification for assigning tasks to an FFRDC.  DOD's 1994 management
plan requires primary sponsors to review descriptions of work given
to FFRDCs to ensure the work is consistent with FFRDC missions.  DOD
relies on this review by primary sponsors to identify any
inappropriate requests for work that task sponsors submit. 
Commenting on the DOD Inspector General's report, the Director of
Defense Research and Engineering argued that the auditors had applied
extreme and unreasonable standards to determine whether FFRDCs had
unique qualifications for tasks.  Further, the Director noted that
the Inspector General had objected to other reasons for assigning
work, such as an FFRDC's familiarity with sponsor requirements and
quick response capability.  These justifications, the Director
observed, are part of the core rationale for maintaining FFRDCs.  The
DSB Task Force's 1995 report agreed that such factors were important
reasons for using FFRDCs.  Nonetheless, DOD has concluded that
additional management controls over assignment of work to FFRDCs were
needed and recently took steps to identify the core work of each
FFRDC more clearly. 


--------------------
\4 As previously discussed, DOD also includes consistency with the
special relationship as a criterion for assigning work to the FFRDCs. 


      ADEQUACY OF FFRDC AUDIT
      CONTROLS HAS BEEN CRITICIZED
------------------------------------------------------- Appendix I:3.3

On the basis of a governmentwide review of FFRDCs, a 1992 report by
the Senate Governmental Affairs Committee stated that required audits
of FFRDC overhead costs were not being completed in a timely manner. 
According to the Committee's report, DCAA performed audits of FFRDCS
sponsored by DOD, the National Aeronautics and Space Administration
(NASA), and the Nuclear Regulatory Commission several years after the
fiscal year had ended.  In addition, the Committee found that several
audits contained repetitive or significant audit exceptions to FFRDC
cost claims that apparently were not resolved at the time of the
deficiency.  DCAA acknowledged that a substantial backlog of overhead
cost audits had developed during the late 1980s and early 1990s, but
stated this problem was not unique to FFRDCs.  According to DCAA
officials, increasing demand for more time-critical price proposal
audits and a shortage of audit staff led to a DOD-wide backlog.  By
1995, DCAA had reduced the overhead audit backlog for FFRDCs. 


      CONTRACT FEES HAVE LONG BEEN
      RECOGNIZED AS A PROBLEM AREA
------------------------------------------------------- Appendix I:3.4

The Senate Governmental Affairs Committee's report also raised
concerns about the justification for FFRDC contract fees.  Contract
fees are discretionary funds provided to FFRDCs in addition to
reimbursement for incurred costs and are similar to profits
commercial contractors earn.  In a 1969 report, we concluded that
nonprofit organizations such as FFRDCs incur some necessary costs
that may not be reimbursed under the procurement regulations, and we
recommended that the Bureau of the Budget (now known as the Office of
Management and Budget (OMB)), develop guidance that specifies the
costs contracting officers should provide that fees cover.  As the
Committee report noted, since governmentwide guidance on fees had not
been prepared, it was difficult to determine whether FFRDC fees were
adequately justified.  In response to the Committee's report, OFPP,
OMB's procurement policy arm, agreed that governmentwide guidance on
contract fees for nonprofit organizations was needed.  OFPP has
deferred action on developing such guidance pending a review of
regulatory changes that DOD and the Department of Energy are
currently developing. 

In the absence of governmentwide guidance, recurring questions have
been raised about how FFRDCs use their fees.  In its 1994 report, for
example, the DOD Inspector General concluded that FFRDCs used $43
million of the $46.9 million in fiscal year 1992 DOD fees for items
that should not have been funded from fees.  The bulk of this $43
million funded independent research projects that should have been
charged to overhead rather than paid from fees.  The remainder of the
$43 million funded unallowable costs and future requirements that the
Inspector General concluded were not necessary for FFRDC operations. 
Similarly, DCAA reviewed fiscal year 1993 fee expenditures at the
MITRE Corporation and concluded that just 11 percent of the
expenditures reviewed were ordinary and necessary to the operation of
the FFRDC.  DCAA reported that MITRE used fees to pay for items such
as lavish entertainment, personal expenses for company officers, and
generous employee benefits.  Our recent work at The Aerospace
Corporation found that the corporation used about $11.5 million of
its $15.5 million management fee in fiscal year 1993 for sponsored
research.  Aerospace used the remainder of its fee and other
corporate resources for capital equipment purchases; real and
leasehold property improvements; and other unreimbursed expenditures
such as contributions, personal use of company cars, conference
meals, trustee expenses, and new business development expenses. 

Federal agencies are attempting to address concerns about FFRDC fees
but are pursuing different approaches.  DOD has concluded that
contracting officers should determine an FFRDC's needs for
discretionary funding and base fee awards on these needs.  In
particular, DOD believes contracting officers need clear guidance on
what fees may appropriately be provided to cover.  DOD has revised
its FFRDC management plan to clarify procedures sponsors and
contracting officers should follow in determining fees.  NASA, in
contrast, has revised its FFRDC contract with the Jet Propulsion
Laboratory to provide a performance-based fee.  NASA provides the
Laboratory a base fee of $6 million annually, and the laboratory can
earn up to an additional $12 million in fees by meeting performance
criteria in its contract.  The Department of Energy is also revising
its FFRDC contracts to incorporate performance criteria.  Fee
provisions in the revised contracts differ somewhat, but some
contracts link payment of fees to meeting contract performance
criteria. 


      REASONABLENESS OF
      COMPENSATION HAS BEEN
      QUESTIONED
------------------------------------------------------- Appendix I:3.5

The proper benchmark for assessing FFRDC employees' compensation,
particularly executives, has long been a problematic issue.  The
Federal Acquisition Regulation at section 31.205-6 requires
contractors to demonstrate that employee compensation is reasonable
in comparison to compensation paid by firms in the same industry,
among other tests.  FFRDCs generally use compensation at private
industry research firms as the benchmark for their compensation.  As
early as 1962, however, the Bell Report asked whether a corporation
similar to an FFRDC--created to provide services to the government
and receiving all its financial support from the government--could be
considered a "private" firm.  The report concluded that private
industry was generally the proper benchmark for setting pay at such
organizations, but it noted that the salaries of top management
merited special attention.  In 1969, Congress imposed a cap on
salaries for FFRDC personnel that lasted for several years.\5 In
1994, Congress imposed a cap on FFRDC executive compensation, equal
to level I of the federal Executive Schedule, that became effective
on July 1, 1995 (P.L.  103-335).  Congress has not passed legislation
to impose this cap through fiscal year 1996. 

The conclusions that may be drawn from recent FFRDC employee
compensation reviews depend on whether private industry or civil
service pay scales are selected as the benchmark.  The DOD Inspector
General, for example, recently reported that salaries for FFRDC chief
executives were generally comparable to private industry chief
executive salaries but uniformly well in excess of the highest civil
service salaries.  Salaries for other FFRDC executives were generally
somewhat less than salaries for other private industry executives but
generally exceeded civil service salaries.  The Inspector General
also reported indications that salaries for scientists and engineers
were higher than salaries in leading research and development
organizations.  A DCAA audit of compensation levels at MITRE found
similar conditions.  Our recent work on executive compensation at The
Aerospace Corporation found that salary costs for executives
increased by 78 percent over the 3-year period ending September 1994. 
This increase resulted from salary increases of up to 29 percent for
individual executives during 1992 and a 45-percent increase in the
number of executives from 22 to 32.  During that time, average annual
executive salaries increased by 23 percent from $125,000 to about
$153,000.  In addition, we recently issued a report on FFRDC
compensation in relation to federal compensation levels, which found
that the average compensation for all fiscal year 1993 FFRDC
employees in the study was $89,000. 

DOD believes employee compensation at its FFRDCs is generally
reasonable and accepts private industry pay scales as the relevant
benchmark.  In its 1995 report, the DSB Task Force noted that
flexible policies with respect to employee pay and benefits is one
private sector characteristic that makes FFRDCs attractive to DOD. 
The Task Force also warned that mandated salary limits would lead to
a deterioration in the quality of FFRDC employees.  DCAA has
identified instances in which it believes pay for certain classes of
FFRDC employees is excessive--compared with pay for comparable
private industry employees--and DOD contracting officers have acted
to resolve these cases.  DOD also commissioned a compensation
consulting firm to review compensation for staff that would have been
subject to the fiscal year 1995 salary cap.  On the basis of this
review, DOD concluded that the staff's compensation was reasonable
and should not be restricted by mandatory limits. 


--------------------
\5 The Armed Forces Appropriations, authorization Act, 1970, P.L. 
91-121. 


      INDEPENDENT REVIEW PANEL HAS
      BEEN CREATED TO ADVISE ON
      FFRDCS
------------------------------------------------------- Appendix I:3.6

Although it endorsed the need for organizations such as FFRDCs, the
DSB Task Force stated in its report that significant mistrust existed
concerning DOD's use and oversight of FFRDCs.  A principal concern,
according to the Task Force, was that DOD assigned work to FFRDCs
that could have been performed as effectively by private industry
sources and acquired using competitive procurement procedures. 
Further, the Task Force found that the lack of opportunities for
public review and comment on DOD's process for managing and assigning
work to FFRDCs--available in the competitive contracting
process--invited mistrust.  To address public skepticism about DOD's
use and management of FFRDCs, the Task Force recommended creation of
an independent review panel of highly respected personnel from
outside DOD.  The panel would review the continuing need for FFRDCs,
FFRDC missions, and DOD's management and oversight mechanisms for
FFRDCs. 


   ALTERNATIVES TO USING FFRDCS
--------------------------------------------------------- Appendix I:4

Concerns have long been expressed by the Professional Services
Council, among others, that DOD does not adequately consider hiring
civil servants or contracting with commercial firms as alternatives
to using FFRDCs.  Even though DOD maintains that in-house and
contractor alternatives to the centers are not as effective, limits
on hiring additional civil servants make in-house alternatives
difficult to pursue and the complexity of awarding competitive
contracts to commercial firms has been mentioned as an incentive for
using FFRDCs.  These factors may hamper DOD's efforts to reduce
dependance on FFRDCs but need not exclude consideration of
alternatives. 


      SHRINKING FEDERAL WORKFORCE
      COMPLICATES TRANSFERRING
      FFRDC WORK
------------------------------------------------------- Appendix I:4.1

Limits on the size of the federal workforce impede efforts to hire
civil servants to perform work contracted out to FFRDCs.  In
analyzing the feasibility of alternatives to FFRDCs, several sponsors
have noted that personnel ceilings would prevent acquiring in-house
resources to accomplish FFRDC functions.  Similarly, the DSB Task
Force concluded in its 1995 report that it was not feasible to
replace FFRDCs with in-house resources because the DOD workforce was
expected to shrink rather than expand in coming years.  Thus,
sponsors seeking to replace FFRDC capabilities must compete for a
shrinking pool of authorized civil service positions. 

We have previously expressed concern that personnel ceilings may lead
agencies to contract out work when it is not desirable to do so.  For
example, our 1991 report stated a common perception among agency
managers that contract dollars were easier to obtain than
authorizations for additional staff.  As a result, we noted instances
in which it was unclear whether agencies had contracted out work that
might have involved inherently governmental functions and that might
have been less expensive if it had been done in-house.  In the
report, we suggested that Congress consider exploring with OMB
allowing civilian agencies to manage their activities within an
authorized budget without regard to personnel ceilings.  In 1994, we
stated that before an agency decides whether to contract out for a
particular service, the agency should be aware of the comparative
cost of contracting out versus using federal employees.  Because DOD
foresees no realistic opportunity to hire additional civil service
staff in lieu of contracting with FFRDCs, DOD's reviews of the
continuing need for FFRDCs do not include such analyses. 

An Air Force initiative, known as Coral Convert, illustrates the
potential benefits of performing such analyses.  During 1991, the Air
Force Materiel Command (AFMC) proposed to hire civil servants to
replace certain contractor personnel at its Electronic Systems Center
and Space and Missile Systems Center.  AFMC estimated that hiring
in-house personnel to replace 500 FFRDC and support service
contractor personnel would save $17.9 million annually.  In addition
to savings, the proposal was expected to improve management control
over the acquisition workforce and provide flexibility to select the
best mix of government and contractor personnel.  In a limited test
of this proposal, the Electronic Systems Center successfully
recruited qualified personnel for civil service positions.  At one
location, for example, 3,300 applicants responded to a newspaper
advertisement for 167 positions.  According to Air Force officials,
the personnel hired in this test are performing effectively, and cost
savings are being achieved.  The test was not expanded, however, due
to federal workforce reductions imposed after the National
Performance Review report. 

The Defense Authorization Act, Fiscal Year 1996, Public Law 104-106,
complicates the picture DOD faces as it assesses the right mix of
in-house and contracted FFRDC resources.  One measure requires DOD to
consider the efficiencies gained through acquisition process
streamlining in developing a plan to reduce its acquisition workforce
by 25 percent over 5 years.  Another measure requires DOD to develop
a plan to consolidate and reduce the size of its FFRDCs.  Certain DOD
FFRDCs--most notably the Air Force's two systems engineering
FFRDCs--provide needed support to DOD's acquisition workforce. 


      SIMPLIFIED PROCEDURES FOR
      PROCURING SERVICES ARE NOW
      AVAILABLE
------------------------------------------------------- Appendix I:4.2

Federal procurement procedures are one motivation for using FFRDCs. 
The Professional Services Council asserts that DOD program managers
frequently place work with FFRDCs to avoid the administrative burdens
of competitive contracting.  DOD maintains that FFRDC staff, due to
their extensive background with sponsors' programs, have a unique
ability to respond to high-priority, time-sensitive analytical
requirements.  DOD acknowledges that procuring services competitively
can be burdensome and time-consuming, but maintains that convenience
in obtaining services is not a motivation for using FFRDCs.  In its
1994 survey of 229 FFRDC projects, however, the DOD Inspector General
found that about 30 percent of task sponsors reported that ease and
quickness of obtaining services influenced their decision to seek
FFRDC support. 

While some task sponsors may perceive FFRDCs to be attractive because
obtaining services appears quick and easy, two alternatives could
alleviate administrative burdens.  In our 1988 report, we recommended
that DOD test using broad agency announcements to obtain information
on the capabilities of non-FFRDCs--including commercial
contractors--to participate in pursuing DOD's research goals.  We
concluded that the procedures associated with such announcements
could be a relatively informal and expeditious way to solicit
industry proposals in support of agency research plans.  More
recently, in its 1994 report, the DOD Inspector General endorsed
broad agency announcements as a tool for gathering information on
alternatives to FFRDCs. 

The task order contracting procedure for advisory and assistance
services authorized under the Federal Acquisition Streamlining Act of
1994, Public Law 103-355, which was revised from previous procedures,
is another mechanism that could be useful.  Task order contracts
provide for the government to issue orders to perform defined tasks
during the contract term but do not commit the government to purchase
more than a minimum quantity of services; services are purchased by
placing task orders under the contract.  Congress was concerned that,
under the previous procedures, such contracts could be abused to
avoid competition (Senate Report 103-258, S.1587, 103 Cong.  2d Sess. 
15 (1994)).  Under the new procedures, when an agency plans to award
a task order contract for advisory and assistance services to exceed
3 years and valued at over $10 million, it must provide for awards to
multiple contractors under a single solicitation.  When placing
individual task orders under such contracts, the agency does not need
to conduct formal competitions; however, it must provide all
contractors awarded contracts under the soliticitation a fair
opportunity to be considered for the order.  Congress intended to
give agencies broad discretion in establishing procedures for the
evaluation and award of orders under such contracts (Senate Report
103-258, id.). 

Experience gained through a planned Air Force initiative could shed
light on how well these procedures work and on how effectively
private contractors can perform FFRDC work.  The Air Force plans to
contract competitively for certain core functions its systems
engineering FFRDCs currently perform and has ranked these functions
to identify those most suitable for competitive contracting.  The
Electronic Systems Center, for example, considers laboratory testing
to explore or verify performance most suitable for competitive
contracting with a non-FFRDC because following an established test
protocol and reporting results involve little exercise of discretion
or judgment.  In contrast, developing and analyzing systems
architectures requires the exercise of considerable discretion and
judgment, as well as a broad knowledge of Air Force systems;
consequently, the Electronic Systems Center does not consider this
function suitable for competitive contracting.  The Air Force will
not start this test until decisions are made on plans to streamline
its acquisition process; some functions its systems engineering
FFRDCs currently perform may be curtailed or eliminated. 


   DOD ACTIONS TO IMPROVE FFRDC
   MANAGEMENT
--------------------------------------------------------- Appendix I:5

Under an action plan submitted to Congress in 1995, DOD has begun
steps to strengthen management of its FFRDCs and address some of the
issues we identified.  In February 1996, DOD updated the status of
four key action plan initiatives, including (1) defining core work
appropriate for FFRDCs; (2) establishing criteria for acceptance of
work outside the core by FFRDCs' parent companies; (3) developing
guidance to ensure that management fees provided to FFRDCs are based
on need and detailed justification; and (4) establishing an
independent advisory panel to review DOD's management, use, and
oversight of its FFRDCs.  As stated in our March 1996 testimony, we
believe these initiatives will move DOD's management of FFRDCs in a
positive direction.  Because the initiatives are just now being
implemented, it is too early to tell how effectively they will be
implemented or how well they will address long-standing concerns
regarding use of FFRDCs. 

First, DOD has adopted criteria for identifying an FFRDC's core work
effort, defined as work that (1) is consistent with the FFRDC's
purpose, mission, capabilities, and core competencies and (2)
requires a special relationship to exist between the FFRDC and its
sponsor.  DOD directed sponsors to review the workload of their
FFRDCs and identify work that should be transitioned out of the FFRDC
and acquired competitively.  According to the Under Secretary of
Defense (Acquisition and Technology), sponsors had identified about
$43 million in noncore work being performed at the FFRDCs; this
represents about 4 percent of the planned $1.2 billion in fiscal year
1996 funding.  This noncore work is currently being transferred out
of the FFRDCs.  While the core work criteria seek to clarify what
tasks are appropriate for an FFRDC, the need for a task to be
performed by an organization having a strategic relationship with the
sponsor is a key element of the criteria.  Thus, applying the
criteria requires making judgments about the relative effectiveness
of various sources for work in the absence of full information on
capabilities, which open competition would provide.  Sponsors'
implementation of the DOD criteria, therefore, may eventually prove
unsatisfactory to critics who seek a simple and unambiguous
definition of work appropriate for the FFRDCs. 

DOD has revised its FFRDC management plan to incorporate guidelines
on diversification by FFRDC parent corporations into non-FFRDC work. 
The guidelines provide that parent corporations may accept non-FFRDC
work subject to sponsor review for compliance with criteria that
requires that the work, among other things, (1) is in the national
interest, (2) does not detract from performance of FFRDC work, and
(3) does not give rise to real or perceived conflicts of interest. 
The management plan does not provide for DOD-level officials to
review or approve such work, but provides sponsors the opportunity to
exercise greater control over the non-FFRDC work parent corporations
undertake.  The plan's ultimate effectiveness in alleviating concerns
about parent corporations' diversification will depend on the quality
and thoroughness of sponsor reviews of proposed non-FFRDC work. 

DOD's revised management plan also clarifies that contracting
officers must consider demonstrated needs in determining management
fees and incorporates guidelines defining legitimate needs.  The
plan's guidelines (1) move allowable costs out of fee and (2)
establish consistent policies on providing fees to cover costs not
reimbursable under the contract that the FFRDC can demonstrate are
needed for its successful operation.  If effectively implemented,
these actions should help to resolve many of the long-standing
concerns over FFRDC use of management fees.  Since much of fee is
used for normally allowable independent research costs, moving
independent research out of fee will substantially reduce fee and
subject these research costs to review and oversight with appropriate
principles for determining costs under contracts with educational
institutions or nonprofit organizations.  Defining ordinary and
necessary--but unallowable--business expenses, which may be covered
by fee, remains a more challenging issue.  DOD's management plan does
not provide examples of ordinary and necessary expenses that fee may
properly be provided to cover.  Consequently, we believe debate will
continue on whether fee can be used for such items as personal
expenses for company officers, entertainment, and new business
development. 

Finally, the advisory panel the DSB Task Force recommended has been
established.  In late 1995, six individuals--who are either DSB
members or consultants--were appointed to the panel, which has been
constituted as a task force of the DSB.  The panel's charter calls
for the members to review and advise DOD on the management of FFRDCs
by

  -- reviewing guidelines for the appropriate scope of work,
     customers, organizational structure, and size of the FFRDCs;

  -- overseeing compliance with DOD's FFRDC management plan;

  -- reviewing sponsors' management of FFRDCs;

  -- reviewing the level and appropriateness of non-DOD and
     nonsponsor work performed by the FFRDCs;

  -- overseeing the comprehensive review process; and

  -- performing selected FFRDC program reviews. 

In January 1996, the panel began a series of fact-finding meetings at
the FFRDCs, which were attended by DOD sponsor personnel and FFRDC
officials.  Representatives of our office attended the panel's
initial fact-finding meetings and observed that the members appeared
to approach their task with the utmost seriousness and challenged the
conventional wisdom by asking tough questions of both DOD and FFRDC
officials.  DOD anticipates that the panel will submit its first
report in the summer of 1996. 


REFERENCES AND RELATED FFRDC
PRODUCTS
========================================================== Appendix II

Defense Research and Development:  Federal Centers' 1993 Compensation
in Relation to Federal Levels (GAO/NSIAD-96-140, July 10, 1996). 

Federally Funded R&D Centers:  Information on the Size and Scope of
DOD-Sponsored Centers (GAO/NSIAD-96-54, Apr.  22, 1996). 

Federally Funded R&D Centers:  Observations on DOD Actions to Improve
Management (GAO/T-NSIAD-96-117, Mar.  5, 1996). 

Defense Research and Development:  Fiscal Year 1993 Trustee and
Advisor Costs at Federally Funded Centers (GAO/NSIAD-96-27, Dec.  26,
1995). 

Federal Research:  Information on Fees for Selected Federally Funded
Research and Development Centers (GAO/RCED-96-31FS, Dec.  8, 1995). 

Federally Funded R&D Centers:  Use of Fee by the MITRE Corporation
(GAO/NSIAD-96-26, Nov.  27, 1995). 

Federally Funded R&D Centers:  Use of Contract Fee by The Aerospace
Corporation (GAO/NSIAD-95-174, Sept.  28, 1995). 

Defense Research and Development:  Affiliations of Fiscal Year 1993
Trustees for Federally Funded Centers (GAO/NSIAD-95-135, July 26,
1995). 

A History of the Department of Defense Federally Funded Research and
Development Centers, Office of Technology Assessment (OTA-BP-ISS-157,
June 1995). 

Report of the DOD Internal Advisory Group on Federally Funded
Research and Development Centers (May 18, 1995). 

Compensation to Presidents, Senior Executives, and Technical Staff at
Federally Funded Research and Development Centers, DOD Office of the
Inspector General (95-182, May 1, 1995). 

Comprehensive Review of the Department of Defense's Fee-Granting
Process for Federally Funded Research and Development Centers,
Director of Defense Research and Engineering (May 1, 1995). 

The Role of Federally Funded Research and Development Centers in the
Mission of the Department of Defense, DSB Task Force (Apr.  25,
1995). 

Addendum to Final Audit Report on Contracting Practices for the Use
and Operations of DOD-Sponsored Federally Funded Research and
Development Centers, DOD Office of the Inspector General (95-048A,
Apr.  19, 1995). 

DOD's Federally Funded Research and Development Centers,
Congressional Research Service (95-489 SPR, Apr.  13, 1995). 

Report on Department of Defense Federally Funded Research and
Development Centers and Affiliated Organizations, Director of Defense
Research and Engineering (Apr.  3, 1995). 

Federally Funded R&D Centers:  Executive Compensation at The
Aerospace Corporation (GAO/NSIAD-95-75, Feb.  7, 1995). 

Contracting Practices for the Use and Operations of DOD-Sponsored
Federally Funded Research and Development Centers, DOD Office of the
Inspector General (95-048, Dec.  2, 1994). 

Survey of Management Controls Over the Use of Federally Funded
Research and Development Centers, Defense Information Systems Agency
(95A-04, Nov.  3, 1994). 

Government Contractors:  Contracting Out Implications of Streamlining
Agency Operations (GAO/T-GGD-95-4, Oct.  5, 1994). 

Sole-Source Justifications for DOD-Sponsored Federally Funded
Research and Development Centers, DOD Office of the Inspector General
(94-012, Nov.  4, 1993). 

DOD's Federally Funded Research and Development Centers,
Congressional Research Service (93-549 SPR, June 3, 1993). 

Inadequate Federal Oversight of Federally Funded Research and
Development Centers, Subcommittee on Oversight of Government
Management, Senate Governmental Affairs Committee (102-98, July
1992). 

Government Contractors:  Are Service Contractors Performing
Inherently Governmental Functions?  (GAO/GGD-92-11, Nov.  18, 1991). 

DOD's Federally Funded Research and Development Centers,
Congressional Research Service (91-378 SPR, Apr.  29, 1991). 

Competition:  Issues on Establishing and Using Federally Funded
Research and Development Centers (GAO/NSIAD-88-22, Mar.  7, 1988). 

Fee Guidelines Still Needed for Government-Sponsored Nonprofit
Organizations (PLRD-82-54, July 7, 1982). 

Management of the Federal Contract Research Centers, Director of
Defense Research and Engineering, Office of the Secretary of Defense
(June 1976). 

Report of the Special Study Group on Federal Contract Research
Centers, Director of Defense Research and Engineering, Office of the
Secretary of Defense (Aug.  30, 1971). 

Need for Improved Guidelines in Contracting for Research with
Government-Sponsored Nonprofit Contractors (B-146810, Feb.  10,
1969). 

Report to the President on Government Contracting for Research and
Development (U.S.  Senate, 87th Congress, 2nd Session, Document No. 
94, May 17, 1962).  This report, prepared by a presidentially
appointed committee led by Bureau of the Budget Director David Bell,
is commonly referred to as "The Bell Report."


INFORMATION ON DOD'S FFRDCS
========================================================= Appendix III

DOD currently sponsors 11 FFRDCs operated by 8 parent organizations. 
Table III.1 provides information on each FFRDC, including the parent
organization, primary sponsor, DOD funding, and members of technical
staff (MTS)\1 provided for fiscal year 1995. 



                                   Table III.1
                     
                     Information on DOD's FFRDCs (fiscal year
                                      1995)

                              (Dollars in millions)

                Parent
FFRDC           organization    Primary sponsor              Obligations     MTS
--------------  --------------  --------------------------  ------------  ------
Systems engineering and integration centers
--------------------------------------------------------------------------------
Aerospace       The Aerospace   Air Force                           $335   1,910
                Corporation

MITRE C\3I      MITRE           Assistant Secretary of               374   2,109
                Corporation     Defense (C\3I)

================================================================================
Subtotal                                                            $709   4,019


Studies and analyses centers
--------------------------------------------------------------------------------
Arroyo Center   RAND            Army                                  20      99

Project AIR     RAND            Air Force                             24     112
FORCE

National        RAND            OSD                                   19     105
Defense
Research
Institute

Center for      The CNA         Navy                                  47     238
Naval Analyses  Corporation

IDA--Studies    IDA             OSD                                   68     372
and Analyses/
Operational
Test and
Evaluation
Center

Logistics       Logistics       OSD                                   29     166
Management      Management
Institute       Institute

================================================================================
Subtotal                                                            $207   1,092


Research and development laboratories
--------------------------------------------------------------------------------
Lincoln         Massachusetts   Air Force                            275   1,017
Laboratory      Institute of
                Technology

Software        Carnegie-       Defense Advanced Research             29     170
Engineering     Mellon          Projects Agency
Institute       University

IDA--           IDA             National Security Agency              33     142
Communications
and Computing\

================================================================================
Subtotal                                                            $386   1,329

================================================================================
Total                                                             $1,253   6,440
--------------------------------------------------------------------------------
Note:  C\3 I, command, control, communication, and intelligence; OSD,
Office of the Secretary of Defense; and IDA, Institute for Defense
Analyses. 

Source:  OSD. 


--------------------
\1 MTS include the direct professional labor of scientists,
engineers, researchers, mathematicians, analysts, economists, and
others who perform professional-level technical work.  MTS is defined
as 1,810 hours of full-time professional effort; MTS does not include
subcontracted effort. 


COMPARISON OF FFRDCS' FORMER SCOPE
OF WORK DESCRIPTIONS AND CORE
COMPETENCIES
========================================================== Appendix IV

FFRDC           Scope of work                    Core competencies
--------------  -------------------------------  -------------------------------
Aerospace        Advanced systems                Launch certification.
                architecture; concept analysis    Systems of systems
                and planning; research,          engineering.
                development, test, and            Systemt development and
                evaluation; experimentation;     acquisition.
                and systems engineering and       Process implementation.
                integration.                      Technology application.
                 Mission-oriented
                investigation and
                experimentation.
                 Multiprogram systems
                engineering.
                 Foreign technology support.
                 Technical support,
                recommendations, and reviews.
                 Technical analysis, support,
                evaluation, and review in the
                field of U.S. national security
                space systems.
                 Long-range planning, systems
                analysis, and comparison
                studies, including technical
                risk management, cost, and
                schedule assessments.

MITRE C\3I       Systems research and            Systems of systems
                planning.                        engineering.
                 System design.                  Systems development and
                 Technical management.          acquisition.
                 Selected fields of research     Process implementation.
                and experimentation, including    Technology application.
                navigation, detection,            Architectures and
                surveillance, identification,    interoperability.
                threat evaluations, warning,
                weather, and intelligence.

Arroyo Center    Geopolitical environment and    Force development and
                its effects on the Army.         technology.
                 Implications of the external    Personnel and training.
                threat environment.               Military logistics.
                 Research in strategy,           Strategy and doctrine.
                military planning, and regional
                security.
                 Restructuring initiatives
                designed to make the Army more
                efficient.
                 Alternative technology
                applications and technical
                strategies.
                 Force composition, size, and
                operational concepts.
                 Interactive modeling and
                simulation capability.
                 Initiatives and designs for
                the Army of the future.
                 Logistics, sustainment, and
                redesign initiatives.

Project AIR      Strategy, doctrine, and force   Strategy and doctrine.
FORCE           structure.                        Force development and
                 Force modernization and        application.
                employment.                       Resource management.
                 Resource management and
                systems acquisition.

National         Research, studies, and          International policy and
Defense         analyses in areas such as        defense strategy.
Research        international security and        Forces and resources policy.
Institute       defense strategy, acquisition     Technology and acquisition
                and technology policy, and       policy.
                forces and resources policy.      Research integration.

Center for       Geopolitical security           Operations analysis.
Naval Analyses  environment.                      System requirements and
                 Roles, missions, and concepts  acquisition.
                of operations.                    Resource analysis.
                 Force planning and              Program planning.
                evaluation.                       Policy, strategy, and
                 System requirements.           doctrine.
                 Fleet tactics and
                capabilities.
                 Joint space and electronic
                warfare/command, control,
                communication, computers,
                intelligence, and information.
                 Cost and operational
                effectiveness analysis.
                 Research, development, and
                acquisition.
                 Infrastructure.
                 Personnel.
                 Medical.
                 Training.
                 Readiness, maintenance, and
                logistics.
                 Modeling and simulation.

IDA-Studies      Alternative technology          Systems evaluations.
and Analyses/   applications and technical        Technology assessments.
Operational     strategies.                       Force and strategy
Test and         Applications of advanced       assessments.
Evaluation      computing and information         Resource and support
Center          systems.                         analyses.
                 Evaluations of systems
                proposed for prototyping,
                development, and procurement.
                 Assessment of test and
                evaluation plans and results.
                 Exploitation of distributed,
                interactive simulation.
                 Force composition, size, and
                operational concepts.
                 Costs of defense programs,
                forces, and supporting
                infrastructure as well as
                analyses of acquisition
                procedures and methods.
                 Research in strategy--
                military planning, regional
                security, and related defense
                policy and management.
                 Evaluations of the readiness
                and performance of systems,
                forces, and military
                organizations, including
                training and support issues.
                 Implications of dual-use and
                international technology.
                 Joint and allied
                interoperability standards.

Logistics        Broad range of policy,          Materiel management.
Management      managerial, and technological     Acquisition.
Institute       issues in acquisition,            Operational logistics.
                logistics, and force              Facilities and environment.
                management.                       Force management.

Lincoln          Ballistic missile defense.      Ballistic missile defense.
Laboratory       Communications.                 Communications.
                 Space surveillance.             Space surveillance.
                 Air defense.                    Air defense.
                 Surface surveillance.           Surface surveillance.
                 Advanced electronics            Advanced electronics
                technology.                      technology.

Software         Technology transition.          Software engineering and
Engineering                                      supporting software
Institute                                        technology.
                                                  Technology transition.

IDA-             Long-term research on           Cryptologic mathematics.
Communications  mission-critical problems         Computing sciences.
and Computing   confronting the sponsor.          Basic communications theory.
                 Difficult operational
                problems of immediate
                importance associated with
                cryptomathematics and
                cryptocomputing.
--------------------------------------------------------------------------------



(See figure in printed edition.)Appendix V
COMMENTS FROM THE DEPARTMENT OF
DEFENSE
========================================================== Appendix IV


MAJOR CONTRIBUTORS TO THIS REPORT
========================================================== Appendix VI

NATIONAL SECURITY AND
INTERNATIONAL AFFAIRS DIVISION,
WASHINGTON, D.C. 

Charles W.  Thompson
Erin Slonaker Noel
Maria Boyreau

BOSTON FIELD OFFICE

Monty Peters


*** End of document. ***