Results Act: Observations on Federal Science Agencies (Testimony,
07/30/97, GAO/T-RCED-97-220).

GAO discussed the implementation of the Government Performance and
Results Act in federal science agencies, focusing on six agencies'
fulfillment of the requirements of the Results Act and the interagency
crosscutting science programs, activities, or functions that are similar
or complementary to those of other federal agencies. The agencies GAO
discussed are the Departments of Commerce, Energy, and Transportation as
well as the Environmental Protection Agency (EPA), National Science
Foundation (NSF), and the National Aeronautics and Space Administration
(NASA).

GAO noted that: (1) overall, the six agencies' draft strategic plans
show progress toward meeting the purposes of the act; however, only one
of the six agencies' plans contains all six of the act's critical
elements; (2) in addition, some of the completed elements were
insufficient; (3) for example, the goals and objectives were frequently
results-oriented, but it was unclear in all of the plans how some of the
goals would be measured; (4) developing effective performance measures
for these program goals will be a major challenge for science agencies;
(5) futhermore, five of the six plans did not include information on
past and future program evaluations and the one inclusion could be
improved; (6) because the draft plans do not contain all six elements,
the Congress is missing critical pieces of information for its
consultations with the agencies; (7) under Office of Management and
Budget guidance, the agencies' final submission should include a summary
of consultation efforts, including crosscutting activities; (8)
currently, the agencies' draft plans generally do not address how
crosscutting activities correspond with those of other agencies; (9) in
addition, the plans generally do not address whether such shared
responsibilities were coordinated in the development of the draft plans;
(10) however, some of the agencies' missions and goals do involve or
overlap those of other agencies; (11) despite the fact that the draft
plans do not reflect coordination activities, several agencies have
initiated efforts to coordinate crosscutting research programs
governmentwide; (12) coordination has occurred primarily at the program
level rather than at the senior management level which is necessary to
ensure consistency of program objectives among agencies; and (13) in
GAO's opinion, recognition of such coordination activities as part of
the final submission will be useful to the Congress in making funding
decisions that involve similar or complementary science programs.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  T-RCED-97-220
     TITLE:  Results Act: Observations on Federal Science Agencies
      DATE:  07/30/97
   SUBJECT:  Strategic planning
             Agency missions
             Interagency relations
             Congressional/executive relations
             Program evaluation
             Research program management
             Research and development
IDENTIFIER:  Performance measures
             
******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************


Cover
================================================================ COVER


Before the Committee on Science
House of Representatives

For Release
on Delivery
Expected at
10:00 a.m.  EDT
Wednesday
July 30, 1997

RESULTS ACT - OBSERVATIONS ON
FEDERAL SCIENCE AGENCIES

Statement by Susan Kladiva
Associate Director, Energy, Resources, and Science Issues
Resources, Community, and Economic
Development Division

GAO/T-RCED-97-220

GAO/RCED-97-220T


(141090)


Abbreviations
=============================================================== ABBREV

  GPRA -
  EPA -
  NSF -
  NASA -
  OMB -

============================================================ Chapter 0

Mr.  Chairman and Members of the Committee: 

We are pleased to be here today to discuss the implementation of the
Government Performance and Results Act, often referred to as the
Results Act or GPRA, in federal science agencies.  A focus on
results, as intended by the Results Act, is aimed at improving the
efficiency and effectiveness of the federal government.  In the
science and technology area, where more than 20 agencies spent $60
billion in fiscal year 1996, the potential for unnecessary overlap is
particularly high and close coordination is essential.  While some
shared responsibilities are fine, uncoordinated program efforts can
waste scarce funds, confuse and frustrate program customers, and
limit the overall effectiveness of the federal effort.  Coordination
among federal programs contributing to the same or similar results
can ensure that goals are consistent and, as appropriate, program
efforts are mutually reinforcing. 

As agreed with the Committee, my statement will focus on six
agencies' fulfillment of the requirements of the Results Act and the
interagency coordination of crosscutting science programs,
activities, or functions that are similar or complementary to those
of other federal agencies.  Regarding the requirements of the Results
Act, we focused on two elements--(1) agencywide goals and objectives
and (2) past and future program evaluations.  The six agencies I will
discuss include the Departments of Commerce, Energy, and
Transportation, as well as the Environmental Protection Agency (EPA),
the National Science Foundation (NSF), and the National Aeronautics
and Space Administration (NASA).  My statement is based on our review
of the agencies' draft strategic plans, discussions with agency
officials, and our general knowledge of the science agencies. 

In summary, Mr.  Chairman: 

Overall, the draft strategic plans show progress toward meeting the
purposes of the Act; however, only one of the six agencies' plans
contains all six of the Act's critical elements.  In addition, some
of the completed elements were insufficient.  For example, the goals
and objectives were frequently results-oriented, but it was unclear
in all of the plans how some of the goals would be measured. 
Developing effective performance measures for these program goals
will be a major challenge for science agencies.  Furthermore, five of
the six plans did not include information on past and future program
evaluations and the one inclusion could be improved.  Because the
draft plans do not contain all six elements, the Congress is missing
critical pieces of information for its consultations with the
agencies. 

Under Office of Management and Budget (OMB) guidance, the agencies'
final submission should include a summary of consultation efforts,
including crosscutting activities.  Currently, the agencies' draft
plans generally do not address how crosscutting activities correspond
with those of other agencies.  In addition, the plans generally do
not address whether such shared responsibilities were coordinated in
the development of the draft plans.  However, some of the agencies'
missions and goals do involve or overlap those of other agencies. 
Despite the fact that the draft plans do not reflect coordination
activities, several agencies have initiated efforts to coordinate
crosscutting research programs governmentwide.  But coordination has
occurred primarily at the program level rather than at the senior
management level which is necessary to ensure consistency of program
objectives among agencies.  In our opinion, recognition of such
coordination activities as part of the final submission will be
useful to the Congress in making funding decisions that involve
similar or complementary science programs. 

Now, I will briefly discuss these issues within the context of each
of the six agencies.  More details are provided in our reports on
each of these agencies' implementation of the Results Act, as listed
in appendix I to my statement. 


   BACKGROUND
---------------------------------------------------------- Chapter 0:1

The Results Act is intended to improve the efficiency and
effectiveness of federal programs by establishing a process to set
goals for program performance and to measure results.  As a starting
point, the Act requires virtually every executive agency to develop a
strategic plan covering at least 5 years.  It also requires that an
agency's strategic plan contain the following six critical elements: 
a mission statement; agencywide goals and objectives; the strategies
and resources needed to achieve the goals and objectives; the
relationship between the long-term goals and objectives and the
annual performance goals; the key external factors that could affect
the achievement of goals; and a description of how program
evaluations were and will be used to establish or revise strategic
goals. 

When developing a strategic plan, the Results Act requires that the
agency shall consult with the Congress and shall solicit and consider
the views and suggestions of those entities potentially affected by
or interested in such a plan.  Furthermore, guidance from OMB states
that when general goals and objectives have crosscutting functions,
programs, or activities, agencies may have a shared responsibility
for defining and achieving them.  Thus, agencies should ensure that
appropriate and timely consultation occurs with other agencies during
the development of strategic plans with crosscutting goals and
objectives.  Though the Act does not include a requirement that the
draft strategic plans should contain a description of how the
activities of an agency relate and will be coordinated with the
activities of other agencies with similar programs, OMB guidance does
provide that the letter transmitting the strategic plan include a
summary of the general scope and nature of the consultation and the
types of entities consulted.  In addition, OMB guidance on
preparation of annual performance plans beginning in fiscal year
1999, states that agencies should indicate those goals and indicators
that are being mutually undertaken in support of programs or
activities of an interagency, crosscutting nature. 

It is important to recognize that the final strategic plans are not
due to the Congress and OMB until September 1997.  Furthermore, the
Results Act anticipated that it may take several planning cycles to
perfect the process and that the final plans would be continually
refined as future planning cycles occur.  Thus, our comments reflect
a snapshot of the plans when they were submitted to the Congress,
between May and July, and are intended to provide insights that may
help this Committee and the agencies work together to successfully
implement the Results Act.  We also recognize that the agencies are
continuing to revise their strategic plans with input from OMB,
congressional staff, and other stakeholders. 


   NATIONAL SCIENCE FOUNDATION
---------------------------------------------------------- Chapter 0:2

While NSF addresses five of the six required elements of the Results
Act, at least four of them need further development, and the sixth
element--key external factors--is not included in its draft strategic
plan.\1 Although NSF defines goals and objectives in the draft plan,
many of these goals are not expressed in a measurable form, making it
unclear whether the Foundation and the Congress will be able to
assess whether the goals are achieved.  NSF's draft plan also does
not discuss how the agency used specific program evaluations to
develop its strategic goals or the other components of the plan. 
Further details are needed on a schedule for future evaluations, the
scope of and methodology for future evaluations, and how the findings
could be useful in assessing NSF's goals and performance plans. 

The draft plan acknowledges the crosscutting nature of NSF's work but
does not show evidence of interagency coordination.  It emphasizes
the importance of NSF's many partners in the research and education
enterprise but does not identify these partners or provide sufficient
information to determine the extent to which NSF and its partners'
functions are duplicative or overlapping.  However, we identified
similarities of mission in the draft plans developed by NSF, Energy,
and Commerce.  For example, the Foundation's mission includes
promoting the progress of science, and one of its overarching goals
is to enable the United States to uphold a position of world
leadership in all aspects of science, mathematics, and engineering. 
Similarly, Energy's science mission is to ensure that the United
States retains its leadership in science and technology.  Also
comparable is Commerce's mission, which includes keeping America
competitive with cutting-edge science and technology. 

Although not identified in NSF's draft plan, agency officials cite
efforts to coordinate the crosscutting areas.  For example, NSF has
shared its draft plan on the Internet.  Furthermore, according to
NSF's Assistant to the Director for Science Policy and Planning, the
Foundation participates in a number of groups such as the National
Science and Technology Council, the Committee on Fundamental Science,
and the Research Roundtable.  While one purpose of these groups is to
coordinate, NSF's draft plan does not mention these groups or the
frequency of their discussions.  We have found that although these
agency forums have provided an important opportunity for agencies to
work together to address common concerns in goal setting and
performance measurement, they have not generally attempted to
coordinate crosscutting program efforts. 


--------------------
\1 We reviewed the draft strategic plan dated June 9, 1997. 


   DEPARTMENT OF ENERGY
---------------------------------------------------------- Chapter 0:3

Energy's draft plan fully addresses only two of the six required
elements--the mission statement and the goals and objectives--and
partially addresses a third--the strategies element.\2 We found that
the goals and objectives cover the agency's major functions and
operations, and that the goals are generally results-oriented. 
However, in reviewing the multiple objectives for each goal, we
identified several objectives that were stated in ways that will make
it difficult to measure whether they are being achieved. 
Furthermore, Energy did not complete the element related to the
impact of program evaluations on the development of strategic goals. 
The Department is finalizing these elements and expects to include
them in the final plan. 

Although Energy does not mention crosscutting programs or
coordination efforts in its draft plan, the Department is sharing its
draft plan with other federal agencies.  Energy's draft plan does not
identify programs and activities that are crosscutting or similar to
those of other federal agencies, primarily because the Department
believes its functions are unique.  On the basis of our work,
however, we believe that Energy's broad missions do involve or
overlap those of other agencies.  As previously discussed, we have
identified the potentially overlapping missions of Energy, Commerce,
and NSF.  Another example of potential overlap is in science
education.  Energy's draft strategic plan states that it will use its
laboratories and the nation's universities to contribute to the
nation's science and mathematics education.  Similarly, NSF's
authorizing legislation directs it to initiate and support science
and engineering education programs at all levels and in all fields of
science and engineering. 

Environmental quality is another area of potential overlap.  The
science and technology portion of Energy's draft plan contains a
strategy to conduct high-quality research on environmental quality. 
Similarly, one of EPA's goals is to develop and apply the best
available science for addressing current and future environmental
hazards as well as new approaches toward improving environmental
protection. 


--------------------
\2 We reviewed the draft strategic plan dated June 16, 1997. 


   DEPARTMENT OF COMMERCE
---------------------------------------------------------- Chapter 0:4

Commerce's draft strategic plan contains four of the six critical
elements, but each of the four, including the goals and objectives
element, has weaknesses.\3 While there are linkages among themes,
goals, objectives, and responsible components, the goals and
objectives are not as results-oriented as they could be.  For
example, the goal to support restructuring export controls for the
21st century could be made more results-oriented by identifying the
purpose of the restructuring.  Moreover, the draft plan does not
explicitly discuss the program evaluations used to establish general
goals and objectives and has no schedule for future program
evaluations. 

Commerce's draft plan also does not identify crosscutting programs
and activities or whether such shared responsibilities were
coordinated in the development of the draft plan.  The draft plan
does describe, in very general terms, some of the existing
partnerships between Commerce agencies and various public and private
entities.  However, as stated earlier, we identified mission overlap
between NSF, Energy, and Commerce.  Another example of potential
overlap occurs in Commerce's grant program to promote the use of
advanced telecommunications in the public and nonprofit sectors.  A
number of other federal agencies, including NSF, also support
telecommunications projects for similar constituencies. 


--------------------
\3 We reviewed the draft strategic plan dated June 1997. 


   NATIONAL AERONAUTICS AND SPACE
   ADMINISTRATION
---------------------------------------------------------- Chapter 0:5

Of the six elements required by the Results Act, NASA includes four
in its draft strategic plan.  The two that are not explained in
enough detail are the relationship between NASA's long-term goals and
annual performance goals and a description of program evaluations.\4
While the goals outlined by NASA appear to meet the Results Act's
requirements, progress toward some of the goals, such as the goal to
acquire and encourage knowledge and technologies that promote our
quality of life, may prove difficult to assess.  As we recently
reported, it is inherently difficult to measure the performance of
research and development programs because the results are typically
not apparent until many years later.\5 The draft does not explicitly
discuss or demonstrate how program evaluations were used in
establishing or revising agency goals and objectives, nor does it
provide a schedule for future evaluations. 

NASA's draft plan also does not identify specific programs and
activities that are crosscutting or similar to those of other federal
agencies, and the plan does not address interagency coordination. 
Many of NASA's objectives, however, are shared with other agencies. 
For example, one of NASA's objectives related to long-term climate
and ozone research involves NSF, the Department of Energy, and
Commerce's National Oceanic and Atmospheric Administration. 
Moreover, like NSF's mission of advancing scientific knowledge, one
of NASA's missions is to advance and communicate scientific knowledge
and understanding of the Earth, the solar system, and the environment
of space for research. 

The draft plan does not address what interagency coordination
occurred to address the shared responsibilities.  However, the draft
plan does note the importance of working with other agencies in
achieving its objectives, and NASA officials stated that coordination
has occurred at the program level.  NASA officials stated that each
strategic enterprise coordinated its objectives with the relevant
agencies at the program level.  They noted that NASA officials
participate in many crosscutting groups, like the Research
Roundtable, where programmatic objectives are discussed.  They also
noted that for the last few years, the agency has shared its
strategic plan with other federal agencies. 


--------------------
\4 We reviewed the draft strategic plan dated May 1997. 

\5 Measuring Performance:  Strengths and Limitations of Research
Indicators (GAO/RCED-97-91, Mar.  21, 1997). 


   ENVIRONMENTAL PROTECTION AGENCY
---------------------------------------------------------- Chapter 0:6

EPA's draft plan contains four of the six elements required by the
Results Act, certain aspects of which could be improved.\6 Although
the goals and objectives--one of the completed elements--are
generally results-oriented and measurable, some do not clearly define
the expected results, and it is unclear how EPA would assess progress
toward achieving them.  Furthermore, the large number of goals (10),
objectives (45), and subobjectives (200) may make it difficult for
the Congress and others to discern EPA's priorities--what will be
most important to the agency over the next several years.  The two
missing elements are (1) the relationship between EPA's long-term
goals and the annual performance goals and (2) program evaluations
used in developing the plan and a schedule for future evaluations. 

EPA's draft plan does not discuss the agency's programs and
activities that are crosscutting or similar to those of other federal
agencies, but it does address the need for coordination with its
stakeholders, which include federal entities.  Our past work has
found that EPA--as the central federal agency responsible for
safeguarding the environment--carries out a number of mission-related
activities that are crosscutting or similar to those of other federal
agencies.  For example, as we discussed previously, EPA and Energy
both conduct research on environmental quality.  In addition, EPA
shares responsibilities with other agencies for collecting and
managing the data needed to perform environmental assessments.  Thus,
EPA must coordinate data for health assessments with a number of
different agencies, including NSF. 

EPA's draft plan makes broad reference to the need for coordination
with federal agencies as well as other stakeholders to accomplish its
mission.  As such, EPA is currently taking steps to coordinate its
plan with other agencies to address crosscutting programs and
activities.  To further refine its plan and determine areas of
potential overlap between EPA and federal agencies with related
responsibilities, EPA identified 16 federal agencies with
crosscutting or similar functions and sent each of them a draft
outline for the strategic plan in May 1997, and the full draft in
early July 1997, for their review and comment.  In addition, EPA is
reviewing these agencies' draft plans to identify areas in which
duplication of activities exists and further coordination is
warranted.  According to EPA officials, the agency is including in
its coordination all of the other five science agencies discussed in
this testimony. 


--------------------
\6 We reviewed the draft strategic plan dated July 1997. 


   DEPARTMENT OF TRANSPORTATION
---------------------------------------------------------- Chapter 0:7

Although Transportation's draft plan includes all six critical
elements, we believe that only three of the six meet the Act's
requirements, including the mission statement, long-term goals, and a
description of program evaluations.\7 The five long-term goals
encompass the Department's major functions and operations and are
within the Department's span of influence.  However, they could be
improved by stating all goals in a measurable form to allow for
future assessments of their achievement.  Furthermore, they are
stated in general terms whereas sufficiently precise goals can better
direct and guide agency staff toward fulfilling the agency's mission. 
The program evaluation information is insufficient to determine the
scope and methodology or the key issues to be addressed.  Without
this information, it is difficult to determine how or if scheduled
evaluations relate to Transportation's goals. 

Transportation's draft plan does not identify the crosscutting
activities nor show evidence of interagency coordination.  Supporting
documents that Transportation used to prepare its draft plan indicate
that the Department considered a number of crosscutting issues but
did not include the information in the plan.  Again, however, one of
the science-related strategies outlined in Transportation's draft
strategic plan focuses on research and technology to foster economic
growth and enable research and education.  Transportation's draft
plan recognizes that there are other federal stakeholders and
provides for establishing partnerships.  However, it is silent on
whether the Department coordinated with other federal agencies that
have programs and activities that are crosscutting or similar to
Transportation's.  According to a senior Transportation official,
coordination is an ongoing activity and no specific coordination was
done for the purposes of preparing the draft plan. 


--------------------
\7 We reviewed the draft strategic plan provided to congressional
committees on July 2, 1997. 


-------------------------------------------------------- Chapter 0:7.1

In conclusion, the Results Act process provides an opportunity for
agencies to begin improving the coordination and execution of
crosscutting science issues across the federal government.  The lack
of coordinated performance goals that are results-oriented may be
problematic as the Congress begins to evaluate programs and
activities that are crosscutting among the science agencies.  In
addition, fully resolving the problem of potential overlap and
duplication will take time and require sustained leadership by this
Committee, OMB, and agency senior management to ensure that science
agencies coordinate their efforts and develop coordinated goals among
crosscutting programs and activities.  Congressional consultations on
agencies' strategic plans provide an ongoing opportunity for the
Congress and the executive branch to work together to minimize the
extent and potential consequences of overlap and fragmentation in
federal program efforts.  Special attention devoted to crosscutting
issues and coordination activities in the strategic plans will
improve this consultation process.  Congressional oversight, such as
is occurring here today, is also key. 

This concludes my statement.  I would be pleased to respond to any
questions you or the Members of the Committee may have. 

RELATED GAO REPORTS

Results Act:  Observations on the National Science Foundation's Draft
Strategic Plan (GAO/RCED-97-203R, July 11, 1997). 

Results Act:  Observations on the Department of Energy's Draft
Strategic Plan (GAO/RCED-97-199R, July 11, 1997). 

The Results Act:  Observations on Commerce's June 1997 Draft
Strategic Plan (GAO/GGD-97-152R, July 14, 1997). 

Results Act:  Observations on NASA's May 1997 Draft Strategic Plan
(GAO/NSIAD-97-205R, July 22, 1997). 

Results Act:  Observations on Environmental Protection Agency's Draft
Strategic Plan (GAO/RCED-97-209R, July 30, 1997). 

Results Act:  Observations on the Department of Transportation's
Draft Strategic Plan (GAO/RCED-97-208R, July 30, 1997). 


*** End of document. ***