Observations on the Fiscal Year 1999 Annual Program Performance Report
and Fiscal Years 2000 and 2001 Annual Performance Plans for Selected
Science Agencies Within the Department of Commerce (Correspondence,
09/25/2000, GAO/GGD-00-197R).

Pursuant to a congressional request, GAO reviewed the Department of
Commerce's fiscal year (FY) 1999 Annual Program Performance Report and
the FY 2000 and FY 2001 plans for selected science-related agencies,
focusing on whether: (1) the National Oceanic and Atmospheric
Administration (NOAA) and the Technology Administration (TA) sections of
Commerce's FY 1999 performance report and FY 2000 and FY 2001
performance plans told a coherent story in addressing science issues;
(2) NOAA's and TA's performance goals, budgetary and full time
equivalent (FTE) staffing resource requirements identified in Commerce's
FY 2001 performance plan linked, or matched, to those in NOAA's and TA's
sections of Commerce's FY 2001 congressional budget submission; and (3)
the NOAA and TA sections of Commerce's FY 2001 performance plan
addressed the weaknesses that GAO identified in its review of Commerce's
FY 2000 performance plan.

GAO noted that: (1) the NOAA and TA sections of Commerce's performance
report and performance plans address science issues in a generally
coherent and consistent manner; (2) NOAA and TA described their
performance goal strategies or objectives, supporting activities, and
crosscutting activities in the FY 2000 and FY 2001 plans, and these
descriptions also remained essentially unchanged; (3) one difference
between the two plans was that an additional goal of protecting the
nation's information infrastructure was added to TA's section of the FY
2001 plan; (4) also, both NOAA and TA expanded their discussions from
the previous year's plan on departmentwide or bureau-specific
initiatives; (5) significant changes were made to the performance
measures and resource requirements and that these changes were not
always explained in the Government Performance and Results Act (GPRA)
documents; (6) although GPRA guidance does not require that such changes
be explained, without explanations the rationale for the changes that
were made and the impact that they would have on addressing science
issues was not clear; (7) no performance measures were established for a
performance goal; (8) additionally, performance measures were not always
established for NOAA's or TA's contribution to departmentwide
initiatives, nor were measures established for their respective
bureau-specific initiatives; (9) GAO's review of the NOAA and TA
sections of the FY 2001 plan and their sections of the FY 2001 budget
submission showed that performance goals, budgetary resources, and FTE
levels were the same in those two documents in most cases but differed
to some degree in other cases; (10) for TA, GAO found direct linkages
for six of its seven performance goals, but not for each goals'
supporting objectives or activities; (11) for both NOAA and TA, there
were no linkages for any of their respective bureau-specific
initiatives; (12) GAO's review did not address the GPRA requirement that
the performance plan itself cover each program activity contained in the
budget, and GAO did not review whether the budgetary implications of the
plan itself were clear and transparent; (13) based on GAO's examination
of its FY 2001 plan, GAO believes Commerce has addressed some, but not
all, of the weaknesses GAO identified when examining the previous year's
plan; and (14) those weaknesses GAO identified were in the areas of
strategies, resources, and performance data verification and validation,
as well as in performance measures.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  GGD-00-197R
     TITLE:  Observations on the Fiscal Year 1999 Annual Program
	     Performance Report and Fiscal Years 2000 and 2001 Annual
	     Performance Plans for Selected Science Agencies Within the
	     Department of Commerce
      DATE:  09/25/2000
   SUBJECT:  Performance measures
	     Strategic planning
	     Agency missions
	     Reporting requirements
	     Mission budgeting
IDENTIFIER:  GPRA
	     Government Performance and Results Act

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Testimony.                                               **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO/GGD-00-197R

United States General Accounting Office General Government Division
Washington, D. C. 20548

Page 1 GAO/ GGD- 00- 197R GPRA Plans For Commerce's Science Agencies

B- 286123 September 25, 2000 The Honorable F. James Sensenbrenner, Jr.
Chairman, Committee on Science House of Representatives

Subject: Observations on the Fiscal Year 1999 Annual Program Performance
Report and Fiscal Years 2000 and 2001 Annual Performance Plans for Selected
Science Agencies Within the Department of Commerce

Dear Mr. Chairman: In response to your April 11, 2000, request and based
upon subsequent discussions we had with your office, this letter provides
our analysis of the Department of Commerce's Fiscal Year 1999 Annual Program
Performance Report (report) and the fiscal year 2000 and fiscal year 2001
plans (plan) for selected science- related agencies. These documents were
required by the Government Performance and Results Act (GPRA). The
performance plans are required to identify annual goals and measurable
target levels for performance. The performance report is required to show
how actual performance in a particular fiscal year compared with performance
as described in that year's performance plan. Together with the GPRA-
required strategic plan, these documents provide the framework for agencies
to manage for results. Commerce is currently revising its strategic plan,
which is to cover fiscal years 2000 to 2005. According to Commerce, the
upcoming fiscal year 2002 annual performance plan is expected to fully
reflect the revised strategic goals and measures.

In conducting our analysis, we focused on the sections of the performance
report and performance plans that dealt with Commerce's two major science
agencies, the National Oceanic and Atmospheric Administration (NOAA) and the
Technology Administration (TA). Together, the budget requests for these two
agencies accounted for approximately 67 percent of Commerce's overall fiscal
year 2001 budget request of $5.4 billion. The objectives of our work were to
determine (1) whether the NOAA and TA sections of Commerce's fiscal year
1999 performance report and fiscal years 2000 and 2001 performance plans
told a coherent story in addressing science issues, (2) whether NOAA's and
TA's performance goals, budgetary and full time equivalent (FTE) staffing
resource requirements identified in Commerce's fiscal year 2001 performance
plan linked, i. e., matched, to those in NOAA's and

B- 286123 Page 2 GAO/ GGD- 00- 197R GPRA Plans For Commerce's Science
Agencies

TA's sections of Commerce's fiscal year 2001 congressional budget
submission, 1 and (3) whether the NOAA and TA sections of Commerce's fiscal
year 2001 performance plan addressed the weaknesses that we identified in
our review of Commerce's fiscal year 2000 performance plan. 2 Detailed
information about the objectives, scope and methodology of our work, as well
as comments provided by Commerce, are discussed near the end of this letter.

Results in Brief The NOAA and TA sections of Commerce's performance report
and performance plans address science issues in a generally coherent and
consistent manner. For example, NOAA's and TA's mission statements and
performance goals remained essentially unchanged in each of the three GPRA
documents. Further, NOAA and TA described their performance goal strategies
or objectives, supporting activities, and crosscutting activities in the
fiscal year 2000 and fiscal year 2001 plans, and these descriptions also
remained essentially unchanged. One difference between the two plans,
however, was that an additional goal of protecting the nation's information
infrastructure was added to TA's section of the fiscal year 2001 plan. Also,
both NOAA and TA expanded their discussions from the previous year's plan on
departmentwide or bureau- specific initiatives. We did note, however, that
significant changes were made to the performance measures and resource
requirements and that these changes were not always explained in the GPRA
documents. Although GPRA guidance does not require that such changes be
explained, without explanations the rationale for the changes that were made
and the impact that they would have on addressing science issues was not
clear. Also, in one case, no performance measures were established for a
performance goal. Additionally, performance measures were not always
established for NOAA's or TA's contribution to departmentwide initiatives,
nor were measures established for their respective bureau- specific
initiatives. As a result, it was not always easy to determine the progress
NOAA and TA had made towards achieving their goals or in contributing to
departmentwide efforts.

Our review of the NOAA and TA sections of the fiscal year 2001 plan and
their sections of the fiscal year 2001 budget submission showed that
performance goals, budgetary resources and FTE levels were the same in those
two documents in most cases but differed to some degree in other cases. For
example, we found direct linkages between the performance plan and budget
submission for NOAA's performance goals and their supporting objectives or
activities. For TA, we found direct linkages for six of its seven
performance goals, but not for each goal's supporting objectives or
activities. And, for both NOAA and TA, there were no linkages for any of
their respective bureau- specific initiatives. However, our review did not
address the GPRA requirement that the performance plan itself cover each
program activity contained in the budget, and we did not review whether the
budgetary implications of the plan itself were clear and transparent.

1 See the objectives, scope and methodology section towards the end of this
letter for limitations in scope related to this second objective. 2
Observations on the Department of Commerce's Fiscal Year 1999 Annual Program
Performance Report and Fiscal Year 2001 Annual Performance Plan (GAO/ GGD-
00- 152R, June 30, 2000).

B- 286123 Page 3 GAO/ GGD- 00- 197R GPRA Plans For Commerce's Science
Agencies

Based on our examination of its fiscal year 2001 plan, we believe Commerce
has addressed some, but not all, of the weaknesses we identified when
examining the previous year's plan. Those weaknesses we identified were in
the areas of strategies, resources, and performance data verification and
validation, as well as in performance measures.

This letter contains recommendations to the Secretary of Commerce to (1)
include explanatory information in its GPRA documents when significant
changes are made to agency performance goals, objectives or activities,
performance measures and resource requirements and (2) direct that Commerce
agencies better link their sections of the GPRA documents to congressional
budget submissions and other budget documents.

In written comments on a draft of this letter, Commerce stated that the
letter raised a number of valid points that Commerce will consider in
refining its next strategic plan and its future annual performance plans and
reports. Commerce also indicated that it will provide explanations when
major changes have been made to performance measures or goals. More specific
comments attributed to NOAA and TA included those agencies' concurrence that
better linkages between performance data and budget data could be made. In
addition, TA raised two questions and stated that it would be pleased to
contribute to any future discussions on the questions that Congress and the
administration might hold. The questions asked whether agencies should be
required to fully explain and document all changes to their performance
measures and whether measures should be developed and reported on for all
initiatives.

Background The larger of Commerce's two major science agencies is NOAA,
whose fiscal year 2001 budget submission of $2.9 billion represented about
54 percent of the total Commerce budget request. NOAA's mission is to
describe and predict changes in the Earth's environment, as well as conserve
and manage coastal and marine resources to ensure sustainable economic
opportunities. Its components consist of the National Environmental
Satellite, Data and Information Service; the National Marine Fisheries
Service; the National Ocean Service; the National Weather Service; and
Oceanic and Atmospheric Research. NOAA carries out its mission by conducting
research aimed at developing new technologies, improving operations, and
supplying the scientific basis for managing natural resources and solving
environmental problems. NOAA acquires observations from many sources,
ranging from satellites and radar to ships and submersibles. According to
NOAA, these sources provide critical data and quality information needed for
the safe conduct of daily life and the basic functioning of a modern
society. NOAA's products and services include short- term weather forecasts,
seasonal climate predictions, long- term global change prognoses,
environmental technologies, nautical charts, marine fisheries statistics and
regulations, assessments of environmental changes, and hazardous materials
response information.

The smaller of the two science agencies, TA, prepared a congressional budget
submission for fiscal year 2001 of $722 million, or about 13 percent of the
total Commerce budget request. According to TA, its mission is to work with
U. S. industry to maximize the technological

B- 286123 Page 4 GAO/ GGD- 00- 197R GPRA Plans For Commerce's Science
Agencies

contribution to the nation's economy, serving as the executive branch focal
point for civilian technology and competitiveness issues. TA says that it
fulfills its mission by maintaining and improving key components of the
technological infrastructure in the United States; fostering the
development, diffusion, and adoption of new technologies and leading
business practices; creating a business and policy environment conducive to
innovation; and disseminating technical information. TA is composed of the
following: the National Institute of Standards and Technology (NIST), the
National Technical Information Service (NTIS), and the Office of Technology
Policy (OTP). TA also houses the Office of Air and Space Commercialization,
which coordinates Commerce's efforts to foster commercial space development
and encourages private sector investment in space.

NOAA's and TA's Sections of the GPRA Plans and Report Address Science Issues
in a Generally Coherent Manner, but Measuring Progress in Attaining Science-
Related Performance Goals Could Be Improved

The NOAA and TA sections of the performance report and performance plans
address science issues in a generally coherent and consistent manner. For
example, NOAA's and TA's mission statements and performance goals remained
essentially unchanged in each of the three GPRA documents. Further, NOAA and
TA listed their performance goal strategies or objectives, supporting
activities, and crosscutting activities in the fiscal years 2000 and 2001
plans, and we noted that these descriptions also remained essentially
unchanged. One difference between the two plans was that an additional goal
of protecting the nation's information infrastructure was added to TA's
section of the fiscal year 2001 plan. Also, both NOAA and TA expanded their
discussions on departmentwide or bureau- specific initiatives.

We did note that significant changes had been made to the performance
measures and resource requirements, and that these changes were not always
explained in the GPRA documents. Although GPRA guidance does not require
that such changes be explained, in the absence of explanations, the
rationale for the performance measure and resource changes that were made
and the impact that they would have on addressing science issues was not
clear. We also noted that, in one case, no performance measures were
established for a performance goal. Additionally, performance measures were
not always established for NOAA's or TA's contribution to departmentwide
initiatives, nor were measures established for their respective bureau-
specific initiatives. As a result, it will not always be easy to determine
the progress NOAA and TA make towards achieving their goals or contributing
to departmentwide efforts.

Performance Measures and Resources Were Changed Without Adequate Explanation

We identified 10 performance measures established by NOAA or TA that had
been dropped or added with little or no explanation provided. We also noted
instances in which-- for specific

B- 286123 Page 5 GAO/ GGD- 00- 197R GPRA Plans For Commerce's Science
Agencies

performance goals-- the requested appropriation amounts identified in the
fiscal year 2000 plan differed significantly from the enacted appropriation
amounts identified in the fiscal year 2001 plan. Yet, the fiscal year 2001
plan offered no explanations as to why this happened or what impact the
changed resource levels might have on the attaining of performance goals.
Finally, we identified one TA project that had been dropped from the
subsequent year's plan with no explanation as to why. While the Office of
Management and Budget's (OMB) Circular No. A- 11, Part 2, which provides
guidance for preparing GPRA documents, does not require explanations for
such changes, the lack of such an explanation in this case makes it more
difficult to understand what progress has been made on science issues.

In the fiscal year 2000 plan, NOAA's goal of promoting safe navigation
contained a performance measure for the “Number of Physical
Oceanographic Real- Time Systems (PORTS) in Place to Provide Quality-
Assured Data in Real- Time for Safe Navigation.” According to NOAA,
PORTS is an information system that, in part, furnishes real- time
navigational data to commercial and recreational mariners. In the plan, NOAA
established a performance target of increasing the number of these systems
by three each year. This measure was dropped from the fiscal year 2001 plan
without explanation, and a new measure for NOAA's goal of safe navigation
was added. The new measure relates to the completion of the National Spatial
Reference System (NSRS)-- a system of reference stations and monuments
across the country-- that is tied to the global positioning system (GPS).
The NSRS helps provide integrity to geographic coordinates obtained from GPS
satellite signals used, in part, for surveying, navigation, and mapping.

In another case, in the fiscal year 2001 plan, a measure--“ Results of
90% of the research activities cited in the 2001 IPCC Third Assessment of
Climate Change”-- was added for NOAA's performance goal of predicting
and assessing decadal to centennial change. 3 NOAA provided no explanation
as to why this measure was added. Further, we found the measure's title to
be unclear, no explanation offered for why the 90 percent target had been
selected, and no discussion included on how the results of the measure's
research activities would be used.

We also noted that, in some cases, changes were made from one plan to
another in the amounts of funding and FTE resources identified for specific
performance goals. These changes were not explained. For example, according
to the fiscal year 2000 plan, NOAA identified a total of $324 million and
1,661 FTEs needed for activities supporting its goal to build sustainable
fisheries. The fiscal year 2001 plan showed that NOAA actually received an
appropriation of $441.6 million and 2,371 FTEs for that effort. This
represented increases of $117.6 million (36 percent) and 710 FTEs (43
percent), yet no explanation for these significant increases was provided in
the plan to help the reader understand how the increases affected the
attaining of the goal.

In another example, according to the fiscal year 2000 plan, NOAA identified
a total of $218 million and 608 FTEs needed for activities supporting its
goal of sustaining healthy coasts.

3 The IPCC is the Intergovernmental Panel on Climate Change which
periodically assesses scientific, technical, and economic information in
evaluating interferences in the climate system.

B- 286123 Page 6 GAO/ GGD- 00- 197R GPRA Plans For Commerce's Science
Agencies

The fiscal year 2001 plan showed that NOAA actually received an
appropriation of $278.7 million and 867 FTEs. This represented increases of
$60.7 million (28 percent) and 259 FTEs (43 percent), yet no explanation for
these increases and how they affected performance levels was provided.
Furthermore, according to the fiscal year 2001 plan, NOAA identified an
additional increase in these amounts to $486.8 million and 912 FTEs for
fiscal year 2001. Again, no explanation for how these increases would affect
performance was provided.

During a meeting with Commerce officials in which we discussed these two
examples, we were told that the differences between the appropriation
amounts requested and enacted were not as significant as they appeared. The
officials said that the budget and FTE amounts shown in the fiscal year 2000
plan-- which purported to represent the total amounts required for the
respective performance goals-- were erroneous in each case. 4 They explained
that the budget and FTE amounts shown in the plan in each case represented
appropriation requests only for a NOAA component organization. In fact,
according to NOAA officials, the amounts portrayed in the plan as the total
resource requirements needed for each of the performance goals were
understated because they did not include the budget and FTE requirements of
the other contributing Commerce organizations.

In the case of TA, according to the fiscal year 2000 plan, a project to
foster the professional development of mathematics and science teachers in
elementary, middle, and secondary schools was to begin as an activity
supporting Commerce's priority initiative of building upon America's
technology infrastructure. According to the plan, although required FTEs
were not identified, the project was shown to require a $500,000
appropriation. The fiscal year 2001 plan did not include this project, nor
did it explain why the project was dropped.

Performance Measures Not Always Established According to Circular No. A- 11,
Part 2, performance measures must be included for performance goals that are
not self- measurable. In most cases, NOAA and TA complied with this
requirement. However, we did note one instance in which a performance goal
was not self- measurable, yet it had no performance measures. Commerce's
2001 plan also identified several departmentwide initiatives, and for most
of those initiatives Commerce identified bureau- specific performance
measures. However, we noted one instance in which bureauspecific performance
measures for a departmentwide initiative had not been established. Finally,
NOAA and TA identified their own bureau- specific initiatives, but did not
identify related performance measures. Circular No. A- 11, Part 2, does not
specifically discuss the treatment of departmentwide or bureau- specific
initiatives. However, establishing performance measures for all such
initiatives, or explaining how these initiatives relate to existing
performance goals and measures, could help the reader to learn whether
progress has been made in all performance areas.

4 According to NOAA officials, the correct total amounts that should have
been shown in the fiscal year 2000 plan for the goal of building sustainable
fisheries were $426. 3 million and 2,371 FTEs. For the goal of sustaining
healthy coasts, they said the correct total amounts that should have been
shown in the plan were $315 million and 944 FTEs.

B- 286123 Page 7 GAO/ GGD- 00- 197R GPRA Plans For Commerce's Science
Agencies

We noted that, in two cases, no performance measures had been established
for either a TA performance goal or a departmentwide initiative. Regarding
the TA performance goal, we mentioned earlier that a new TA performance goal
of protecting the national information infrastructure had been included in
the fiscal year 2001 plan. No performance measure was established for the
goal. In the plan, TA explained that the goal was needed, in part, because
the ubiquitous and interconnected nature of information technology increases
the extent to which even limited attacks or failures can broadly disrupt the
nation's information infrastructure. TA also identified the principal
activity supporting this goal to be the establishment and operation of the
Institute for Information Infrastructure Protection (IIIP), which will lead
a partnership among industry, academia, and government to develop the R& D
capacity, technologies, and knowledge needed to protect the nation's
information infrastructure. TA identified a measure for the goal entitled
“Activity metrics related to program establishment.” However, a
close read of the discussion on that measure showed that no measurable
targets had been developed. TA explained that, as the IIIP becomes
established, it will build appropriate outcome measures into its long- term
program and operational plans. Although it may be premature to establish
appropriate outcome measures for attaining the performance goal, some
intermediate output measures associated with establishing the IIIP would
have provided stakeholders a better sense of what TA expected to accomplish
in fiscal year 2001.

In the case of the departmentwide initiative, according to the fiscal year
2001 plan, there was a priority initiative of “Addressing Critical
Construction Needs.” This $33.8 million initiative included
construction and renovation of NOAA and TA facilities. However, unlike the
other departmentwide initiatives, no performance measures or targets were
established for this initiative. Appropriate output measures could have
communicated NOAA's and TA's intended fiscal year 2001 performance in this
area.

In the case of bureau- specific initiatives, one of NOAA's four such
initiatives was entitled “the Lands Legacy.” Under this $265.8
million initiative, according to the fiscal year 2001 plan, NOAA was to--
among other activities-- provide resources to enhance national marine
sanctuaries and maintain and protect critical estuaries. No performance
measures were established for this or for NOAA's other three bureau-
specific initiatives. Similarly, TA, in the fiscal year 2001 plan,
identified three bureau- specific initiatives. For example, one was titled
“New Super- Fast Methods of Materials in Chemical Research.”
This $4.5 million initiative uses combinatorial methods to accelerate
research, development, and testing of new materials, ranging from
pharmaceuticals to metal alloys to biological products. Again, no
performance measures were established for this or for TA's other two bureau-
specific initiatives.

B- 286123 Page 8 GAO/ GGD- 00- 197R GPRA Plans For Commerce's Science
Agencies

Goals and Resources in NOAA's and TA's Sections of the Fiscal Year 2001 Plan
Generally Matched Those in Their Sections of the Fiscal Year 2001 Budget
Submission, but Better Linkages Can Be Made in the Future

Our review of the NOAA and TA sections of the fiscal year 2001 plan and
their sections of the fiscal year 2001 budget submission showed that, in
most cases, the identified performance goals, budgetary resources and FTE
levels were the same. In other cases, however, they differed to some degree.
For example, we found direct linkages between the performance plan and
budget submission for NOAA's performance goals and their supporting
objectives or activities. For TA, we found direct linkages for six of its
seven performance goals, but not for each goal's supporting objectives or
activities. And, for both NOAA and TA, there were no linkages for any of
their respective bureau- specific initiatives.

NOAA's section of the performance plan provided budget and FTE information
for each of its seven performance goals, as well as budget information for
each goal's supporting objectives or activities. We were able to easily
track this information to NOAA's section of Commerce's fiscal year 2001
budget submission. This was so because NOAA had included in its budget
submission a fiscal year 2001 resources summary table that (1) identified
resource needs by goals and objectives and (2) corresponded directly to the
information presented in its section of the performance plan. On the other
hand, while TA's section of the performance plan also provided budget and
FTE information for each of its seven performance goals, it did not provide
budget information for each goal's supporting objectives or activities like
NOAA's did. Therefore, we were unable to identify linkages to TA's budget
submission at the objectives or activities level. At the performance goal
level, although TA did not include in its budget submission a summary table
identifying resource requirements by goals and objectives as NOAA did, we
were nevertheless able to independently identify linkages to the budget
submission for six of TA's seven performance goals. However, for one of the
seven- to assure and improve measurements and standards-- we had to obtain
the assistance of a TA budget officer to determine how the fiscal year 2001
budgetary amount identified in the plan for that goal linked to the fiscal
year 2001 budget submission. The amount identified in the plan for that goal
was $368.2 million, while the amount identified in the budget submission was
$337.5 million. The budget officer explained that, because component
activities for that goal were identified in different places within the
budget submission, calculations were needed to reconcile the difference in
the amounts.

Both NOAA and TA, in the performance plan, identified bureau- specific
initiatives that were listed and discussed separately from their performance
goals and objectives. Budgetary resources for these initiatives were not
identified in the fiscal year 2001 plan, but were identified in the fiscal
year 2001 budget submission.

B- 286123 Page 9 GAO/ GGD- 00- 197R GPRA Plans For Commerce's Science
Agencies

Not All Performance Plan Weaknesses Were Addressed by the Fiscal Year 2001
Plan

In our review of Commerce's fiscal year 2000 plan, we reported that the
Department had considerable success in improving its planning and the
overall format and presentation of its plan compared with the fiscal year
1999 plan. 5 However, our review of the fiscal year 2000 plan also
identified several areas in which improvements could still be made. Based on
our examination of its fiscal year 2001 plan, we believe Commerce addressed
some, but not all, of the weaknesses we had identified in the fiscal year
2000 plan.

In our report on the fiscal year 2000 plan, we identified general areas of
weakness, including those related to strategies, resources, and performance
data verification and validation. We also identified weaknesses in the area
of performance goals and measures, and earlier in this letter we highlighted
some of the problems that still exist in that area. The following
information describes in more detail the other weaknesses we identified and
the extent to which they have been addressed in the NOAA and TA sections of
Commerce's fiscal year 2001 plan.

Strategies and Resources In our report on the fiscal year 2000 plan, we
stated that the plan's descriptions of strategies and resources were
generally vague and did not clearly relate to the achievement of identified
performance goals and targets. Our review of the fiscal year 2001 plan
showed that, for the NOAA and TA sections of the plan, Commerce has improved
its descriptions of strategies as well as resource requirements.

We previously stated that the strategies and resources associated with
performance goals in the fiscal year 2000 plan were, in some cases, general
and not tailored specifically for the achievement of these goals. In
addition, we stated that the fiscal year 2000 plan did not clearly show how
funding, FTE, and information technology (IT) resources would be allocated
to performance goals. Further, we said the plan did not generally discuss
how external factors that could affect targeted performance levels would be
mitigated.

In the fiscal year 2001 plan, the Department used largely the same language
to identify the strategies for NOAA's and TA's performance goals as was used
in the fiscal year 2000 plan. However, as discussed earlier, one improvement
was that the NOAA section of the fiscal year 2001 plan related a
specifically identified budget amount to each strategy. Our review of the
fiscal year 2001 plan also showed that, for NOAA and TA, Commerce clearly
listed budget, FTE, and IT resource requirements for 13 of these two
bureaus' 14 performance goals. The only exception was that TA did not
delineate an IT resource requirement for its goal to protect the National
Information Infrastructure.

5 Observations on the Department of Commerce's Fiscal Year 2000 Performance
Plan (GAO/ GGD- 99- 117R, July 20, 1999).

B- 286123 Page 10 GAO/ GGD- 00- 197R GPRA Plans For Commerce's Science
Agencies

Regarding mitigation strategies for addressing external factors that could
affect targeted performance, the fiscal year 2001 plan offered some
improvement over the fiscal year 2000 plan. But the 2001 plan, like the 2000
plan, did not provide mitigation strategies for external factors affecting
some goals. While the NOAA and TA sections of the fiscal year 2000 plan did
not identify strategies for addressing external factors, these sections of
the fiscal year 2001 plan did identify mitigation strategies for external
factors for 5 of 14 goals. However, the plan did not provide mitigation
strategies to address external factors listed for the other nine performance
goals. For example, under its performance goal of sustaining healthy coasts,
NOAA identified several coastal stewardship activities that depend on the
contributions of states, territories, and federal agencies. While it
recognized that the failure of one or more of these partners to fulfill
their contributions to these activities could seriously affect the effort to
attain the performance goal, NOAA did not provide a mitigation strategy for
addressing this factor. In addition, under TA's performance goal of
analyzing and developing technology policies, it identified the objective of
coordinating and leading state and federal partnership efforts for key
interagency technology programs. In the plan, TA identified as an external
factor the fact that outputs associated with these partnership programs
depend on the interest and commitment of the participants. However, TA did
not identify a strategy for addressing that factor.

Performance Data Verification and Validation We previously reported that the
fiscal year 2000 plan did not adequately (1) explain how Commerce bureaus
would verify and validate data, (2) identify sources of the data, and (3)
identify limitations of the data for specific goals and measures.
Verification refers to the accuracy of the data, and validation refers to
the appropriateness of the data for measuring performance. Accurate and
valid information is important for measuring progress towards achieving
performance goals and for making informed program decisions. Our review of
the fiscal year 2001 plan showed that, for NOAA and TA, efforts were made to
improve data verification and validation plans and to identify the sources
and limitations of the data. Nevertheless, in some cases, NOAA and TA could
further describe the sources and validity of the data used to measure
progress in attaining their performance goals.

In the fiscal year 2001 plan, NOAA and TA, in some cases, provided expanded
explanations in the data validation and verification sections of the plan.
For example, in the case of NOAA's goal of providing advance short- term
warnings and forecasts-- specifically, the related measure to increase lead
time and accuracy of tornado warnings-- the fiscal year 2001 plan provided
an expanded explanation of how data used to measure performance would be
collected, verified, and reported. The plan also indicated that the
frequency of collecting and reporting those data would be increased from an
annual basis to a monthly basis. However, there was no information provided
on the validity of the data for this performance measure. The language used
to describe data limitations remained unchanged from the prior year's plan.

In another case, for TA's goal to assure and improve measurements and
standards- specifically, the related measure of standard reference materials
available-- the data

B- 286123 Page 11 GAO/ GGD- 00- 197R GPRA Plans For Commerce's Science
Agencies

verification and validation section of the fiscal year 2001 plan provided an
expanded discussion of data verification and highlighted the data
limitations. Similarly expanded information was provided for some of TA's
other goals and measures as well. Nevertheless, little explanation was
generally provided for the sources of the data, and, as with NOAA,
information regarding the validity of the data was lacking.

Conclusions The NOAA and TA sections of the fiscal year 1999 performance
report and fiscal years 2000 and 2001 plans tell a generally coherent story
in addressing science issues in that their missions and performance goals
were consistently stated in each document. However, since the issuance of
the fiscal year 2000 plan, significant, unexplained changes have been made
to NOAA's and TA's performance measures and identified resource needs. While
explanations for these significant changes are not required by either GPRA
or by OMB, we believe such explanations would provide a more complete
picture of what the agencies are trying to accomplish and how they plan to
do so. Further, the exercise of developing explanations for such changes, we
believe, would help identify the kinds of errors we encountered when
comparing data from one plan to another. Also, we believe opportunities
exist for establishing performance measures for departmentwide and bureau-
specific initiatives. The explanations for changes and the added performance
measures would be useful to Congress as it carries out its oversight and
appropriation responsibilities, to internal and external auditors as they
evaluate agency programs and activities, and to agency managers and line
personnel as they work together to effectively implement agency programs and
activities.

Further, we believe that better linkages, i. e., a closer match, between
information in the GPRA documents and in the NOAA and TA sections of the
budget submission would help provide users with perspective in terms of the
size and significance of the goals, objectives, or activities, especially as
the goals, objectives, or activities evolve or change over time. Also, we
believe that identifying budgetary resources for departmentwide initiatives
in the annual performance plans would provide stronger linkages between the
plan and the budget submission. In addition, we believe that linkages could
be improved for TA in the future if it were to identify budgetary resources
for its individual objectives or activities and include a corresponding
summary table in its budget submission.

Finally, we believe that continuing efforts by Commerce to address
weaknesses we identified when examining the fiscal year 2000 performance
plan, especially those related to performance data verification and
validation, would help increase confidence in the results reported in the
future.

B- 286123 Page 12 GAO/ GGD- 00- 197R GPRA Plans For Commerce's Science
Agencies

Recommendations We recommend that the Secretary of Commerce (1) include in
the GPRA documents explanatory information related to significant changes
made to agency performance goals, objectives or activities, performance
measures, and resource requirements and (2) direct that Commerce agencies
better link their sections of the GPRA documents to congressional budget
submissions and other budget documents by identifying budgetary resources at
the bureau- specific initiative levels and the performance goal objective or
activity levels in both the GPRA and budget documents.

Agency Comments and Our Evaluation On September 20, 2000, the Secretary of
Commerce submitted to us written comments on a draft version of this letter.
Those comments included Commerce's general observations on our findings and
recommendations as well as specific comments from NOAA and TA. Some of the
specific comments included questions about implementing GPRA. We have
presented those questions below for consideration by those involved in
implementing GPRA and utilizing GPRA- related documents.

Commerce's general observations were that our letter raised a number of
valid points that Commerce will consider in refining its next strategic plan
and its future annual performance plans and reports. In addition, Commerce
indicated that it will implement at least part of our recommendation by
providing explanations when major changes have been made to performance
measures or goals. Commerce did not indicate, in its general observations,
whether or not it agreed with the other part of our recommendation-- that
is, the need to better link sections of the GPRA documents to congressional
budget submissions and other budget documents.

Regarding NOAA's and TA's specific comments, both concurred that better
linkages between performance data and budget data could be made. For
example, NOAA stated that our finding that budget initiatives (e. g., the
$265.8 million bureau- specific Lands Legacy initiative) and performance
measures could be more explicitly linked was valid. In fact, NOAA stated
that it concurred with our recommendation that called for a better linkage
between the GPRA documents and the congressional budget submissions and
other budget documents by identifying budgetary resources at the bureau-
specific initiative levels. TA stated that the linkage between its budget
data, contained in both the performance plan and the budget submission,
could be better articulated and that its future budget submissions will
include a table that aligns its budget data with the structure of the annual
performance plan.

NOAA and TA raised some concerns about other matters covered in our draft.
For example, in response to our concern about significant, unexplained
differences between appropriation amounts requested and amounts actually
provided, NOAA stated that “ funds requested by the administration are
not always appropriated by Congress, and explanations are not

B- 286123 Page 13 GAO/ GGD- 00- 197R GPRA Plans For Commerce's Science
Agencies

necessarily provided, so it would not be productive or prudent to speculate
on this aspect.” We agree that it would be inappropriate to speculate
on why funding differences occurred. However, because significant funding
differences can materially affect the attainment of performance goals, we
believe that it would be beneficial to provide the reader with the rationale
for the funding differences, if available, as well as an explanation of how
the funding differences will affect attainment of the goals.

NOAA further indicated that the impact of changed resource levels would be
more appropriately addressed in the annual performance report as opposed to
the annual performance plan. We agree that “actual impact” from
the significantly changed resource levels would be appropriately addressed
in the subsequent performance report. However, we also believe that
information regarding planned or “expected impact” from such
resource changes would be appropriately included in the performance plan,
since the available resources directly affect planned attainment of
performance goals.

In its specific comments, TA raised two questions and stated that it would
be pleased to contribute to any future discussions that GPRA leaders in
Congress and the administration might hold on these two questions. The first
question was: “Should agencies be required to fully explain and
document all changes to their performance measures in any given performance
plan and corresponding performance report, or should this type of
information be conveyed through other channels (for instance, through
stakeholder consultations)?” The second question was: “Should
agencies be required to provide and subsequently report on performance
measures for all initiatives, or should GPRA performance plans and reports,
focus on the ‘vital few' measures that best characterize performance
in core programmatic areas?”

Finally, TA commented that GPRA remains unclear on the extent to which
agencies should formally document anticipated performance levels associated
with all initiatives, regardless of level, scope, and character. TA wondered
whether a $500,000 initiative should receive the same level of metric
development and reporting as a $40 million initiative and whether
performance measures for construction initiatives should be developed on a
par with measures for new and/ or existing research programs.

Objectives, Scope, and Methodology The objectives of our work were to
determine (1) whether the NOAA and TA sections of Commerce's fiscal year
1999 performance report and fiscal years 2000 and 2001 performance plans
told a coherent story in addressing science issues, (2) whether NOAA's and
TA's performance goals, budgetary and FTE resource requirements identified
in Commerce's fiscal year 2001 performance plan linked to NOAA's and TA's
sections of Commerce's fiscal year 2001 congressional budget submission, and
(3) whether the NOAA and TA sections of Commerce's fiscal year 2001
performance plan addressed the weaknesses that we identified in our review
of Commerce's fiscal year 2000 performance plan.

The scope of our work was limited to the two Commerce science- related
agencies: NOAA and TA. The fiscal year 2001 budget requests for these two
science- related agencies, together,

B- 286123 Page 14 GAO/ GGD- 00- 197R GPRA Plans For Commerce's Science
Agencies

accounted for approximately 67 percent of Commerce's overall fiscal year
2001 budget request.

Concerning our second objective, GPRA requires “ each agency to
prepare an annual performance plan covering each program activity set forth
in the budget of such agency.” Circular No. A- 11, Part 2, further
requires that, “Each program activity in an agency's Program and
Financing Schedules in the Budget Appendix must be covered by a performance
goal or indicator in the annual performance plan.” The scope of this
review did not include examining compliance with those requirements. A
review of those requirements is included in the scope of another, ongoing,
GAO review. The results of that review should be reported in the near
future.

Our methodology included a review of information contained in Commerce's
fiscal year 1999 performance report and fiscal years 2000 and 2001
performance plans related to the two science agencies. This information
included the agencies' mission statements, priorities, performance goals and
measures, and resource requirements. We also examined Commerce's fiscal year
2001 budget submission to Congress for the two science- related agencies. We
evaluated the extent to which there were linkages between those agencies'
resource requirements, as identified in their sections of Commerce's fiscal
year 2001 performance plan, and the information contained in their sections
of Commerce's fiscal year 2001 congressional budget submission. In addition,
to determine the extent to which NOAA's and TA's sections of Commerce's
fiscal year 2001 plan addressed weaknesses we had identified in the prior
year's plan, we first identified those weaknesses by examining our prior
report on Commerce's fiscal year 2000 performance plan. We then examined the
NOAA and TA sections of Commerce's fiscal year 2001 performance plan to
determine the extent to which they addressed those weaknesses. Finally, we
submitted written questions to, and interviewed, appropriate Commerce
officials to obtain answers and explanatory information regarding questions
that arose during our review.

We did not independently verify the information contained in the performance
report or plans. We conducted our review in July and August 2000 in
accordance with generally accepted government auditing standards.

As arranged with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this letter until 30 days from
the date of this letter. At that time, we will send copies of this letter to
Representatives Vernon J. Ehlers and Ralph M. Hall in their capacities as
Vice Chairman and Ranking Minority Member, respectively, of the House
Science Committee. We will also, at that time, send copies to
Representatives Harold Rogers and Jose E. Serrano in their capacities as
Chairman and Ranking Minority Member, respectively, of the House
Appropriations Subcommittee on Commerce, Justice, State and Judiciary. And,
at that time, we will send copies to Representatives Dan Burton and Henry
Waxman in their capacities as Chairman and Ranking Minority Member,
respectively, of the House Committee on Government Reform.

Also, at that time, we will send copies to Senators Judd Gregg and Ernest
Hollings in their capacities as Chairman and Ranking Minority Member,
respectively, of the Senate

B- 286123 Page 15 GAO/ GGD- 00- 197R GPRA Plans For Commerce's Science
Agencies

Appropriations Subcommittee on Commerce, Justice, State and the Judiciary;
to Senator John McCain in his capacity as Chairman of the Senate Commerce,
Science, and Transportation Committee; and to Senators Fred Thompson and
Joseph Lieberman in their capacities as Chairman and Ranking Minority
Member, respectively, of the Senate Governmental Affairs Committee. At that
time, we will also send copies to the Honorable Jacob J. Lew, Director of
OMB, and we will publish this letter on our Internet Web site, www. gao. gov

If you have any questions about this report, please contact me or N. Scott
Einhorn on (202) 512- 8676. Key contributors to this assignment were Gerard
Burke and Jeremy Latimer.

Sincerely yours, Michael Brostek Associate Director, Federal Management

And Workforce Issues

Page 16 GAO/ GGD- 00- 197R GPRA Plans For Commerce's Science Agencies

Page 17 GAO/ GGD- 00- 197R GPRA Plans For Commerce's Science Agencies

Ordering Copies of GAOReports The first copy of each GAO report and
testimony is free. Additional copies are $2 each. Orders should be sent to
the following address, accompanied by a check or money order made out to the
Superintendent of Documents, when necessary. VISA and MasterCard credit
cards are accepted, also. Orders for 100 or more copies to be mailed to a
single address are discounted 25 percent.

Order by mail: U. S. General Accounting Office P. O. Box 37050 Washington,
DC 20013

or visit: Room 1100 700 4 th St. NW (corner of 4 th and G Sts. NW) U. S.
General Accounting Office Washington, DC

Orders may also be placed by calling (202) 512- 6000 or by using fax number
(202) 512- 6061, or TDD (202) 512- 2537.

Each day, GAO issues a list of newly available reports and testimony. To
receive facsimile copies of the daily list or any list from the past 30
days, please call (202) 512- 6000 using a touchtone phone. A recorded menu
will provide information on how to obtain these lists.

Viewing GAO Reports on the Internet For information on how to access GAO
reports on the INTERNET, send e- mail message with “info” in the
body to:

info@ www. gao. gov or visit GAO's World Wide Web Home Page at: http:// www.
gao. gov Reporting Fraud, Waste, and Abuse in Federal Programs To contact
GAO FraudNET use: Web site: http:// www. gao. gov/ fraudnet/ fraudnet. htm
E- Mail: fraudnet@ gao. gov Telephone: 1- 800- 424- 5454 (automated
answering system)

United States General Accounting Office Washington, D. C. 20548- 0001

Official Business Penalty for Private Use $300

Address Correction Requested Bulk Rate

Postage & Fees Paid GAO Permit No. G100

(410598)
*** End of document. ***