Department of Commerce: Status of Achieving Key Outcomes and	 
Addressing Major Management Challenges (15-JUN-01, GAO-01-793).  
								 
This report reviews the Department of Commerce's fiscal year 2000
performance report and fiscal year 2002 performance plan required
by the Government Performance and Results Act to assess 	 
Commerce's progress on achieving selected key outcomes. 	 
Commerce's combined performance report and performance plan is a 
significant improvement over its fiscal year 1999 performance	 
report and fiscal year 2001 performance plan and addresses the	 
recommendations made in GAO's June 2000 report. Furthermore, the 
report indicates that Commerce has made progress toward achieving
two of the selected key outcomes. However, progress toward	 
achieving the remaining two outcomes is unclear largely because  
of weaknesses related to measuring performance. Specifically,	 
some of the measures are output-oriented, rather than outcome	 
oriented; some measures have known limitations, which Commerce	 
acknowledges; many of the measures used to assess performance in 
the past are being discontinued for the future; and Commerce	 
plans to rely on one, narrowly focused measure to demonstrate	 
progress for each performance goal related to these key outcomes.
Furthermore, other data exist within the International Trade	 
Administration (ITA) that could support additional measures	 
related to these performance goals.				 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-01-793 					        
    ACCNO:   A01116						        
  TITLE:     Department of Commerce: Status of Achieving Key Outcomes 
             and Addressing Major Management Challenges                       
     DATE:   06/15/2001 
  SUBJECT:   Performance measures				 
	     Reporting requirements				 
	     Strategic planning 				 
	     Government Performance and Results Act		 
	     GPRA						 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Testimony.                                               **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-01-793
     
Report to the Ranking Minority Member, Committee on Governmental Affairs,
United States Senate

United States General Accounting Office

GAO

June 2001 DEPARTMENT OF COMMERCE

Status of Achieving Key Outcomes and Addressing Major Management Challenges

GAO- 01- 793

Page 1 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

June 15, 2001 The Honorable Fred Thompson Ranking Minority Member Committee
on Governmental Affairs United States Senate

Dear Senator Thompson: As you requested, we reviewed the Department of
Commerce?s fiscal year 2000 performance report and fiscal year 2002
performance plan required by the Government Performance and Results Act of
1993 (GPRA) to assess Commerce?s progress in achieving selected key outcomes
that you identified as important mission areas for the agency. 1 These are
the same outcomes we addressed in our June 2000 review of Commerce?s fiscal
year 1999 performance report and fiscal year 2001 performance plan to
provide a baseline by which to measure the agency?s performance from year to
year. 2 The selected key outcomes are as follows:

 jobs are created and businesses remain successful,  weather forecasts and
severe weather warnings are accurate and timely,  U. S. businesses have
greater access to international markets, and  the U. S. is secure from the
proliferation of dual- use commodities and

chemical weapons. 3 As agreed, using the selected key outcomes for Commerce
as a framework, we (1) assessed the progress Commerce has made in achieving
these outcomes and the strategies the agency has in place to achieve them
and (2) compared Commerce?s fiscal year 2000 performance report and fiscal
year 2002 performance plan with its prior- year performance report and plan.
In addition, we agreed to analyze how Commerce addressed major management
challenges that Commerce?s Office of Inspector General and we identified,
including the

1 This report is one of a series of reports on the 24 Chief Financial
Officers Act agencies? fiscal year 2000 performance reports and fiscal year
2002 performance plans. 2 Observations on the Department of Commerce?s
Fiscal Year 1999 Annual Program Performance Report and Fiscal Year 2001
Annual Performance Plan (GAO/ GGD- 00- 152R, June 30, 2000).

3 Dual- use commodities are those with both commercial and military
applications.

United States General Accounting Office Washington, DC 20548

Page 2 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

governmentwide high- risk areas of strategic human capital management 4 and
information security. Appendix I provides detailed information on how
Commerce addressed these challenges.

Commerce?s report indicates that it has made progress in achieving two of
the selected key outcomes; however, progress on the remaining two outcomes
is unclear. Commerce?s strategies for achieving these outcomes are generally
clear and reasonable. Specifically,

Planned outcome: Jobs are created and businesses remain successful. On the
basis of Commerce?s report, it made good progress in achieving its
performance goals related to this key selected outcome. Among the reported
achievements were creating or retaining over twice as many jobs in
distressed areas as planned and obtaining millions of dollars in private
sector investments in distressed communities ahead of schedule. In addition,
Commerce provides clear and reasonable strategies for meeting its fiscal
year 2002 goals. For example, Commerce plans to provide grants for economic
development projects in distressed communities, provide grants to implement
economic adjustment strategies in communities that experience sudden losses
of jobs and severe economic distress, strengthen and expand its strategic
planning and investment activities through a network of regional planning
organizations, and disseminate information on economic development issues
affecting distressed communities to entities involved in helping those
communities.

Planned outcome: Weather forecasts and severe weather warnings are accurate
and timely. Based on our review of Commerce?s report, it made limited
progress in achieving its performance goal related to this outcome.
Specifically, Commerce met its targets for flash floods and winter storms
that are covered by warnings. On the other hand, it did not issue warnings
for flash floods, winter storms, and tornadoes with as much lead time as
targeted and its false alarm rate for tornado warnings was higher than
targeted. In some cases, Commerce?s performance stayed about the same as
last year even though the target was not met. For example, the reported
accuracy for marine forecasts involving winds and waves remained at 50
percent, slightly below the target of 51 percent. Commerce noted that

4 Key elements of modern human capital management include strategic human
capital planning and organizational alignment; leadership continuity and
succession planning; acquiring and developing staffs whose size, skills, and
deployment meet agency needs; and creating results- oriented organizational
cultures. Results in Brief

Page 3 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

improvements are difficult to observe on an annual basis and that historical
trends show that the agency continues to improve the accuracy and lead time
of warnings related to severe weather hazards. Its strategies for further
improving its performance are clear and reasonable and include continuing
its investments in research and new technologies, maintaining continuous
satellite coverage, strengthening weather observing and prediction systems,
and improving the dissemination of weather forecasts and warnings to the
public and others.

Planned outcome: U. S. businesses have greater access to international
markets. Commerce?s progress made toward this key outcome is unclear because
of the reported performance for specific measures, as well as the
limitations on those measures. For example, some of the targets related to
this outcome were not met and performance for those targets during fiscal
year 2000 was lower than performance during fiscal year 1999. Also, it is
not clear whether Commerce met its targets for two measures. For example,
Commerce reported that it met the original target for the number of ?new-
to- market? firms, explained that it had revised this target upward in the
fiscal year 2001 performance plan, and stated that it did not meet the
revised target. However, in its accountability report, Commerce included the
same explanation and said that it did not meet its target for this measure.
Commerce did not explain why it used the lower targets for this report.
Furthermore, Commerce reported that its ability to demonstrate future
progress is hindered by the limitations of the measures that it uses and
that it faces inherent difficulties in quantifying its progress due to the
reluctance of its business clients to volunteer proprietary information, the
unanticipated shifts in the world economy, and the challenge of isolating
its contribution to U. S. trade objectives from that of other agencies.
Commerce reported that it has taken steps to improve its ability to
demonstrate progress towards its goals by discontinuing some measures and
adding new measures. In addition, Commerce presented a multi- step strategy
for improving its performance. While this strategy seems clear and
reasonable, the prospects for improving the agency?s ability to demonstrate
progress are unclear because, for each of its four performance goals for
fiscal year 2002, it plans to rely on one, narrowly focused measure to
demonstrate progress. For example, Commerce will measure its progress in
increasing U. S. exports by using the number of

?new- to- market? firms. Commerce states that this measure is circumstantial
evidence of overall export growth and adds that it cannot measure actual
dollar value of exports because of limitations on the data, such as the
reluctance of businesses to reveal that information. In some cases, other
data exist that could support additional measures related to these
performance goals.

Page 4 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

Planned outcome: The U. S. is secure from the proliferation of dual- use
commodities and chemical weapons. Commerce?s progress related to this key
outcome is unclear. As we observed last year, the Bureau of Export
Administration?s ability to demonstrate progress toward this outcome has
been hampered by the tendency of its measures to reflect workload and
outputs- rather than outcomes- and a lack of clarity in the presentation of
its performance goals. Specifically, while Commerce reported that it made
some progress in achieving the performance targets related to this key
outcome, its measures included primarily outputs such as the number of
strategic industry analyses completed, the number of enforcement outreach
visits, and the number of end- use visits conducted. Also, Commerce plans to
discontinue most of the measures that it used to assess its performance
related to this key outcome. While Commerce described reasonable strategies
for future achievement related to key outcome, its prospects for
demonstrating progress are unclear because it plans to rely on one measure
for each of its performance goals for fiscal year 2002.

In many cases, Commerce explained why targets were not met and described
strategies for the future. Program evaluations were also discussed. For
example, the Economic Development Administration said that it used program
evaluations to develop valid performance measures, verify results, provide a
more complete understanding of overall program performance, and improve
program performance. Similarly, the Bureau of Export Administration noted
that it used the results of program evaluations to identify improvements in
data entry and review. In several cases, Commerce cited the results of
evaluations that the Inspector General and we had conducted and described
actions that it took as a result of those evaluations. Commerce?s reporting
format includes information on data sources, validation and verification
efforts, and known limitations for the data.

Commerce?s FY 2000 Annual Program Performance Report and FY 2002 Performance
Plan is a significant improvement over its fiscal year 1999 performance
report and fiscal year 2001 performance plan and addresses the
recommendations made in our June 2000 report to (1) classify performance
measures as met or unmet to more clearly communicate the agency?s
performance, (2) consistently explain why targets were not met when
differences occur between actual and targeted performance, and (3) provide
information about plans and schedules for meeting targets when they are not
met. Commerce clearly showed whether targets for performance goals were met
or not met, rather than using the three- tiered scoring system that was used
in last year?s report. Furthermore,

Page 5 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

Commerce?s combined performance report and plan includes summaries of the
mission; priorities and management challenges; actual and targeted levels of
performance; resource requirements; detailed descriptions of the performance
goals and measures; and explanations of whether performance targets were
met, not met, changed, added, or discontinued.

In January 2001, we identified two governmentwide high- risk areas-
strategic human capital management and information security- and reported on
four major management challenges facing Commerce. Commerce?s combined report
and plan contained goals and measures for all of these challenges except for
information security; and, under a broad goal, it contains measures related
to information security. Where measures were identified, Commerce reported
its progress related to those measures.

Commerce?s combined report and plan is a significant improvement from last
year and addresses the recommendations made in our June 2000 report.
However, due to weaknesses related to measuring performance, it is not
always clear whether Commerce is making progress toward achieving its
outcomes. Consequently, we recommend that the Secretary of Commerce direct
its component agencies to reassess the measures that are used to assess
progress in achieving their organizational performance goals related to the
key outcomes that cover international trade and export control.

GPRA is intended to shift the focus of government decisionmaking,
management, and accountability from activities and processes to the results
and outcomes achieved by federal programs. New and valuable information on
the plans, goals, and strategies of federal agencies has been provided since
federal agencies began implementing GPRA. Under GPRA, annual performance
plans are to clearly inform the Congress and the public of the (1) annual
performance goals for agencies? major programs and activities, (2) measures
that will be used to gauge performance, (3) strategies and resources
required to achieve the performance goals, and (4) procedures that will be
used to verify and validate performance information. These annual plans,
issued soon after transmittal of the President?s budget, provide a direct
linkage between an agency?s longerterm goals and mission and day- to- day
activities. 5 Annual performance

5 The fiscal year 2002 performance plan is the fourth of these annual plans
under GPRA. Background

Page 6 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

reports are to subsequently report on the degree to which performance goals
were met. The issuance of the agencies? performance reports, due by March 31
of each year, represents a new and potentially more substantive phase in the
implementation of GPRA- the opportunity to assess federal agencies? actual
performance for the prior fiscal year and to consider what steps are needed
to improve performance, and reduce costs in the future. 6 To meet the
reporting requirements for this year, Commerce issued a combined performance
report and performance plan.

According to Commerce, its basic mission is to promote job creation and
improve living standards for all Americans through economic growth,
technological competitiveness, and sustainable development. The component
agencies that are accountable for the selected key outcomes covered in this
report are responsible for providing assistance to distressed communities;
forecasting and warning the public about severe weather, such as hurricanes,
tornadoes, and floods; supporting export policies that affect millions of
domestic jobs and monitoring hundreds of international trade agreements; and
helping stop the proliferation of weapons of mass destruction and providing
technical assistance to Russia and newly emerging countries concerning
export controls. Activities not covered by the selected outcomes include
supporting the domestic and international management of living marine
resources, including overseeing 2,000,000 square miles of ocean and 300
marine species; fostering telecommunications policies that promote economic
and technical advancement; protecting patents, trademarks, and intellectual
property rights; and conducting the decennial census, demographic and
economic censuses, and over 200 annual surveys. While Commerce shares
responsibilities for these functions with a number of other federal
departments and agencies, it has a lead role in one area- promoting economic
development in distressed areas.

6 The fiscal year 2000 performance report is the second of these annual
reports under GPRA.

Page 7 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

This section discusses our analysis of Commerce?s performance in achieving
selected key outcomes and the strategies- particularly those related to
strategic human capital management and information technology- that it has
in place for achieving these outcomes.

On the basis of Commerce?s report, the Economic Development Administration
(EDA) made good progress in achieving its two performance goals related to
this key selected outcome: (1) to create jobs and private enterprise in
economically distressed communities and (2) to build local capacity to
achieve and sustain economic growth. In March 1999, we reported that
measuring job creation that results from economic development programs is
problematic. 7 EDA reports that it adjusted its targets related to creating
and retaining permanent jobs based on our report, resulting in conservative
targets. In addition, EDA uses multiple measures to assess its performance
related to this outcome. Among the achievements that EDA reports were
creating or retaining over twice as many jobs in distressed areas as
planned, obtaining millions of dollars in private sector investments in
distressed communities ahead of schedule, obtaining more state and local
funds than anticipated, awarding a larger portion of its grants to areas of
highest distress, and decreasing the processing time for grant applications.
The one performance measure for which the target was not met was the
percentage of Economic Development District and Indian tribe planning
grantees whose Comprehensive Economic Development Strategies were submitted
on time and were acceptable. Commerce explained that this target is not
based on statutory requirements and was originally an estimate. In addition,
some delays may have stemmed from new legislation that changed the process
for some grantees. EDA said that it is analyzing trends to determine if
adjustments need to be made to future targets.

EDA presents clear and reasonable strategies for the future, which include
the following:

7 See Economic Development: Observations Regarding the Economic Development
Administration?s May 1998 Final Report on Its Public Works Program

(GAO/ RCED- 99- 11R, Mar. 23, 1999). Assessment of

Commerce?s Progress and Strategies in Achieving Selected Key Outcomes Create
Jobs and Maintain Successful Businesses

Page 8 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

 providing grants for economic development projects in distressed
communities,

 providing grants to implement economic adjustment strategies in
communities that experience sudden losses of jobs and severe economic
distress,

 strengthening and expanding its strategic planning and investment
activities through a network of regional planning organizations, and

 disseminating information on economic development issues affecting
distressed communities to entities involved in helping those communities.

EDA said that it used program evaluations to develop valid performance
measures, verify results, provide a more complete understanding of overall
program performance, and improve program performance. EDA also identified
specific program evaluations that it had used in the past, as well as
evaluations that are currently under way. EDA reports that it plans to
collaborate with other federal agencies on ways to evaluate and measure
crosscutting activities to improve economic development assistance for
distressed communities- a topic that was covered in our September 2000
report. 8

For each of its performance measures, EDA describes the data source, as well
as its data validation and verification efforts. In addition, EDA generally
discusses data limitations. For example, for two measures related to the
ratings that EDA receives from its clients, EDA states that this is the
first year that data have been reported and that it will evaluate trends
after more years of data are available.

The National Oceanic and Atmospheric Administration (NOAA) made limited
progress in achieving its performance goal related to this outcome- to
advance short- term warnings and forecasts. NOAA reported its fiscal year
2000 performance for this goal using measures such as the lead time and
accuracy of forecasts and warnings. Specifically, NOAA met its targets for
flash floods and winter storms that are covered by warnings. On the other
hand, it did not issue warnings for flash floods, winter storms, and
tornadoes with as much lead time as targeted and its false alarm rate for
tornado warnings was higher than targeted. NOAA?s performance on one
measure- the accuracy of warnings for hurricanes-

8 See Economic Development: Multiple Federal Programs Fund Similar Economic
Development Activities (GAO/ RCED/ GGD- 00- 220, Sept. 29, 2000). Provide
Accurate and

Timely Weather Forecasts and Severe Weather Warnings

Page 9 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

was not applicable this past year because no hurricanes made landfall during
the 2000 season. In some cases, NOAA?s performance stayed about the same as
last year even though the target was not met for fiscal year 2000. For
example, the reported accuracy for marine forecasts involving winds and
waves remained at 50 percent, slightly below the target of 51 percent.
Similarly, the reported lead time for severe weather warnings for flash
floods increased from 41 minutes to 43 minutes, but fell short of the target
of 55 minutes.

In reporting on these measures, Commerce?s report offered explanations for
its failure to meet targets, which often involved atypical weather
conditions. For example, in explaining why it did not meet its three targets
for tornado warnings, NOAA noted that the 2000 tornado season had few well-
organized storm systems, resulting in fewer and weaker tornadoes that are
harder to detect. Similarly, in explaining why it did not meet its target of
20 percent accuracy in 3- day precipitation forecasts, NOAA indicated that
the past year had been relatively dry, a factor that can lower the accuracy
score. In addition, NOAA noted that this measure was being used for the
first time and, therefore, was considered a baseline for this measure.

NOAA noted that improvements are difficult to observe on an annual basis and
that historical trends show that the agency continues to improve the
accuracy and lead time of warnings related to severe weather hazards. NOAA?s
strategies for further improving its performance are clear and reasonable.
These strategies include continuing its investments in research and new
technologies, maintaining continuous satellite coverage, strengthening
weather observing and prediction systems, and improving the dissemination of
weather forecasts and warnings to the public and others. NOAA also stated
that it uses annual program evaluations at its National Weather Service
(NWS) field offices as part of its quality control to ensure the reliability
of its data and products; however, it did not elaborate on the results of
these evaluations.

For each of its performance measures, NOAA describes the data source, as
well as its data validation and verification efforts. In addition, NOAA
generally discusses data limitations. For example, for its measure related
to hurricane warnings, NOAA states that there is a large variability in the
hurricane warning program due to sample sizes and types of storms.

Our prior work drew attention to problems that NWS faced in developing and
implementing weather- forecasting technologies. Because of these and other
problems, we identified NWS? efforts to modernize its critical

Page 10 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

weather systems as a high- risk area in our 1995, 1997, and 1999 high- risk
series reports. 9 We made a series of recommendations aimed at helping NWS
improve its management of the modernization effort, and NWS has implemented
many of those recommendations. Most recently, in January 2001, we removed
NWS? modernization effort from our high- risk list because of the progress
NWS had made in addressing our recommendations and in implementing a
critical information processing system. NWS is continuing to improve its
modernization systems and has reported that these efforts are critical to
its ability to improve warnings and forecasts over time.

Commerce?s progress made toward ensuring that U. S. businesses have greater
access to international markets is unclear for at least three reasons.
First, most of the targets related to this outcome were not met and
performance for those targets during fiscal year 2000 was lower than
performance during fiscal year 1999. For example, the International Trade
Administration (ITA) did not meet its target for the number of counseling
sessions held with U. S. businesses and held about 20,000 fewer sessions in
fiscal year 2000 than it held in fiscal year 1999. ITA stated that the
number of counseling sessions did not reflect its level of success given the
emphasis on using electronic means to communicate with U. S. firms, but ITA
did not describe its efforts to communicate electronically. Also, ITA did
not meet the target for the value of gross exports supported through
advocacy efforts and reported that the dollar value was $8.9 billion for
fiscal year 2000, as opposed to $9.8 billion for fiscal year 1999. Second,
it is not clear whether ITA met its targets for two measures. Specifically,
ITA reported that it met the original targets for the number of ?new- to-
market? and ?new- to- export? firms. ITA explained that it had revised these
targets upward in the fiscal year 2001 performance plan and did not meet the
revised targets; however, ITA did not explain why it used the lower targets
for this report. In addition, based on the same data, Commerce?s
accountability report for fiscal year 2000 states that the target for the
number of ?new- to- market? firms was not met. Third, ITA reported that its
ability to demonstrate future progress is hindered by the limitations of the
measures that it uses. ITA reported that it faces inherent difficulties in
quantifying its progress due to the reluctance of its business clients to

9 See High- Risk Series: An Overview (GAO/ HR- 95- 1, Feb. 1995); High- Risk
Series: Information Management and Technology (GAO/ HR- 97- 9, Feb. 1997);
and High- Risk Series: An Update (GAO/ HR- 99- 1, Jan. 1999). Increase
Access of U. S.

Businesses to International Markets

Page 11 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

volunteer proprietary information and the unanticipated shifts in the world
economy. In addition, ITA faces the challenge of isolating its contribution
to U. S. trade objectives from that of U. S. industry and other U. S.
agencies.

While ITA met its targets for the number of antidumping and countervailing
duty cases processed and for the dollar value of market openings, it is
discontinuing use of the dollar value of market openings because it does not
directly relate to ITA?s success in enforcing trade laws and agreements. 10
ITA also plans to discontinue using the number of counseling sessions held
with U. S. businesses and the value of gross exports supported as
performance measures. While ITA explains its reasons for discontinuing
measures, some of the explanations were very brief. For example, ITA states
that the value of gross exports is being discontinued as a measure to
reflect departmental goals and objectives but does not describe those goals
and objectives or explain the relationship of this measure to those goals
and objectives.

ITA has taken steps to improve its ability to demonstrate progress towards
its goals by discontinuing three measures and adding two new measures. In
addition, ITA presented a multi- step strategy for improving its
performance. While this strategy is clear and reasonable, ITA?s prospects
for success in improving its ability to demonstrate progress are unclear
because, for each of its four performance goals for fiscal year 2002, it
plans to rely on one, narrowly focused measure to demonstrate progress. For
example, ITA will measure its progress in increasing U. S. exports by
implementing the National Export Strategy by using the number of ?newto-

market? firms. ITA states that this measure is circumstantial evidence of
overall export growth and adds that it cannot measure actual dollar value of
exports because of limitations on the data, such as the reluctance of
businesses to reveal that information. In another instance, Commerce will
use the number of anti- dumping and countervailing duty cases processed to
measure its progress in improving American competitiveness and access to
foreign markets by enforcing compliance with U. S. trade laws. However, in
summarizing the limitations to this information, ITA states that the number
of cases processed depends entirely on the number of injurious foreign trade
actions taken and on the responses of parties other than ITA.

10 In addition, in our June 2000 report, we noted that this measure was not
directly linked to ITA?s performance goals.

Page 12 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

In some cases, data exist that could support additional measures related to
ITA?s performance goals. For example, as we reported in March 2000, ITA?s
Trade Compliance Center relies on multiple sources of information and
reporting mechanisms to ensure that trade agreements are monitored and
compliance issues are promptly addressed. 11 These sources of information
include U. S. businesses, other parts of Commerce, and U. S. embassies
overseas. The Trade Compliance Center also produces regular reports on
possible compliance violations for use by other executive branch agencies.
Thus, ITA already collects and maintains a rich set of useful data that is
directly related to one of its key performance goals- improving
competitiveness and access to foreign markets by enforcing compliance with
U. S. trade laws and agreements.

ITA identified both internal and external program evaluations that were
related to its performance goals. Specifically, ITA reported that, in
response to one of our reports, it clarified criteria for the types of
agreements that are to be included in a trade database. Also, ITA reported
that it conducted a number of internal reviews that identified needed
changes in programs, administration, and management that were implemented at
the end of these reviews.

For each of its performance measures, ITA describes the data source, as well
as its data validation and verification efforts. In addition, ITA generally
discusses data limitations. For example, ITA explains that its success in
collecting data on ?new- to- export? firms is dependent on its clients?
willingness to provide such information.

The Bureau of Export Administration?s (BXA) progress toward ensuring that
the United States is secure from the proliferation of dual- use commodities
and chemical weapons is unclear. As we observed last year, BXA?s ability to
demonstrate progress toward this outcome has been hampered by the tendency
of its measures to reflect workload and outputs- rather than outcomes- and a
lack of clarity in the presentation of its performance goals.

BXA reported that it met its targets for the number of strategic industry
analyses completed, the number of investigations accepted for criminal or
administrative remedies, the number of enforcement outreach visits, the

11 International Trade: Strategy Needed to Better Monitor and Enforce Trade
Agreements

(GAO/ NSIAD- 00- 76, Mar. 14, 2000). Ensure That the United

States Is Secure From Proliferation of Dual- Use Commodities and Chemical
Weapons

Page 13 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

number of end- use visits conducted, and the number of nonproliferation and
export control international cooperative exchanges. However, BXA reported
that it plans to discontinue using the last four of these measures for
external reporting in the future. BXA said that it was changing its measures
to align them more closely with new departmental objectives. However, BXA
did not describe those objectives or explain why these measures do not
support those objectives. In addition, BXA did not achieve its targets for
average export licensing processing time, high- risk transactions deterred,
licensing decisions made, number of investigations completed, and export
assistance seminars and conferences attended. BXA plans to discontinue using
all of these measures except the number of high- risk actions deterred.

While BXA attributed its lack of success in meeting these targets to several
factors- including the liberalization of export controls, the growing
complexity of its investigations, and staff limits- its explanations were
not always complete. For example, BXA said that the decline in the number of
high- risk transactions deterred was due to unforeseen regulatory, policy,
and technological changes, but did not describe those changes or explain
their impact on the measure. In addition, BXA stated that it did not meet
its target for number of licensing decisions due to unanticipated
liberalizations of export controls. However, Commerce has advocated such
changes and the report did not provide enough information to explain how
these liberalizations differed from those advocated by Commerce. In another
case, BXA said that it did not meet its targeted number of seminars and
conferences due to budget constraints and its inability to fill critical
personnel vacancies; however, it did not provide supporting details.

BXA broadly described strategies for meeting the targets for its five
performance goals associated with this key outcome. However, like ITA, BXA?s
prospects for success in improving its ability to demonstrate progress are
unclear because for fiscal year 2002, it plans to rely on one measure to
demonstrate progress for each of these performance goals. For example, to
ensure that the U. S. defense industrial base is healthy and competitive,
BXA describes strategies such as coordinating federal support for U. S.
defense firms involved in procurement competitions overseas, facilitating
business contacts between U. S. industry and potential customers,
maintaining the competitiveness of U. S. defense firms, and minimizing the
burden of offsets placed on firms that make defense sales to foreign
customers. To measure its performance related to this goal, BXA plans to use
the dollar value of contracts won in international competitions by U. S.
defense firms. While this measure may

Page 14 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

be an indicator of BXA?s success in this area, additional measures would
provide further evidence of progress.

Like the other component agencies, BXA identified program evaluations that
were related to its performance goals. Specifically, BXA reported that, in
response to reports from Commerce?s Office of Inspector General and us, BXA
initiated its own program reviews related to data validation. BXA reported
that it used the results of these program evaluations to identify
improvements in data entry and review, among other things.

For each of its performance measures, BXA describes the data source, as well
as its data validation and verification efforts. However, unlike the other
component agencies covered by this report, BXA generally did not discuss
data limitations.

Commerce?s FY 2000 Annual Program Performance Report and FY 2002 Performance
Plan is a significant improvement over its fiscal year 1999 performance
report and fiscal year 2001 performance plan and addresses the
recommendations made in our June 2000 report. Most notably, Commerce clearly
showed whether targets for performance goals were met or not met, rather
than using the three- tiered scoring system that was used in last year?s
report. Furthermore, for most of its component agencies and for departmental
management, Commerce?s combined performance report and plan includes
summaries of the mission, priorities and management challenges, actual and
targeted levels of performance, and resource requirements. These summaries
are followed by detailed explanations of the performance goals and measures
that include explanations of whether performance targets were met or not
met, as well as any changes, additions, or deletions from the prior year. In
addition, an appendix in the combined document summarizes performance
measures across component agencies, links them to Commerce?s strategic
goals, and designates whether the targets were met or not met.

In our June 2000 report, we recommended that the Secretary of Commerce
ensure that the fiscal year 2000 performance report (1) classify performance
measures as met or unmet to more clearly communicate the agency?s
performance, (2) consistently explain why targets were not met when
differences occur between actual and targeted performance, and (3) provide
information about plans and schedules for meeting targets when they are not
met. Commerce?s combined report and plan addresses these recommendations.
For each performance measure, the document includes a table that lists the
targeted and actual performance for fiscal years 1999 Comparison of

Commerce?s Fiscal Year 2000 Performance Report and Fiscal Year 2002
Performance Plan With the Prior Year?s Report and Plan

Page 15 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

and 2000, clearly indicates whether the targets were met or not met, and
lists targets for fiscal years 2001 and 2002. 12 The table is followed by a
narrative explanation of that measure. For the measures whose targets were
exceeded or not met, the description usually explains whether Commerce has
adjusted its performance targets for future years based on actual
performance. For example, EDA reports that it has created or retained over
12,000 jobs in distressed areas, yet one of its fiscal year 2001 targets is
to create or retain 7,201 jobs by fiscal year 2004. The narrative explains
that this is the first year for which data have become available for this
measure and the high number of jobs created or retained could have been due,
in part, to strong economic growth; consequently, additional data are
required before adjusting long- term projections. In another case, ITA
reported that the number of ?new- to- export? firms for fiscal year 2000 was
33,514, which was fewer than in fiscal year 1999; consequently, ITA reduced
its fiscal year 2001 target for this measure from 36,949 firms to 30,336
firms to reflect trends experienced in fiscal year 2000. Each performance
goal also includes a list of the measures that were discontinued,
accompanied by explanations; an action plan, including planned strategies; a
description of crosscutting activities both inside and outside of the
department; and a discussion of external factors and accompanying mitigation
strategies. In addition, the document includes two appendixes, one that
lists all of the fiscal year 2000 measures and indicates whether they were
met or not met and one that lists the performance measures and targets for
fiscal years 2001 and 2002.

In January 2001, we identified two governmentwide high- risk areas-
strategic human capital management and information security- and reported on
four major management challenges facing Commerce. 13 Commerce?s combined
report and plan contained goals and measures directly related to all of
these challenges. Where measures were identified, Commerce reported its
progress related to those measures.

12 As in the past, Commerce clearly identifies the data source for each
measure, as well as its plans to validate and verify the data. 13 Major
Management Challenges and Program Risks: A Governmentwide Perspective

(GAO- 01- 241, Jan. 2001) and Major Management Challenges and Program Risks:
Department of Commerce (GAO- 01- 243, Jan. 2001). Commerce?s Efforts

to Address Its Major Management Challenges Identified by Us

Page 16 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

Commerce?s combined report and plan includes a performance goal of
acquiring, managing, and developing a diverse, skilled, and flexible staff,
using information technology as an essential tool. In describing its
measures related to this goal, Commerce states that our high- risk report on
strategic human capital management encourages agencies to consider strategic
human capital management as a strategic asset. 14 Commerce notes that it
held a departmentwide meeting in August 2000 that involved senior managers
who generated ideas for changing Commerce?s human capital strategy and
incorporating human capital management into all major management systems. To
assess how well it achieves this goal, Commerce plans to measure

 implementation of a process to identify current and projected
missionrelated workforce needs,

 the value of human resources services,

 the efficiency and effectiveness of hiring systems,

 recruiting activities,

 the alignment of performance management with mission accomplishment and
overall recognition, and

 implementation of a telecommuting program. For fiscal year 2000, Commerce
reports achievements that include developing a workforce analysis plan and
identified workforce planning tools, pilot testing a Web- based recruitment
and referral system, integrating performance management and awards, and
pilot testing a telecommuting program. In addition, Commerce cited human
capital initiatives as part of its strategy for addressing its challenge of
strengthening financial management controls. Specifically, Commerce plans to
establish financial leadership positions at all bureaus and develop a
professional education program.

For each of its component agencies, Commerce?s combined report and plan
includes a table of resource requirements- including full- time equivalent
positions, as well as a summary of the skills needed in that component
agency. For example, BXA?s skill summary includes an extensive knowledge of
the legislation and Executive Orders related to controlling dual- use
commodities; knowledge of world political and economic systems, current
trends in U. S. trade, and national security and

14 See High- Risk Series: An Update (GAO- 01- 263, Jan. 2001). Strategic
Human Capital

Management

Page 17 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

foreign policy issues; and analytical abilities for complex licensing and
policy decisions, as well as regulatory interpretations.

Several component agencies discussed challenges related to human capital.
The Bureau of the Census states that its strategies to achieve its mission
include valuing its employees, and lists its workplace and workforce as a
management challenge. By fostering diversity in its workforce, the Bureau
hopes to create surveys that are more sensitive to the cultural base of the
people being surveyed, as well as retain a trained and skilled workforce.
One of ITA?s management challenges is to position human capital as a
strategic asset. It projects that one- third of its employees can retire
within 5 years and lists strategies for maintaining its workforce.
Similarly, BXA notes that it lacks sufficient technical and analytical
personnel and will lose people who retire. BXA also includes maintaining
technical and analytical abilities as one of its management challenges,
although it does not have goals and measures that are directly related to
this challenge. NOAA and the Technology Administration discuss investing in
people under their respective sections on priorities.

While Commerce?s combined report and plan does not identify a specific
performance goal for strengthening departmentwide information security, some
of its goals contain measures related to information security. For example,
under Department Management, Commerce?s performance goal to acquire and
manage the fiscal and related resources necessary to support program goals
contains a measure to protect information and staff at field sites from risk
or disaster. For fiscal years 2001 and 2002, the measure will be to ensure
the protection of critical, classified, and sensitive Commerce computer
systems and information from compromise or exploitation by adversaries, and
will include inspecting, accrediting, and certifying computer systems.
Another performance goal under Department Management is to acquire and
manage the technology resources to support program goals. This goal includes
a measure to increase the maturity of its information technology security
program, which is a new measure for which the first target is established
for fiscal year 2001.

Information security and related issues such as critical infrastructure
protection are also identified in the report as management challenges by the
following component agencies: EDA, BXA, NOAA, the Bureau of the Census, the
Minority Business Development Agency, and the National Institute of
Standards and Technology. The report provides little specific information on
the progress of resolving these challenges at these Information Security

Page 18 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

component agencies, and the related performance goals or measures identified
were shown as new for fiscal year 2000.

The fiscal year 2002 plan does not highlight the priority for establishing
safeguards against unconventional national security threats that was
included in the fiscal year 2001 plan. However, according to the plan, some
of Commerce?s component agencies have performance goals and/ or measures for
critical infrastructure protection. These are as follows:

 The National Institute of Standards and Technology has a goal to protect
the national information infrastructure. The primary focus of this goal is
to establish a grants program for new research into advanced technologies,
measurements, and methods that can raise the level of reliability and
security of critical information technology- based systems and networks. The
current measure for this goal is the timely and successful completion of
activity milestones to establish the program. In the longer term, outcome
measures will be developed that gauge the security, reliability, quality,
and survivability of information technology systems and networks.

 BXA has a new performance goal to ensure that the nation?s infrastructure
components are secured in accordance with an integrated plan. To begin in
fiscal year 2002, the measure for this goal is the number of agency plans
implemented within the framework of the National Critical Infrastructure
Protection Plan.

 The National Telecommunications and Information Administration included a
new performance goal to minimize the effects of crisis by preparing the U.
S. telecommunications and information infrastructure protection programs. To
begin in fiscal year 2002, the measure for this goal is to increase the
number of state, city, and county governments actively engaged in critical
infrastructure protection programs. However, in commenting on a draft of
this report, Commerce officials said they will discontinue this measure
because Commerce did not receive funding for efforts related to it.

The four management challenges that we identified are the same as the four
outcomes covered by this report. As discussed above, we found that
Commerce?s performance report described the agency?s progress in achieving
these challenges and contained goals and measures directly related to each
of them.

Commerce?s combined report and plan is a significant improvement over its
fiscal year 1999 performance report and fiscal year 2001 performance plan
and addresses the recommendations made in our June 2000 report. Major
Management

Challenges Conclusions

Page 19 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

Furthermore, the report indicates that Commerce has made progress toward
achieving two of the selected key outcomes. However, progress toward
achieving the remaining two outcomes is unclear largely because of
weaknesses related to measuring performance. Specifically, some of the
measures are output- oriented, rather than outcome oriented; some measures
have known limitations, which Commerce acknowledges; many of the measures
used to assess performance in the past are being discontinued for the
future; and Commerce plans to rely on one, narrowly focused measure to
demonstrate progress for each performance goal related to these key
outcomes. Furthermore, other data exist within ITA that could support
additional measures related to its performance goals.

We recommend that the Secretary of Commerce direct ITA and BXA to reassess
the measures that are used to assess progress in achieving their
organizational performance goals related to the outcomes covered by this
report. Specifically, ITA and BXA should consider using more than one
measure to address goals that are multifaceted or are difficult to measure.
In addition, ITA should consider developing trade agreement compliance and
enforcement measures based on data currently gathered by its trade
compliance unit.

Our evaluation was generally based on the requirements of GPRA, the Reports
Consolidation Act of 2000, guidance to agencies from the Office of
Management and Budget (OMB) for developing performance plans and reports
(OMB Circular A- 11, Part 2), previous reports and evaluations by us and
others, our knowledge of Commerce?s operations and programs, our
identification of best practices concerning performance planning and
reporting, and our observations on Commerce?s other GPRA- related efforts.
We also discussed our review with Commerce?s Office of Inspector General.
The agency outcomes that were used as the basis for our review were
identified by the Ranking Minority Member of the Senate Governmental Affairs
Committee as important mission areas for the agency and do not reflect the
outcomes for all of Commerce?s programs or activities. The major management
challenges confronting Commerce, including the governmentwide high- risk
areas of strategic human capital management and information security, were
identified in January 2001 in our performance and accountability series and
high- risk update or in December 2000 by Commerce?s Office of Inspector
General. We did not independently verify the information contained in the
performance report and plan, although we did draw from our other work in
assessing the validity, reliability, and timeliness of Commerce?s
performance data. We Recommendation

Scope and Methodology

Page 20 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

conducted our review from April through June 2001 in accordance with
generally accepted government auditing standards.

We provided copies of a draft of this report to Commerce for its review and
comment. We met with the Deputy Director, Office of Budget, and
representatives from many of Commerce?s component agencies to obtain their
comments. In general, they agreed with the information presented in the
draft report and provided technical clarifications, which we incorporated
where appropriate. While Commerce indicated that it would send its comments
in writing, we did not receive written comments in time to include them in
this report.

As arranged with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days after
the date of this letter. At that time, we will send copies to appropriate
congressional committees; the Secretary of Commerce; and the Director,
Office of Management and Budget. Copies will also be made available at www.
gao. gov.

If you or your staff have any questions, please call me at (202) 512- 8678.
Key contributors to this report were Debra Johnson, Stephen Lord, Tina
Morgan, Colleen Phillips, William Ritt, Debra Rucker, Nancy Simmons, and
Pierre Toureille.

Sincerely yours, Davi M. D?Agostino Director, Financial Markets and

Community Investment Agency Comments

Appendix I: Observations on Commerce?s Efforts to Address Its Major
Management Challenges

Page 21 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

The major management challenges confronting Commerce, which include the
governmentwide high- risk areas of strategic human capital management and
information security, are shown in table 1. The first column lists the
challenges that Commerce?s Office of Inspector General and/ or we
identified. 1 The second column discusses the progress Commerce made in
resolving its challenges as discussed in Commerce?s

FY 2000 Annual Program Performance Report and FY 2002 Performance Plan. The
third column discusses the extent to which Commerce?s combined document
includes performance goals and measures to address the challenges.
Commerce?s combined document discussed its progress in resolving all of the
identified challenges.

For its 15 challenges, Commerce?s performance plan contained goals or
measures that were directly related to 11 of the challenges and goals or
measures that were indirectly related to 3 of the challenges. The remaining
challenge was not applicable because it dealt with improving Commerce?s GPRA
efforts.

Table 1: Commerce?s Major Management Challenges Major management challenge

Progress in resolving the major management challenge as discussed in the
fiscal year 2000 performance report Applicable goals and measures in the

fiscal year 2002 performance plan GAO- designated governmentwide high risk
challenges

Strategic human capital management: GAO has identified shortcomings at
multiple agencies involving key elements of modern human capital, including
strategic human capital management planning and organizational alignment;
leadership continuity and succession planning; acquiring and developing
staffs whose size, skills, and deployment meet agency needs; and creating
results- oriented organizational cultures.

Under the Department Management section of the report, Commerce describes
activities undertaken to address departmentwide management challenges. One
of the management challenges listed in this section of the report is ?de-
layer management levels to streamline organizations,? which involves
restructuring organizations to reduce the number of supervisory positions
and eliminate unnecessary managerial levels. Commerce also includes a
performance goal of acquiring, managing, and developing a diverse, skilled,
and flexible staff, using information technology as an essential

Commerce includes six performance measures related to the performance goal
of acquiring, managing, and developing a diverse, skilled, and flexible
staff, using information technology as an essential tool. These measures are
as follows:

 implement a process to identify current and projected mission- related
workforce needs,

 assess human capital and the value of human resources services,

 increase the efficiency and effectiveness of hiring systems,

 increase recruitment opportunities and improve diversity,

 increase the alignment of

1 See Semiannual Report to the Congress, Department of Commerce, Office of
Inspector General, Sept. 30, 2000. According to the semiannual report, these
challenges were selected because they were important to Commerce?s mission
or the nation?s well- being, were complex, involved sizeable expenditures,
or needed management improvements. Appendix I: Observations on Commerce?s

Efforts to Address Its Major Management Challenges

Appendix I: Observations on Commerce?s Efforts to Address Its Major
Management Challenges

Page 22 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

Major management challenge Progress in resolving the major

management challenge as discussed in the fiscal year 2000 performance report
Applicable goals and measures in the

fiscal year 2002 performance plan

tool. Under this goal, Commerce reports that it met all of the targets
related to the six performance measures associated with the goal.
Specifically, Commerce

 developed a workforce analysis plan and identified workforce planning
tools,

 held a departmentwide meeting in August 2000 that involved senior managers
who generated ideas for changing Commerce?s human capital strategy and
incorporating human capital management into all major management systems,

 pilot tested a Web- based recruitment and referral system,

 established formal informationsharing mechanisms with nine colleges and
universities with Hispanic ties,

 integrated performance management and awards, and

 pilot tested a telecommuting program.

In addition, for each component agency, Commerce includes a table of
resource requirements- including fulltime equivalent positions, as well as a
summary of the skills needed in that component agency. For example, BXA?s
skill summary includes an extensive knowledge of the legislation and
Executive Orders related to controlling dual- use commodities; knowledge of
world political and economic systems, current trends in U. S. trade, and
national security and foreign policy issues; and analytical abilities for
complex licensing and policy decisions, as well as regulatory
interpretations.

Several component agencies list challenges related to human capital. The
Bureau of the Census states that its strategies to achieve its mission
include valuing its employees and lists its workplace and workforce as a
management challenge. By fostering

performance management with mission accomplishment and overall recognition,
and

 implement a telecommuting program. The component agencies that included
human capital issues in their priorities or management challenges did not
include specific performance goals or measures related to human capital
management.

In addition, Commerce includes human capital initiatives as part of its
strategy for addressing its challenge strengthening financial management
controls. Specifically, Commerce plans to establish financial leadership
positions at all bureaus and developing a professional education program.

Appendix I: Observations on Commerce?s Efforts to Address Its Major
Management Challenges

Page 23 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

Major management challenge Progress in resolving the major

management challenge as discussed in the fiscal year 2000 performance report
Applicable goals and measures in the

fiscal year 2002 performance plan

diversity in its workforce, the Bureau hopes to create surveys that are more
sensitive to the cultural base of the people being surveyed, as well as
retain a trained and skilled workforce. One of ITA?s management challenges
is to position human capital as a strategic asset. It projects that one-
third of its employees can retire within 5 years and lists strategies for
maintaining its workforce. Similarly, BXA notes that it lacks sufficient
technical and analytical personnel and will lose people who retire.
Consequently, BXA lists maintaining technical and analytical abilities as a
management challenge. NOAA and the Technology Administration discuss
investing in people under their respective sections on priorities.
Information Security: Our January 2001 highrisk update noted that the
agencies? and governmentwide efforts to strengthen information security have
gained momentum and expanded. Nevertheless, recent audits continue to show
federal computer systems are riddled with weaknesses that make them highly
vulnerable to computer- based attacks and place a broad range of critical
operations and assets at risk of fraud, misuse, and disruption.

(In 2000, Commerce?s Inspector General also identified strengthening
departmentwide information security as a management challenge.)

Under the Department Management section of the report, Commerce describes
activities undertaken to address management challenges, one of which is
strengthening departmentwide information security. Commerce?s description of
this management challenge cites recently completed Inspector General reviews
as part of the efforts to resolve this challenge. The report further states
that these Inspector General reviews of the Department?s protection of its
information systems under the critical infrastructure protection program and
the security of its financial management systems and their related networks
(as part of the fiscal year 1999 financial statements audits), confirmed
that information security needs additional management attention and
improvements. The report states that additional reviews are planned as part
of Department Management?s oversight responsibilities under the government
information security reform provisions of the Fiscal Year 2001 Defense
Authorization Act (P. L. 106- 398).

For one performance measure- to protect information and staff at field

The fiscal year 2002 performance plan does not identify a specific
performance goal for strengthening departmentwide information security, but
some of its goals contain measures related to information security. For
example, under Department Management, Commerce?s performance goal to acquire
and manage the fiscal and related resources necessary to support program
goals contains a measure to protect information and staff at field sites
from risk or disaster. For fiscal years 2001 and 2002, the measure will be
to ensure the protection of critical, classified, and sensitive Commerce
computer systems and information from compromise or exploitation by
adversaries, and will include inspecting, accrediting, and certifying
computer systems.

Another performance goal under Department Management is to acquire and
manage the technology resources to support program goals. This goal includes
a measure to increase the maturity of its information technology security
program, which is a new measure for which the first target is established
for fiscal year 2001.

The fiscal year 2002 plan does not highlight the priority for establishing

Appendix I: Observations on Commerce?s Efforts to Address Its Major
Management Challenges

Page 24 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

Major management challenge Progress in resolving the major

management challenge as discussed in the fiscal year 2000 performance report
Applicable goals and measures in the

fiscal year 2002 performance plan

sites from risk or disaster, the plan shows that the targets were met for
fiscal year 2000 with a review of all security containers at 10 Commerce
field facilities. Another measure- to increase information technology
security program maturity- is new and was not measured in fiscal year 2000.

Information security and related issues such as critical infrastructure
protection are also identified in the report as management challenges by the
following component agencies: EDA, BXA, NOAA, the Bureau of the Census, the
Minority Business Development Agency, and the National Institute of
Standards and Technology. The report provides little specific information on
the progress of resolving these challenges at these component agencies, and
the related performance goals or measures identified were shown as new for
fiscal year 2000.

safeguards against unconventional national security threats that was
included in the fiscal year 2001 plan. However, according to the plan, some
of Commerce?s component agencies have performance goals and/ or measures for
critical infrastructure protection. These are as follows:

 The National Institute of Standards and Technology has a goal to protect
the national information infrastructure. The primary focus of this goal is
to establish a grants program for new research into advanced technologies,
measurements, and methods that can raise the level of reliability and
security of critical information technology- based systems and networks. The
current measure for this goal is the timely and successful completion of
activity milestones to establish the program. In the longer term, outcome
measures will be developed that gauge the security, reliability, quality,
and survivability of information technology systems and networks.

 BXA has a new performance goal to ensure that the nation?s infrastructure
components are secured in accordance with an integrated plan. To begin in
fiscal year 2002, the measure for this goal is the number of agency plans
implemented within the framework of the National Critical Infrastructure
Protection Plan.

 The National Telecommunications and Information Administration included a
new performance goal to minimize the effects of crisis by preparing the U.
S. telecommunications and information infrastructure protection programs. To
begin in fiscal year 2002, the measure for this goal is to increase the
number of state, city, and county governments actively engaged in critical
infrastructure protection programs. However, in commenting on a draft of
this report, Commerce officials said that they plan to discontinue this
measure because Commerce did not receive funding for efforts related to it.

Appendix I: Observations on Commerce?s Efforts to Address Its Major
Management Challenges

Page 25 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

Major management challenge Progress in resolving the major

management challenge as discussed in the fiscal year 2000 performance report
Applicable goals and measures in the

fiscal year 2002 performance plan

Other component agencies that identified information security as a
management challenge- EDA, NOAA, the Bureau of the Census, and the Minority
Business Development Agency- do not have specific goals or measures related
to information security identified in the plan. However, in commenting on a
draft of this report, Commerce noted that information security is viewed as
a departmentwide function.

GAO- designated major management challenges

The Economy in Distressed Communities: EDA works with local, state, and
national organizations to help distressed communities compete in regional,
national, and global markets. EDA provides grants for economic planning,
technical assistance, research, and infrastructure improvements. These
grants are intended to be catalysts to stimulate other public and private
investments in distressed communities. Revolving loan funds help communities
and businesses respond to severe economic dislocations caused by layoffs,
plant shutdowns, trade impacts, natural disasters, and the closure of
military bases and energy labs.

This challenge is discussed in the report under the outcome ?Create Jobs and
Maintain Successful Businesses.?

This challenge is discussed in the report under the outcome ?Create Jobs and
Maintain Successful Businesses.?

Accurate and Timely Weather Forecasts and Warnings: NOAA is responsible for
providing accurate and timely weather forecasts and warnings to protect
human life and property. Every year hundreds of lives and billions of
dollars are lost due to severe storms, floods, and other natural hazards.
While preventing these occurrences is not possible, improved prediction
capabilities can minimize impact.

This challenge is discussed in the report under the outcome ?Provide

Accurate and Timely Weather Forecasts and Severe Weather Warnings.?

This challenge is discussed in the report under the outcome ?Provide
Accurate and Timely Weather Forecasts and Severe Weather Warnings.?

Businesses? Access to International Markets: ITA works to promote U. S.
foreign trade by monitoring the international exchange of goods and services
and by helping U. S. companies export, partner with industry, and open
markets. ITA also works with a variety of government agencies and other
organizations to formulate policy and monitor market access to ensure that
U. S. companies are competitive in foreign markets while protecting American
security.

This challenge is discussed in the report under the outcome ?Increase

Access of U. S. Businesses to International Markets.?

This challenge is discussed in the report under the outcome ?Increase Access
of U. S. Businesses to International Markets.?

GAO- and Inspector General- designated major management challenges

Control of Exports of Dual- Use Commodities This is a newly designated
challenge by This challenge is discussed in the report

Appendix I: Observations on Commerce?s Efforts to Address Its Major
Management Challenges

Page 26 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

Major management challenge Progress in resolving the major

management challenge as discussed in the fiscal year 2000 performance report
Applicable goals and measures in the

fiscal year 2002 performance plan

And Chemical Weapons: The United States controls the export of certain goods
and technologies for national security and foreign policy (including
nonproliferation) purposes. BXA issues licenses authorizing businesses to
export certain dual- use commodities. Controls on dual- use items must
balance the need to protect national security and foreign policy interests
with the desire not to unduly hamper trade opportunities and
competitiveness.

the Inspector General. This challenge is discussed in the report under the
outcome ?Ensure That the U. S. Is Secure From Proliferation of Dual- Use
Commodities And Chemical Weapons.?

under the outcome ?Ensure That the U. S. Is Secure From Proliferation of
Dual- Use Commodities And Chemical Weapons.?

Inspector General- designated major management challenges as of September
2000

Strengthen Departmentwide Information Security: Protecting Commerce?s
computer systems from cyber- attack is a significant challenge that
continues to pose a threat to computer system security. A large number of
interconnected, widely distributed computer systems support vital Commerce
operations and provide essential services to the public. These systems
create a need to improve procedural and technical security measures. Strong
computer security measures are vital to protect the secrecy and privacy of
information, the integrity of computer systems and their networks, and the
availability of services to users.

This is a newly designated challenge by the Inspector General.

This challenge is discussed under

?Information Security? as a GAOdesignated governmentwide high- risk
challenge.

This challenge is discussed under

?Information Security? as a GAOdesignated governmentwide high- risk
challenge.

Increase the Accuracy and Control the Cost of the 2000 Decennial Census:
While the Bureau of the Census has received census forms from millions of
households and has attempted to follow- up with millions more households to
collect the census data, it must continue to check the accuracy of the
collected data. The accuracy of the 2000 census count is crucial because,
among other things, it serves as the basis for determining the number of
seats each state will have in the House of Representatives.

Under the Bureau of the Census, Commerce indicated targets for performance
beginning with fiscal year 2001 that would address part of this challenge.

In addition, under the Department Management section of the report, Commerce
describes activities undertaken to address management challenges, one of
which is to increase the accuracy and control the cost of the 2000 decennial
census. Commerce states that this challenge has been and continues to be the
subject of many audits whose recommendations have led to improvements in
this area.

Under the Bureau of the Census, Commerce includes one performance goal that
is indirectly related to the accuracy of the Census- conducting the
decennial census. This goal has one performance measure, which is to
disseminate Census 2000 products; however, it does not address the accuracy
of those products.

There was no specific performance goal related to controlling costs.
However, under the Department Management performance goal of acquiring and
managing the technology resources to support program goals, the performance
measure of increasing the goods and services provided via electronic means,
which covers selected Census surveys, is indirectly related to controlling
costs. Successfully Implement a Departmentwide Financial Management System:
Commerce has had difficulty developing and implementing a single, integrated
financial system. For more than 10 years, Commerce?s lack of such a system
has been reported as a material

Under the Department Management section of the report, Commerce describes
activities undertaken to address management challenges, one of which is to
successfully implement a departmentwide financial management

The Department Management section of Commerce?s combined report and plan
goal of acquiring and managing the fiscal and related resources necessary to
support program goals covers this challenge. It includes a performance

Appendix I: Observations on Commerce?s Efforts to Address Its Major
Management Challenges

Page 27 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

Major management challenge Progress in resolving the major

management challenge as discussed in the fiscal year 2000 performance report
Applicable goals and measures in the

fiscal year 2002 performance plan

internal control weakness in the Secretary?s annual reports to the President
under the Federal Managers? Financial Integrity Act. The financial systems,
taken as a whole, have are not compliant with GAO principles and standards
or with requirements of the Chief Financial Officers Act, the Joint
Financial Management Improvement Program, or OMB. To correct this weakness,
Commerce began planning for the acquisition and development of a
departmentwide financial system- the Commerce Administrative Management
System- in 1992. While the Inspector General believes that fiscal year 1999
marked a turning point for Commerce in its efforts to improve its financial
management systems and resolve this major management challenge, it will
remain until Commerce complies with the federal requirement for a single,
integrated financial system.

system. The report acknowledges that Commerce has reported the lack of a
single, integrated financial system has been reported as a material internal
control weakness for more than 10 years. In response, Commerce began
implementing the Commerce Administrative Management System. In the past 2
years, Commerce has streamlined system requirements and made progress in
implementing the system at EDA, NOAA, the Bureau of the Census, the National
Institute of Standards and Technology, and the Office of the Secretary.
Commerce plans to achieve full implementation by 2004.

measure to ?meet milestones in implementing the departmentwide financial
system (cumulative).? According to the plan, Commerce had implemented the
system in five bureaus by the end of fiscal year 2000.

Successfully Implement the Patent and Trademark Office?s Transition to
Performance- based Organization: The American Inventors Protection Act of
1999 (P. L. 106- 113), which was signed into law on November 29, 1999,
established the U. S. Patent and Trademark Office (PTO) as a performance-
based organization, which provides it with greater flexibility and
independence so that it can be run more like a business; increased authority
and responsibility for decisions about its management and operations; and
more independent control of its budget allocations and expenditures,
personnel decisions and processes, and procurement operations. Despite the
potential benefits, PTO?s transition to a performance- based organization,
which officially began March 29, 2000, is a formidable undertaking.

This is a newly designated challenge by the Inspector General.

PTO reports that it met four of six targets for its three performance goals
in fiscal year 2000. Specifically, to strengthen intellectual property
protection in the United States and abroad, PTO increased the number of
completed technical assistance activities from 99 to 106, exceeding its goal
of 102. PTO attributed this to an increase in the level of requests for
assistance, and, consequently, has increased the fiscal year 2001 target.
PTO also met both of the targets related to patent services by (1) receiving
a satisfaction rating of 64 percent from its customers, which exceed its
target of 60 percent; and (2) handling requests within 25 months of their
filing date. Finally, PTO met one of its three measures related to trademark
services by disposing of or registering trademarks within 17.3 months of
their filing. PTO did not meet the targets for customer satisfaction or
average time to examiner?s first action.

Under the Department Management section of the report, Commerce

PTO has three performance goals and six performance measures that can be
used to assess its performance, and, thus, could be used to indirectly
assess its transition to a performance- based organization. One performance
goal is to strengthen intellectual property protection in the United States
and abroad, making it more accessible, affordable, and enforceable. The
measures for this goal is the number of technical assistance activities
completed in assisting developing countries and countries moving to a market
economy.

A second performance goal is to enhance the quality of patent products and
services, transition to e- government, and optimize patent processing time.
The three measures for this goal are

 the percentage of customers satisfied overall, based on a customer survey;

 the percentage of patents granted that do not qualify for term extension
for exceeding 36 months (this is a new measure beginning in fiscal year
2001); and

 the average number of months from the time a patent request is filed until
an action is taken to issue a patent or abandon the request.

Appendix I: Observations on Commerce?s Efforts to Address Its Major
Management Challenges

Page 28 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

Major management challenge Progress in resolving the major

management challenge as discussed in the fiscal year 2000 performance report
Applicable goals and measures in the

fiscal year 2002 performance plan

describes activities undertaken to address management challenges, one of
which is to successfully implement PTO?s transition to a performancebased
organization.

A third performance goal is to enhance the quality of trademark products and
services, transition to e- government, and minimize trademark processing
time. The three measures for this goal are

 the percentage of customers satisfied overall, based on customer surveys;

 the average number of months to an examiner?s first action; and

 the average number of months from the time a trademark request is filed
until an action is taken to register the trademark or dispose of the
request.

The Department Management section of the plan did not contain goals or
measures related to this challenge. Address Issues Related to the Mission
and Financial Viability of the National Technical Information Service
(NTIS): The Inspector General reported that disappointments related to NTIS?
operating results continued to call into question NTIS? ability to be self-
sustaining. By taking aggressive cost- cutting actions, such as transferring
some of its staff to other Commerce agencies and significantly reducing
spending on key information technology, NTIS was able to avert losses.
However, concern remains about the steady decline in the sales of NTIS?
products and services, which was caused by fundamental changes in the
marketplace.

According to Commerce?s report, NTIS continues to meet the challenge of
permanent preservation and ready access of the taxpayers? investment in
research and development through acquiring, organizing, and preserving the
titles added annually to the permanent collection. At the same time, the
plan states (1) that Commerce asked that NTIS? functions be transferred to
the Library of Congress and (2) that studies are underway to assess NTIS?
mission and functions and their place within the federal government.

In fiscal year 2000, NTIS met its target for number of documents reproduced
from electronic media, but did not meet its target for the number of items
included in its archive.

The Technology Administration section of Commerce?s report lists one
performance goal related to NTIS- to collect, organize, preserve, and
disseminate government scientific, technical, and business- related
information. Under this goal are the following performance measures:

 the number of items in the archive and

 the number of documents reproduced from electronic media.

This goal and its measures are indirectly related to the issue of NTIS?
financial viability in that they provide information on demand for NTIS?
services.

Increase the Effectiveness of Fishery Management: Ensuring healthy stocks of
fish and other marine animals in the coastal waters beyond each state?s
jurisdiction is a federal responsibility that is carried out principally by
NOAA?s National Marine Fisheries Service and eight regional fishery
management councils. Among other things, the councils develop measures-
which are subject to review by the National Marine Fisheries Service and
approval by the Secretary of Commerce- to manage various species. These
measures are

This is a newly designated challenge by the Inspector General.

NOAA indicates that sustaining fisheries is one of its priorities. Its
efforts include programs that will expand observer programs, improve fishery
information technology, provide targeted improvements in stock assessment,
increase days at sea, increase both scientific and vessel crew

NOAA has one performance goal with four measures that can be used to assess
the effectiveness of fishery management. The performance goal of building
sustainable fisheries has the following four performance measures:

 fewer overfished fisheries,

 stocks having sufficient essential fish habitat,

 increase in employment in noncapture fishing and other sectors in fishing
communities, and

Appendix I: Observations on Commerce?s Efforts to Address Its Major
Management Challenges

Page 29 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

Major management challenge Progress in resolving the major

management challenge as discussed in the fiscal year 2000 performance report
Applicable goals and measures in the

fiscal year 2002 performance plan

often controversial because they impose fish quotas that affect both the
survival of a species and the economic health of the fishing industry and
many coastal communities.

staffing, improve fisheries enforcement, and address specific habitat
issues. NOAA met two of the four targets related to performance measures
under its performance goal of building sustainable fisheries. Specifically,
NOAA met its target for having fewer overfished fisheries, but acknowledges
that the changes were mainly due to technical changes in the definition of
overfishing. a Their current and future goals anticipated these changes and
future performance should relate more to progress in attaining this goal.
NOAA also met its target for stocks having sufficient essential fish
habitat. NOAA states that the results for the remaining two measures, which
relate to increasing employment in fishing communities and increasing
economic contribution of aquaculture will not be available until as late as
fiscal year 2002 due to reporting cycles for the data involved.

In addition, under the Department Management section of the report, Commerce
describes activities undertaken to address management challenges, one of
which is to increase the effectiveness of fishery management.

 increase in economic contribution of sustainable aquaculture to gross
domestic product.

The Department Management section of Commerce?s combined report and plan did
not contain goals or measures related to this challenge.

Successfully Implement Acquisition Reform Initiatives: Commerce increasingly
relies on contractors to provide the goods and services essential to its
operations. Commerce spends more than $1 billion each year- about 25 percent
of its annual appropriations- through large contracts and simplified
acquisitions. The purpose of acquisition reform and its streamlining
initiatives is to reduce the time and money spent in acquiring needed goods
and services by relying on the competitive marketplace. While Commerce has
actively participated in implementing acquisition reform, its efforts to
streamline initiatives warrant extra scrutiny.

This is a newly designated challenge by the Inspector General.

Under the Department Management section of the report, Commerce describes
activities undertaken to address management challenges, one of which is to
successfully implement acquisition reform initiatives.

Commerce exceeded its target (75 percent) for credit card purchases by using
a credit card to complete 88 percent of its actions involving less than
$25,000. In addition, Commerce met the June 2000 deadline for submitting its
inventory of activities that can be provided by commercial sources.

The Department Management section of Commerce?s combined report and plan
contains several measures related to this challenge under its performance
goal of acquiring and managing the fiscal and related resources necessary to
support program goals. These measures include

 increase transactions using credit card purchasing,

 expand competition and more accurate federal activities inventory reform
act inventories,

 increase the portion of contract funds obligated for performance- based
contracting (this is a new measure for fiscal year 2001), and

 expand the application of online procurement (this is a new measure for
fiscal year 2001).

Appendix I: Observations on Commerce?s Efforts to Address Its Major
Management Challenges

Page 30 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

Major management challenge Progress in resolving the major

management challenge as discussed in the fiscal year 2000 performance report
Applicable goals and measures in the

fiscal year 2002 performance plan

Strengthen Financial Management Controls in Order to Maintain a ?Clean?
Opinion on the Department?s Consolidated Financial Statements: Federal law
requires audited financial statements that present an entity?s financial
position and results of operations, as well as other information needed by
the Congress, agency executives, and the public to assess management?s
performance. Commerce received its first- ever unqualified (clean) opinion
on its fiscal year 1999 consolidated financial statements. Continued
emphasis on further improvements is essential in order for Commerce and its
reporting entities to correct the material weaknesses and other deficiencies
identified in the fiscal year 1999 audits and to maintain their clean
opinions in future years.

Under the Department Management section of the report, Commerce describes
activities undertaken to address management challenges, one of which is to
strengthen financial management controls in order to maintain a ?clean?
opinion on its consolidated financial statements. Although substantial
improvements have been made in financial management- including receiving its
first unqualified opinion on its fiscal year 1999 consolidated financial
statements- Commerce acknowledges that it needs to create a financial
management environment that provides accurate and timely financial
information. Consequently, Commerce has undertaken three major financial
management initiatives

 improving financial accountability by strengthening the integrity of
financial operations and providing guidance to reporting entities,

 improving financial management systems through the Commerce Administrative
Management System development project, and

 developing human resources by establishing financial leadership positions
at all bureaus and developing a professional education program.

The Department Management section of Commerce?s combined report and plan
goal of acquiring and managing the fiscal and related resources necessary to
support program goals covers this challenge. It includes a performance
measure to ?maintain 100 percent funds covered by clean audits,? which
Commerce reported was met in fiscal year 2000.

Continue to Improve the Department?s Strategic Planning and Performance
Measurement in Accordance With the Government Performance and Results Act:
While the Inspector General and others agreed that Commerce?s fiscal year
2000 annual performance plan was significantly better than its fiscal year
1999 plan, there was room for additional improvement. Of particular concern
was the need for Commerce to ensure that the data to be used in measuring
performance were accurate, complete, and reliable. While Commerce has been
responsive to past criticisms of the documents it has produced to meet GPRA
requirements, continued management attention is essential.

Not applicable Not applicable

Appendix I: Observations on Commerce?s Efforts to Address Its Major
Management Challenges

Page 31 GAO- 01- 793 Commerce's Status in Achieving Key Outcomes

a In commenting on a draft of this report, Commerce officials said that the
definitions related to overfishing change frequently to reflect such
variables as the species that are included in the analyses.

Source: GAO?s analysis of information from Commerce?s Office of Inspector
General and GPRA documents.

(250022)

The first copy of each GAO report is free. Additional copies of reports are
$2 each. A check or money order should be made out to the Superintendent of
Documents. VISA and MasterCard credit cards are also accepted.

Orders for 100 or more copies to be mailed to a single address are
discounted 25 percent.

Orders by mail:

U. S. General Accounting Office P. O. Box 37050 Washington, DC 20013

Orders by visiting:

Room 1100 700 4 th St., NW (corner of 4 th and G Sts. NW) Washington, DC
20013

Orders by phone:

(202) 512- 6000 fax: (202) 512- 6061 TDD (202) 512- 2537

Each day, GAO issues a list of newly available reports and testimony. To
receive facsimile copies of the daily list or any list from the past 30
days, please call (202) 512- 6000 using a touchtone phone. A recorded menu
will provide information on how to obtain these lists.

Orders by Internet

For information on how to access GAO reports on the Internet, send an email
message with ?info? in the body to:

Info@ www. gao. gov or visit GAO?s World Wide Web home page at: http:// www.
gao. gov

Contact one:

 Web site: http:// www. gao. gov/ fraudnet/ fraudnet. htm

 E- mail: fraudnet@ gao. gov

 1- 800- 424- 5454 (automated answering system) Ordering Information

To Report Fraud, Waste, and Abuse in Federal Programs
*** End of document. ***