-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-01-159SP		

TITLE:     Determining Performance and Accountability Challenges 
and High Risks

DATE:   11/01/2000 
				                                                                         
----------------------------------------------------------------- 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Testimony.                                               **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************

GAO- 01- 159SP

A November 2000 Determining Performance and Accountability Challenges and High
Risks

GAO-01-159SP

GAO United States General Accounting Office

Page 1 GAO- 01- 159SP Performance and Accountability Challenges and High
Risks

3 Introduction 4 Criteria for Identifying and Assessing Major Program and
Mission

Areas 5 Criteria for Assessing Agency Management and Program Performance 6
Criteria for Determining if Individual Performance and Accountability

Challenges Merit Designation as High- Risk Areas 7 Criteria for Removing
High- Risk Designations 14 Calling Attention to Performance and
Accountability Challenges and

High- Risk Problems 15 Figure 1: Major Management Functions Key to High-
Performing

Organizations 6 Figure 2: Information Sources for Assessing Major Program
and

Mission Areas and Management Functions 7 Figure 3: Qualitative Factors Used
in Determining High Risk 9 Figure 4: Factors to Consider in Assessing the
Effectiveness of

Corrective Measures to Resolve Material Weaknesses 10 Figure 5: Determining
High- Risk Programs and Functions 12 Figure 6: Criteria for Determining
Governmentwide High Risks 13 Figure 7: Criteria Agencies Must Meet Before
High- Risk Designations

Can Be Removed 15 Foreword

Contents Determining Performance and Accountability Challenges and High
Risks

Figures

Page 2 GAO- 01- 159SP Performance and Accountability Challenges and High
Risks

Page 3 GAO- 01- 159SP Performance and Accountability Challenges and High
Risks

Since 1990, we have periodically reported on government programs and
functions that we have identified as “high risk” because of
their greater vulnerabilities to waste, fraud, abuse, and mismanagement. In
January 1999, we issued our first Performance and Accountability Series,
which discussed the major issues 20 agencies faced in addressing performance
and accountability challenges.

Over time, as high- risk areas have been corrected and other risks have
emerged, we have removed areas from the list and added new ones to keep the
Congress up to date on areas needing attention. In the January 1999 report
series, we advised the Congress that because an increasing amount of
information is becoming available as a result of implementation of various
federal management reform legislation, we planned to reassess the
methodologies and criteria used to determine which operations and functions
should be included in the Performance and Accountability Series and those
which should be designated as high risk.

This document highlights the resulting criteria and process for determining
performance and accountability challenges and those that are deemed to be
high risks. It will be used by GAO auditors in making these determinations
and by the Congress and the executive branch agencies in understanding our
basis for reporting in these areas. As in the past, the ultimate
determinations will continue to involve the independent and objective
judgment of GAO professionals in applying the criteria.

We provided the opportunity to comment on an exposure draft of this document
to interested parties, including congressional staff from the Senate
Committee on Governmental Affairs and the House Committee on Government
Reform, both of which have been involved with the program since its
inception in 1990; heads of major agencies; the Chief Financial Officers
Council; the President's Council on Integrity and Efficiency; and others.
The exposure draft was also posted to our Internet Web site. The comments we
received were generally positive, and we used this input to make a number of
clarifications in finalizing this guidance.

David M. Walker Comptroller General of the United States Foreword

Page 4 GAO- 01- 159SP Performance and Accountability Challenges and High
Risks

We have developed- and will continue to refine- criteria to use as a basis
in forming judgments on

identifying and assessing the performance of the federal government's major
program and mission areas,

assessing agencies' management functions to determine how they contributed
to program performance and affected the agencies' ability to ensure
accountability and achieve results,

determining if individual performance and accountability challenges merit
designation as high- risk areas as well as determining governmentwide high
risks, and

removing high- risk designations. The criteria presented in this guidance
document will be used by GAO auditors to assess performance and
accountability for an agency's major program and mission areas and
management functions. Based on these assessments, we will include in our
Performance and Accountability Series a report for each major agency that
presents the related challenges we identify for that agency.

Before we issue these reports, we will meet with agency representatives to
discuss the preliminary observations from our assessments and will consider
their views in finalizing our determinations of any related performance and
accountability challenges and high- risk areas. As in the past, the ultimate
determinations in these matters will continue to be based on the independent
and objective judgment of GAO professionals in applying the criteria.

In determining performance and accountability challenges and assessing
whether they rise to the level of high- risk exposure, we consider the
requirements of program- specific legislation. We also consider the
requirements of management reform legislation, such as the Chief Financial
Officers Act of 1990, the Government Performance and Results Act of 1993,
the Government Management Reform Act of 1994, the Federal Financial
Management Improvement Act of 1996, and the Clinger- Cohen Act of 1996.
Determining Performance and Accountability

Challenges and High Risks Introduction

Determining Performance and Accountability Challenges and High Risks

Page 5 GAO- 01- 159SP Performance and Accountability Challenges and High
Risks

Also, the Standards for Internal Control in the Federal Government 1 provide
a foundation for assessing the effects and underlying causes of control
weaknesses for major program and mission areas and key management functions.
We issued these standards pursuant to requirements of the Federal Managers'
Financial Integrity Act. Internal control is a major part of managing an
organization. It comprises the plans, methods, and procedures used to meet
missions, goals, and objectives and, in doing so, supports performance-
based management. Internal control also serves as the first line of defense
in safeguarding assets and preventing and detecting errors and fraud. In
short, internal control, which is synonymous with management control, helps
government program managers achieve desired results through effective
stewardship of public resources. The standards provide the overall framework
for establishing and maintaining internal control and for identifying and
addressing major performance and accountability challenges and high- risk
areas.

For each major agency, we will identify major program and mission areas that
will form the primary basis for reporting in the Performance and
Accountability Series and that

are at the center of congressional and executive branch attention,

have high public interest and/ or large- dollar outlays,

figure prominently in agencies' strategic plans and annual performance plans
and reports, or

have known performance and accountability or high- risk issues. The major
program and mission areas on which we will focus will be selected in
consultation with the Congress and coordinated with the Office of Management
and Budget, the top leadership of the major federal agencies, and the
inspectors general at these agencies. Thus, the Performance and
Accountability Series may not include all of an agency's program and mission
areas, but would focus on those that are among the most important for each
agency. Program and mission areas that in the past were designated as high
risk or had performance and accountability

1 GAO/ AIMD- 00- 21. 3.1, November 1999. Criteria for Identifying

and Assessing Major Program and Mission Areas

Determining Performance and Accountability Challenges and High Risks

Page 6 GAO- 01- 159SP Performance and Accountability Challenges and High
Risks

concerns will be carefully scrutinized to determine if the risks and
concerns have been resolved.

We will use agencies' strategic plans, annual performance plans and reports,
accountability reports, and audited financial statements to identify the key
goals, strategies, performance measures, and reported performance for each
of the major program and mission areas. This information, supplemented by
relevant GAO products, inspectors general reports, and other independent
analyses, will enable us to develop a profile of the actual performance for
each of the selected program and mission areas.

Based on our experience in examining a wide range of government programs, we
have found that effective performance of the management functions shown in
figure 1 are key to creating and sustaining highperforming organizations.
Internal control is a critical aspect of each of these management functions.

Figure 1: Major Management Functions Key to High- Performing Organizations

We will assess the results that agencies' major program and mission areas
achieve and how well the management functions key to high performance are
being carried out. When weaknesses are identified in one or more management
functions, it does not necessarily indicate that the entire major program or
mission area is not managed adequately.

For those program and mission areas where performance can be improved
substantially, we will identify the root causes of the current performance
Criteria for Assessing

Agency Management and Program Performance

� Strategic planning

� Budget formulation and execution

� Organizational alignment and control

� Performance measurement

� Human capital strategies

� Financial management

� Information technology

� Acquisition Internal control

underpins each management

function

Determining Performance and Accountability Challenges and High Risks

Page 7 GAO- 01- 159SP Performance and Accountability Challenges and High
Risks

levels and help pinpoint improvement opportunities. Also, our assessments
will help to identify best practices that show the specific contributions
better management makes to high performance.

In addition, in assessing major program and mission areas and management
functions and in making high- risk determinations, we will draw from
information available through a number of sources. We will primarily rely on
recent GAO audit reports and studies flowing from our strategic plan,
legislatively mandated work, or requested efforts. These and other sources
of information we will use are shown in figure 2.

Figure 2: Information Sources for Assessing Major Program and Mission Areas
and Management Functions

An individual performance and accountability challenge merits a high- risk
designation when it meets the criteria presented in this section. We will
first determine whether the performance and accountability challenge
involves

a program or mission area having national significance or

a management function that is key to performance and accountability. We will
then determine whether the risk stems from one of the following: Criteria
for

Determining if Individual Performance and Accountability Challenges Merit
Designation as HighRisk Areas

� GAO audit reports, testimonies, studies, surveys, and ongoing work

� Annual audited financial statements for departments and agencies

� Inspectors general reports

� Agency (1) accountability reports, (2) strategic plans and annual
performance plans and reports, (3) program performance reviews, (4) program
evaluation and other internal reports, and (5) internal control assessments,
including those done under the Federal Managers' Financial Integrity Act

� The President's annual budget showing governmentwide performance plans and
management objectives

� Relevant reports by outside study panels and commissions

� Reports by congressional committees

Determining Performance and Accountability Challenges and High Risks

Page 8 GAO- 01- 159SP Performance and Accountability Challenges and High
Risks

An inherent risk, which may arise when the nature of a program creates
susceptibility to fraud, waste, and abuse. A program involving payments to
claimants for services provided by third parties could involve inherent
risk, for example, due to the need for and difficulty of verifying the
accuracy of a large volume of claims.

A systemic problem, which may arise when the programmatic, management
support, or financial systems, policies, and procedures established by an
agency to carry out a program are ineffective, creating a material weakness.

Next, we will consider a number of qualitative and quantitative factors.
Additionally, before making a high- risk designation, we will consider the
corrective measures an agency may have planned or underway to resolve a
material weakness and the status and effectiveness of these actions. These
considerations, as well as criteria for determining governmentwide high
risks, are discussed further in the following sections. In all cases, the
ultimate determination of high risk will be made based on the independent
and objective judgment of GAO professionals.

We will consider the qualitative factors outlined in figure 3. These
qualitative factors are not meant to be all- inclusive. Other important
qualitative elements of risk may also be applicable to a given situation.
Considering Qualitative

and Quantitative Factors

Determining Performance and Accountability Challenges and High Risks

Page 9 GAO- 01- 159SP Performance and Accountability Challenges and High
Risks

Figure 3: Qualitative Factors Used in Determining High Risk

In addition to qualitative factors, we will also consider the exposure to
loss in monetary or other quantitative terms. At a minimum, $1 billion must
be at risk in areas such as

the value of major assets (e. g., loans receivable) being impaired;

revenue sources (e. g., taxes due) not being realized;

major agency assets (e. g., inventory or property) being lost, stolen,
damaged, wasted, or underutilized;

improper payments; and

Risk is seriously detrimental to

� Health or safety

� Service delivery

� National security

� National defense

� Economic growth

� Privacy or citizens' rights

Risk could result in

� Significantly impaired service

� Program failure

� Significantly reduced effectiveness

� Significantly reduced efficiency

� Injury or loss of life

� Unreliable decision- making data

� Reduced confidence in government

� Unauthorized disclosure, manipulation, or misuse of sensitive information,
such as personal, financial management, or programmatic data maintained in
computerized systems

Determining Performance and Accountability Challenges and High Risks

Page 10 GAO- 01- 159SP Performance and Accountability Challenges and High
Risks

contingencies or potential liabilities (e. g., environmental cleanup costs).

The $1 billion threshold relates to that portion of a major program or
mission area that is at risk not to the financial aspects of the program or
mission area as a whole.

In making high- risk determinations, we will analyze the risks from
qualitative and quantitative standpoints. A program or function may be
highly vulnerable to risk arising from a qualitative factor, such as loss of
life, but may not necessarily meet the minimum quantitative dollar
threshold. Conversely, it is possible for an exposure to be significant
quantitatively, that is, placing $1 billion or more at risk, but not involve
a qualitative factor. In some instances, individual qualitative and
quantitative factors alone will not be high risk, but in combination, they
may call for a high- risk designation. Thus, we will consider the totality
of qualitative and quantitative factors in deciding whether a high- risk
designation is warranted.

Before assigning a high- risk designation, we will determine and assess the
effectiveness of an agency's planned or ongoing corrective actions to
address a material weakness. In this regard, we will consider factors such
as those shown in figure 4.

Figure 4: Factors to Consider in Assessing the Effectiveness of Corrective
Measures to Resolve Material Weaknesses

Considering Corrective Measures

� Whether the agency has demonstrated its commitment to resolving the
problem

� The extent of an agency's progress to strengthen controls to address the
problem

� Whether the proposed remaining corrective action plans are appropriate

� Whether effective solutions will be substantially completed near term, as
further discussed below

� Whether the solutions will resolve the root cause of the problem

Determining Performance and Accountability Challenges and High Risks

Page 11 GAO- 01- 159SP Performance and Accountability Challenges and High
Risks

For our high- risk initiative, near term is considered to be within the 2-
year period covered by the term of the Congress to which a high- risk update
report is addressed. In considering whether a corrective action will be
substantially completed in the near term, a high level of certainty must be
evident. Agencies will need to be able to demonstrate concrete results to
date, with a clear path toward addressing remaining problems. The final
determination will be based on GAO's professional judgment.

The criteria we use to determine high- risk programs and functions,
including qualitative, quantitative, and corrective action factors, are
outlined in figure 5. High- Risk Criteria

Summarized

Determining Performance and Accountability Challenges and High Risks

Page 12 GAO- 01- 159SP Performance and Accountability Challenges and High
Risks

Figure 5: Determining High- Risk Programs and Functions

a See Standards for Internal Control in the Federal Government( GAO/ AIMD-
00- 21. 3.1, November 1999). Are corrective

measures effective?

Yes No Yes

Qualitative factors Quantitative factors

� Significantly impaired service?

� Program failure?

� Significantly reduced effectiveness?

� Significantly reduced efficiency?

� Injury or loss of life?

� Unreliable decision- making data?

� Reduced confidence in government?

� Sensitive information?

� Health and safety?

� Service delivery?

� National security?

� National defense?

� Economic growth?

� Privacy or citizens' rights? Is $1 billion or more at risk in

areas such as:

� The value of major assets (e. g., loans receivables) being impaired?

� Revenue sources (e. g., taxes due) not being realized?

� Major agency assets (e. g., property) being lost, stolen, damaged, or
wasted?

� Improper payments?

� Contingencies or potential liabilities?

No

� Has the agency demonstrated a commitment to resolve the material control
weakness?

� Has substantial progress been made to strengthen controls to address the
risk? a

� Are corrective action plans appropriate?

� Will effective solutions be completed near term and resolve the root
causes of the problem?

Or Yes Yes

Is the exposure to loss material in quantitative terms?

Does the risk stem from

Could the risk be seriously detrimental to

Could the risk result in

Program or function is high risk

Program or function is not

high risk

� A program that has national significance?

� A management function that is key to performance and accountability?

� An inherent vulnerability?

� A systemic problem? Does the exposure

relate to No

No Based on professional judgment,

does the combination of qualitative and quantitative factors

make the program or function high risk?

Determining Performance and Accountability Challenges and High Risks

Page 13 GAO- 01- 159SP Performance and Accountability Challenges and High
Risks

In some instances, several agencies may share a common problem that results
in a high- risk situation. When this occurs, we will use the criteria in
figure 6 to determine whether a governmentwide high- risk designation is
warranted.

Figure 6: Criteria for Determining Governmentwide High Risks

If these criteria are met, we may designate the matter as a governmentwide
high- risk area.

For example, in 1997, GAO designated the Year 2000 (Y2K) computing challenge
as a governmentwide high risk, given its broad- based implications and the
potential serious disruption from a Y2K failure. Subsequently, Y2K was
successfully addressed by providing high- level congressional and executive
branch leadership, understanding the importance of addressing the issues,
providing standard guidance, employing a constructive engagement approach,
facilitating progress and monitoring performance, and implementing
fundamental improvements.

Another example is computer security, which is widely recognized around the
world by public and private organizations as a high- risk problem. In
September 1996, we reported that serious computer security weaknesses were
identified at 10 of the 15 largest federal agencies. In February 1997, we
designated information security as a governmentwide high- risk area because
of growing evidence indicating that controls over computerized operations
were not effective and compelling information that risks were increasing.
Criteria for Determining

Governmentwide High Risks

The material weakness must

� Be evident at multiple agencies

� Affect a significant portion of the government's total budget or other
resources

� Stem from a deficiency that should be monitored and addressed through
individual agency actions as well as through Office of Management and Budget
initiatives, legislative action, and/ or congressional oversight

Determining Performance and Accountability Challenges and High Risks

Page 14 GAO- 01- 159SP Performance and Accountability Challenges and High
Risks

It may be possible for several agencies to have the same material weakness
and for it to rise to a high- risk level for some or all of those agencies,
but based on our criteria, it is not considered to be a governmentwide high
risk. In such cases, the weakness would be reported as a performance and
accountability challenge or a high- risk area for each of the agencies it
affects, as appropriate.

When legislative and agency actions, including those in response to our
recommendations, result in significant progress toward resolving a highrisk
problem, we will remove the high- risk designation. In these cases, we will
continue to closely monitor the areas. If significant problems again arise,
we will consider reapplying the high- risk designation.

Since our high- risk initiative began in 1990, we have removed the high-
risk designation from several programs. For example, in 1991, we designated
the Bank Insurance Fund high risk because unprecedented numbers of bank
failures and insurance losses in the late 1980s and early 1990s had depleted
the fund's reserves. In 1995, we removed this high- risk designation because
the fund's risk had been reduced due to legislative actions to rebuild the
depleted deposit insurance fund and to require significant governance,
regulatory, and accounting reforms.

More recently, in 1999, we determined that the U. S. Customs Service had
made considerable progress in addressing its financial management
weaknesses. Given the significant improvement efforts, including those
related to assessing and collecting revenues, we removed our high- risk
designation from Customs' financial management.

The criteria we use for determining whether to remove a high- risk
designation are shown in figure 7. Criteria for Removing

High- Risk Designations

Determining Performance and Accountability Challenges and High Risks

Page 15 GAO- 01- 159SP Performance and Accountability Challenges and High
Risks

Figure 7: Criteria Agencies Must Meet Before High- Risk Designations Can Be
Removed

The criteria presented in this guidance document are intended to help us,
the Congress, and the executive branch give attention to identifying and
resolving performance and accountability challenges and high- risk problems.
Our Performance and Accountability Series and High- Risk Updates are an
important way to gauge progress in achieving this objective.

Through these efforts, GAO is committed to helping the Congress improve the
economy, efficiency, and effectiveness of federal operations. Doing so
should also enhance the public's respect for and confidence in the federal
government.

(924060) Calling Attention to

Performance and Accountability Challenges and HighRisk Problems

� A demonstrated strong commitment and top leadership support to address the
risk( s)

� The capacity (that is, the people and other resources) to resolve the
risk( s)

� A corrective action plan( s) that

� defines the root causes,

� identifies effective solutions, and

� provides for substantially completing corrective measures near term,
including but not limited to, steps necessary to implement solutions we
recommended

� A program instituted to monitor and independently validate the
effectiveness and sustainability of corrective measures

� The ability to demonstrate progress in having implemented corrective
measures

Ordering Information The first copy of each GAO report is free. Additional
copies of reports are $2 each. A check or money order should be made out to
the Superintendent of Documents. VISA and MasterCard credit cards are
accepted, also.

Orders for 100 or more copies to be mailed to a single address are
discounted 25 percent.

Orders by mail: U. S. General Accounting Office P. O. Box 37050 Washington,
DC 20013

Orders by visiting: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW)
U. S. General Accounting Office Washington, DC

Orders by phone: (202) 512- 6000 fax: (202) 512- 6061 TDD (202) 512- 2537

Each day, GAO issues a list of newly available reports and testimony. To
receive facsimile copies of the daily list or any list from the past 30
days, please call (202) 512- 6000 using a touchtone phone. A recorded menu
will provide information on how to obtain these lists.

Orders by Internet: For information on how to access GAO reports on the
Internet, send an e- mail message with “info” in the body to:
info@ www. gao. gov or visit GAO's World Wide Web home page at: http:// www.
gao. gov

To Report Fraud, Waste, or Abuse in Federal Programs

Contact one: Web site: http:// www. gao. gov/ fraudnet/ fraudnet. htm e-
mail: fraudnet@ gao. gov 1- 800- 424- 5454 (automated answering system)

United States General Accounting Office Washington, D. C. 20548- 0001

Official Business Penalty for Private Use $300

Address Correction Requested Bulk Rate

Postage & Fees Paid GAO Permit No. GI00
*** End of document. ***