Multilateral Development: World Bank Reforms on Schedule But Difficult
Work Remains (Briefing Report, 04/05/95, GAO/NSIAD-95-131BR).
Pursuant to a congressional request, GAO reviewed the World Bank's
progress in implementing its action plan, focusing on the Bank's: (1)
country assistance strategies and portfolio reviews; and (2) independent
inspection panel.
GAO found that: (1) the World Bank is generally on schedule in carrying
out its action plan for reform, however, it is too early to assess the
plan's impact on improving project quality; (2) the Bank prepared 62
country assistance strategies in fiscal year (FY) 1992 which examined
countries' economic performance and development agenda, external factors
affecting financing, and the Bank's strategy for achieving development
objectives; (3) two strategy papers had no information on
creditworthiness or country risk, and none of the strategies contained
data on nondevelopment expenditures; (4) the Bank conducted 38 country
portfolio performance reviews during FY 1994, but it would not release
the reviews because it considered them proprietary data; (5) the
independent Inspection Panel was established in 1993 to investigate
allegations that the Bank had not complied with its own policies; (6)
the Bank has taken steps to encourage openness in its operations, but it
still needs to improve overseas access to public information and
overcome language barriers; and (7) the extent to which the action plan
addresses poverty reduction and sustainable development and
participation in planning by borrowers and affected populations requires
further study.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: NSIAD-95-131BR
TITLE: Multilateral Development: World Bank Reforms on Schedule
But Difficult Work Remains
DATE: 04/05/95
SUBJECT: International economic relations
Foreign economic assistance
Foreign financial assistance
Bank loans
Foreign loans
Economic policies
Financial management
Bank management
**************************************************************************
* This file contains an ASCII representation of the text of a GAO *
* report. Delineations within the text indicating chapter titles, *
* headings, and bullets are preserved. Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are *
* identified by double and single lines. The numbers on the right end *
* of these lines indicate the position of each of the subsections in the *
* document outline. These numbers do NOT correspond with the page *
* numbers of the printed product. *
* *
* No attempt has been made to display graphic images, although figure *
* captions are reproduced. Tables are included, but may not resemble *
* those in the printed version. *
* *
* A printed copy of this report may be obtained from the GAO Document *
* Distribution Facility by calling (202) 512-6000, by faxing your *
* request to (301) 258-4066, or by writing to P.O. Box 6015, *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time. *
**************************************************************************
Cover
================================================================ COVER
Briefing Report to the Chairman, Committee on the Budget, House of
Representatives
April 1995
MULTILATERAL DEVELOPMENT - WORLD
BANK REFORMS ON SCHEDULE BUT
DIFFICULT WORK REMAINS
GAO/NSIAD-95-131BR
Multilateral Development
Abbreviations
=============================================================== ABBREV
CAS - Country Assistance Strategy
CPPR - Country Portfolio Performance Review
IBRD - International Bank for Reconstruction and Development
ICR - Implementation Completion Report
IDA - International Development Association
IEPS - Initial Executive Project Summary
OED - Operations Evaluation Department
PID - Project Information Document
Letter
=============================================================== LETTER
B-260151
April 5, 1995
The Honorable John Kasich
Chairman, Committee on the Budget
House of Representatives
Dear Mr. Chairman:
In June 1994, we reported on the World Bank's efforts to carry out a
plan of action intended to remedy the growing number of
unsatisfactory projects the Bank finances and reorient it to the core
mission of reducing poverty through sustainable development.\1 You
recently requested an update of the Bank's efforts to implement the
87 tasks in its action plan. We determined Bank progress on all
tasks but focused on 12 key tasks related to (1) country assistance
strategies and portfolio reviews, (2) the independent inspection
panel, and (3) project completion reviews. At your request, we also
reviewed the Bank's implementation of its openness policy. On March
20, 1995, we briefed your committee staff on these issues. This
report contains the information conveyed at the briefing.
--------------------
\1 Multilateral Development: Status of World Bank Reforms
(GAO/NSIAD-94-190BR, June 6, 1994).
BACKGROUND
------------------------------------------------------------ Letter :1
The World Bank occupies an important position in donors' development
assistance strategies through two types of loans. The International
Bank for Reconstruction and Development (IBRD) provides project
financing at commercial interest rates for middle-income developing
countries. The International Development Association (IDA) provides
credits on concessional terms to low-income countries. In fiscal
year 1994, the Bank's Board of Executive Directors approved 137 IBRD
loans totaling about
$14.2 billion and IDA credits totaling $6.6 billion. For fiscal year
1996, the administration has requested about $28 million in paid-in
capital for IBRD and about $1.37 billion for IDA.
The Bank's action plan for reform is the result of the 1992 report of
the Bank's Task Force on Portfolio Management.\2 The Task Force found
a growing number of unsuccessful Bank projects and attributed this to
poor project design, management, and implementation; an institutional
culture focused on loan approval and disbursements; country factors,
such as poor economic policies and weak institutional capacities; and
external factors beyond the Bank's control. The action plan,
approved in June 1993, is a set of management and operational reforms
intended to reorient the Bank toward improved portfolio management
and project quality.
--------------------
\2 Effective Implementation: Key to Development Impact, Report of
the Portfolio Management Task Force, World Bank: Sept. 22, 1992.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :2
The World Bank is generally on schedule in carrying out its action
plan for reform; 70 of the plan's 87 tasks had been completed at the
time of our review and the remaining were in process. However, it is
still too early to assess the action plan's impact on improving
project quality. Some tasks are procedural in nature and their value
will be determined as they are incorporated into Bank operations.
For example, tasks such as identifying best practices, producing
reports, and developing guidelines are first steps toward the more
difficult work of changing Bank focus from loan approval to project
management and sustainable development.
Country assistance strategies are becoming an important tool for
portfolio management. The Bank prepared 62 country assistance
strategies in fiscal year 1994. We reviewed 10 strategies written
between May 1993 and May 1994 (including those for Argentina and
Nicaragua where we conducted fieldwork). The strategies we reviewed
generally included most of the key information, but some gaps were
identified. The strategies examined the country's economic
performance and development agenda, external factors affecting
financing, and the Bank's strategy for achieving development
objectives. The strategies also summarized portfolio performance,
based mainly on the country portfolio performance reviews. However,
two strategy papers had no information on creditworthiness or country
risk, and none of the strategies contained data on nondevelopment
expenditures, as required by Bank guidance.
According to Bank officials, country portfolio performance reviews
are used to assess the overall performance of a country's portfolio
and resolve issues based on project-specific deficiencies. According
to a Bank report, the Bank conducted 38 country portfolio performance
reviews during fiscal year 1994. Country assistance strategies are
to take into account the results of these portfolio reviews, and the
reviews are also to be used in considering new loans. The Bank did
not provide us with portfolio performance reviews because it
considers this proprietary information. We therefore could not
confirm the extent of any links among the strategies, the portfolio
reviews, and new lending. According to officials in Argentina, it
will take several years before these linkages are fully established
and their impacts can be seen in Bank project lending patterns.
The Inspection Panel was established in 1993 as an independent forum
to investigate allegations that the Bank has not complied with its
own policies and procedures on particular projects. Currently
working on its first case, the Panel is investigating allegations
that the Bank violated several of its policies or procedures on a
hydroelectric project. The Panel recently determined that three of
the allegations required further investigation, that the Bank
complied with relevant policies and procedures on one, and that the
remaining allegation was not valid. Although the Panel disclosed its
deliberations, the claimants expressed some dissatisfaction with the
results. Nongovernmental organization officials welcome the Panel's
establishment, but have expressed concern about some operating
procedures, such as when information should be made available
directly to the public. World Bank officials pointed out that
claimants have the option of releasing information on the case
directly to the public, but the Bank would not do so while the case
is ongoing.
The Bank has taken steps to encourage openness in its operations.
Upon request, the Bank's Public Information Center provides documents
that give summary information on proposed projects and various
economic, sector, environmental, and technical reports. Under the
new Bank disclosure policy, the Bank must provide sufficient reason
for not disclosing other requested information. Also, the Bank
acknowledges that steps are needed to improve overseas access to
public information and to overcome language barriers.
During the course of our review, officials from nongovernmental
organizations, reports by the World Bank, and other reports
identified concerns about the Bank's reform effort. We did not
conduct a full review of these concerns; however, based on our
collective work on Bank issues to date, we believe two concerns
warrant further study. These include the extent to which the action
plan addresses (1) poverty reduction and sustainable development and
(2) participation in planning and executing Bank projects by
borrowers and affected populations.
SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :3
We interviewed and collected data from World Bank officials in
headquarters offices, including the Office of the U.S. Executive
Director, the Operations Policy Department, and the Operations
Evaluation Department. We verified that the Bank had implemented 26
tasks in the action plan by reviewing supporting documentation. We
further focused on 12 key tasks related to country assistance
strategies and portfolio reviews, the independent inspection panel,
and project completion reviews. Such tasks included (1) identifying
and disseminating best practices for country portfolio performance
reviews and country assistance strategies, (2) reflecting country
portfolio performance in country assistance strategies, (3)
implementing new guidelines for project completion reporting, and (4)
establishing an independent inspection panel. We also reviewed the
Bank's implementation of its openness policy. We discussed Bank
reform efforts with officials of the Treasury and State Departments
and the U.S. Agency for International Development and with officials
of several nongovernmental organizations that closely follow Bank
operations. We visited the Bank's Public Information Center and
obtained access to its publications through the Internet. To
determine how Bank reforms may have affected field operations, we
visited Argentina, a middle-income country with a large hard loan
portfolio, and Nicaragua, a low-income country with a large
concessional loan program.
Our review was conducted between December 1994 and March 1995
according to generally accepted government auditing standards. We
discussed our findings with Treasury and Bank officials and
incorporated their comments as appropriate.
---------------------------------------------------------- Letter :3.1
We are sending copies of this report to the Secretaries of Treasury
and State and the Administrator of the Agency for International
Development. Copies will also be made available to others upon
request.
If you or your staff have any questions about this report, please
call Tetsuo Miyabara, Assistant Director, International Affairs
Issues, on (202) 512-8974. Other major contributors to this report
include Audrey Solis and Sherlie Svestka of the International Affairs
Issues group and Anindya Bhattacharya of the New York Regional
Office.
Sincerely yours,
Joseph E. Kelley
Director-in-Charge
International Affairs Issues
STATUS OF WORLD BANK REFORMS
============================================================ Chapter I
(See figure in printed
edition.)
(See figure in printed
edition.)
(See figure in printed
edition.)
The Bank's action plan responds to the 1992 report of the Task Force
on Portfolio Management that attributed a growth in the number of
unsuccessful Bank projects to poor design and implementation,
borrower policies, and some factors beyond the Bank's control and
made recommendations to strengthen loan portfolio management. Of the
87 tasks in the action plan, 70 have been completed and 17 are in
varying stages of implementation. One task has taken longer than
expected--establishing performance indicators and benchmarks, such as
student enrollment or contraception prevalence rates. The Bank is
developing indicators for 15 sectors. According to a Bank official,
indicators for 13 sectors are in final draft and should be released
by August 1995. The remaining two are to be issued by November 1995.
As we reported in June 1994, it will be several years before the full
impact of the action plan can be determined. The plan is still in
implementation, and efforts to create an internal environment that
supports portfolio management will take years to become evident in
Bank processes.
The Bank and its Board of Executive Directors are monitoring the
reform efforts primarily through its Annual Review of Portfolio
Performance and reports of the Operations Evaluation Department and
the Joint Audit Committee. Bank officials told us that the report of
the Annual Review of Portfolio Performance for 1994, scheduled for
release in April 1995, contains data that will show project
performance improvements, such as loan cancellations and project
restructurings.
(See figure in printed
edition.)
We reviewed 10 CASs reflecting different regions, types of loans, and
portfolio sizes.\1 Each CAS is required to have the following
information: economic and social performance, external factors, the
country's development agenda, and the Bank's assistance strategy.
The assistance strategy section is to include information on
macroeconomic prospects, portfolio implementation and performance,
relevant findings of economic and sector work, relevant governance
issues, and cooperation with other donor institutions. A final
section summarizes issues for Board discussion, including risk
evaluation and criteria for assessing progress in key areas.
The strategies we reviewed generally included the information
required and set forth strategies linked to current conditions,
performance, and development priorities. Most provided ample detail.
For example, the CAS for Tanzania outlined the steps needed to
improve project implementation. It also included discussions on how
lending would be adjusted in the absence of needed economic reforms
and measurement criteria for both economic adjustment and investment
lending. However, some CASs were missing important information. For
example, CASs for countries with International Bank for
Reconstruction and Development (IBRD) loans are to summarize current
and prospective creditworthiness, Bank exposure, and possible
implications for new lending. The China CAS included no discussion
of creditworthiness, despite the fact that China has more than $2.1
billion in IBRD loans for fiscal year 1994. The China CAS neither
evaluated risks nor set forth criteria for assessing progress.
To encourage borrower investment in social sectors, CASs are to
consider trends in nondevelopment spending by the borrower--for
International Development Association (IDA) countries, this includes
military expenditures. The CASs we reviewed did not describe
nondevelopment spending, although the Nicaragua CAS provided
information on personnel reductions in the armed forces.
Officials at Treasury and the U.S. Executive Director's Office
agreed with our overall assessment of the varying quality. However,
they point out that this process was only started in 1992 and that
they have noted continuing improvements in the information and
analysis provided.
(See figure in printed
edition.)
To strengthen the Bank's transition to a country-based lending
strategy from a project-by-project approach, the Task Force
recommended that the results of the country portfolio performance
review be incorporated into each country's lending strategy. The
Task Force had found that the design, composition, and size of future
lending programs did not sufficiently reflect the lessons of
implementation experience. The Bank's action plan includes tasks for
conducting country portfolio performance reviews and incorporating
the results into the CAS document.
During fiscal year 1994, the Bank conducted 38 country portfolio
performance reviews and prepared 62 CASs for Board discussion. We
could not confirm whether the portfolio review results were fully
reflected in the CASs because the Bank declined to provide us with
the portfolio performance documents. The Bank maintains that this is
proprietary borrower information. According to World Bank officials
in Argentina, the impact of these reviews on project lending
strategies will take several years to become apparent.
(See figure in printed
edition.)
The action plan includes two tasks related to revising the format of
the report prepared at the conclusion of the Bank's involvement with
a project--the Implementation Completion Report (ICR). These reports
are analyzed by staff of the Operations Evaluation Department (OED).
Bank officials believe that the revised format will enhance the
evaluations that OED staff conduct on completed projects using the
ICRs. The Director of OED explained that the three new features of
the reports--descriptions of the future of project operations,
assessments by the regional staff responsible for the project, and
the provision of data and analysis to support these assessments--will
give the OED staff a better indication of how the project has
achieved its objectives, its sustainability, the performance of both
the Bank and the borrower, and an assessment of anticipated outcomes.
We reviewed the seven ICRs completed between July 1994 (when they
became mandatory) and January 1995 and found them to be in compliance
with the requirements of the new format. We found in each of these
reports that the required information was provided or explanations
were given for why it could not be provided.
(See figure in printed
edition.)
The Inspection Panel is an independent forum established by the World
Bank's Board of Executive Directors in September 1993 to investigate
complaints from groups directly affected by Bank-financed projects.
The Panel's mandate is to investigate claims that the Bank has failed
to abide by its operational policies and procedures in project design
and implementation. The Panel consists of a full-time chairman and
two part-time members appointed in April 1994. Members are nominated
by the Bank's president and appointed by the Board of Executive
Directors to whom they directly report. The Panel's operating
procedures were adopted in August 1994 and are to be reviewed within
the next year and modified as appropriate.
In October 1994, the Panel received its first formal complaint. It
concerned the Arun III Hydroelectric Project in Nepal and alleged
Bank noncompliance with its operational policies and procedures on
(1) economic evaluation of investment operations, (2) disclosure of
information policy, (3) environmental assessment, (4) involuntary
resettlement, and (5) indigenous peoples. On November 22, 1994, Bank
management responded to the Panel that it had complied with all
requirements. After its initial investigation, the Panel
recommended, and the Board subsequently authorized, a full
investigation of possible violations of Bank policies relating to (1)
environmental assessment, (2) involuntary resettlement, and (3)
indigenous peoples.
The Panel determined that the Bank had followed its procedures with
respect to economic analysis and recommended no further
investigation. Regarding disclosure policies, the Panel determined
the Bank could not be judged by the most recent disclosure policy
that was approved in August 1993, while the initial project
preparation began almost 8 years ago. Correspondence to the Panel
from the claimants in Nepal demanded that compliance with economic
analysis and disclosure policies be included in the full
investigation, but the Panel has not reversed its decision that these
two areas do not warrant further investigation.
Nongovernmental organizations generally welcome the establishment of
the Panel. However, some nongovernmental officials have expressed
concern about several operating procedures, such as when information
should be made available directly to the public. Bank officials
pointed out that the claimants have the option of releasing
information on the case to the general public, but the Bank would not
do so while a case is under investigation.
(See figure in printed
edition.)
The Bank's main venue for disseminating information is the Public
Information Center in Washington, D.C., and its field offices in
London, Paris, and Tokyo. Free documents include (1) Project
Information Documents (PIDs)--the most requested document--which are
brief 2-page summaries on projects being identified for Board
approval, (2) environmental reviews, and (3) "precis"--a series of
findings and recommendations from evaluation studies and audits.
Documents such as project staff appraisal reports and various
economic, sector, technical, and environmental reports are available
at a cost of $15 each. Documents can also be obtained or ordered
from the Center through the Internet.
In 1994, the Center received almost 17,000 information requests--42
percent from businesses, 20 percent from public agencies, 9 percent
from Bank staff, 4 percent from academia, and 2 percent from
nongovernmental organizations. The remaining 23 percent are not
broken down by clientele. Resident missions and country departments
may also provide information not included in the Center's statistics.
The Bank has taken steps to provide more information to the public
and nongovernmental officials have said that the Bank is more open
than in the past; however, some concerns remain. While many
technical reports are available, the most complete technical project
document--the staff appraisal report--is available only after project
approval. Nongovernmental officials remain concerned that important
technical information may not be available for timely input by
affected populations.
A recent Bank report noted that start-up had been slow due to some
unfamiliarity on the part of Bank staff on what was to be made
available to the public but is improving. Concerns remain regarding
access to information in locations where Bank projects are
implemented. A related concern is the language barrier--Bank
documents are available mostly in English, which may not serve local
populations, especially those in remote areas. The Bank plans to
address these concerns by training Bank staff on the openness policy,
translating some documents into French and Spanish, and studying ways
to increase the role of resident missions and local governments to
ensure that key information is available in project areas.
(See figure in printed
edition.)
During our review, we learned that the Bank has undertaken some
reforms that, although not addressed in the action plan, have the
potential for improving Bank operations and project quality. They
include changes in the budget processes, efforts to better
disseminate OED evaluation results, and OED efforts to improve
project proposals.
According to Treasury officials, one of the more important reforms
that the Bank has undergone in recent years is the change in the
budgeting process. Previously all costs were simply presented in
terms of staff time without unit level managers, such as those
responsible for regional programs, having to be concerned about
actual costs. Unit managers now must manage dollar accounts from
which they pay for salaries, travel, benefits, consultants, and
overhead. This new procedure makes managers more accountable for
decisions on how they have chosen to spend the funds allocated to
their unit.
In February 1994, recognizing that lessons learned from previous
evaluation reports are not always reflected in the design of new
projects, the Bank established a task force to review processes and
identify practices worth replicating to disseminate OED evaluation
results. The task force is expected to have a final report by the
end of fiscal year 1995.
In November 1993, OED established a pilot review process to comment
informally on selected projects early in the design phase. OED
reviews the initial executive project summary (IEPS), which is the
earliest formal document outlining the goals of a new lending
operation and pertinent issues, risks, and evaluation experience.
OED received 129 IEPSs in fiscal year 1994 and sent comments on 64
back to the regional staff. In 27 percent of those commented on, the
project goals were found to be poorly defined. In 48 percent, risk
assessments appeared to be unrealistic and in 67 percent, provisions
for monitoring and evaluation were nonexistent or judged
unsatisfactory. OED has reported that regional staff has been
interested in their comments, and OED plans to continue this pilot
activity through fiscal year 1995. The Bank reports that these OED
reviews of IEPSs will enable that department to determine whether
changes in project quality at entry are occurring.
(See figure in printed
edition.)
Nongovernmental organization officials, World Bank reports, and other
reports identified some concerns about World Bank reform efforts. We
did not conduct a full examination of these concerns; however, based
on our collective work on Bank issues to date, we believe two
concerns may warrant further study. First, there is a question of
the extent to which the action plan's management and operational
improvements address poverty reduction and sustainable development.
According to nongovernmental officials, Bank emphasis on management
improvements alone will not be sufficient to address poverty and
promote sustainable development. World Bank officials point out that
the action plan is complemented by other policies and initiatives for
achieving these goals.
The second area is a long-standing concern of nongovernmental
organizations that people affected by Bank projects should be able to
participate in some of the project planning and decision-making.
They point out, and the Bank agrees, that projects would be more
effective if project beneficiaries participated in planning and
therefore had a direct stake in project success. In response to this
criticism, Bank officials said they have made efforts to incorporate
broader participation in planning and implementing projects, as part
of their objective to become a more open and transparent
organization. For example, Bank officials point out that about half
the projects approved in fiscal year 1994 involved nongovernmental
participation. We reviewed 8 project proposals submitted since the
action plan was established, but only two mentioned participation by
affected populations as a part of the process for developing the
proposals.
--------------------
\1 Argentina, Brazil, China, Egypt, Ghana, Indonesia, Kenya, Mexico,
Nicaragua, and Tanzania.