Foreign Assistance: Assessment of Selected USAID Projects in Russia
(Letter Report, 08/03/95, GAO/NSIAD-95-156).
Pursuant to a congressional request, GAO reviewed the Agency for
International Development's (AID) assistance projects in Russia,
focusing on whether: (1) individual AID projects met their objectives
and contributed to systemic reforms; (2) the projects had common
characteristics that contributed to their successful or unsuccessful
outcomes; and (3) AID adequately managed the Russian projects.
GAO found that: (1) some of the projects reviewed fully met or exceeded
their objectives, while other projects met few or none of their
objectives; (2) 3 AID projects contributed to fundamental structural
changes in Russia because they had sustainability built into their
design and they focused on national or regional issues; (3) the
successful projects had broad and strong support from all levels of the
Russian government, U.S. contractors with long-term physical presence in
Russia, a broad scope to maximize benefits, and specific sustainability
objectives, and complemented or supported Russian initiatives; (4)
Russian officials' commitment to reform in certain sectors was critical
to project success; (5) the unsuccessful projects were poorly designed
and implemented and often had little or no impact on problems; (6) AID
made certain exceptions to its normal procedures and processes in its
desire to respond quickly to assist Russia; and (7) AID failed to
adequately manage some projects because of problems in delegating
management and monitoring responsibility to the Moscow AID office,
inadequate staff, and inadequate management information systems.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: NSIAD-95-156
TITLE: Foreign Assistance: Assessment of Selected USAID Projects
in Russia
DATE: 08/03/95
SUBJECT: Federal aid to foreign countries
Foreign economic assistance
Project monitoring
Foreign governments
Human resources utilization
Housing programs
Health care programs
Coal mining
Agricultural programs
Management information systems
IDENTIFIER: Russia
**************************************************************************
* This file contains an ASCII representation of the text of a GAO *
* report. Delineations within the text indicating chapter titles, *
* headings, and bullets are preserved. Major divisions and subdivisions *
* of the text, such as Chapters, Sections, and Appendixes, are *
* identified by double and single lines. The numbers on the right end *
* of these lines indicate the position of each of the subsections in the *
* document outline. These numbers do NOT correspond with the page *
* numbers of the printed product. *
* *
* No attempt has been made to display graphic images, although figure *
* captions are reproduced. Tables are included, but may not resemble *
* those in the printed version. *
* *
* A printed copy of this report may be obtained from the GAO Document *
* Distribution Facility by calling (202) 512-6000, by faxing your *
* request to (301) 258-4066, or by writing to P.O. Box 6015, *
* Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
* for printed documents at this time. *
**************************************************************************
Cover
================================================================ COVER
Report to Congressional Requesters
August 1995
FOREIGN ASSISTANCE - ASSESSMENT OF
SELECTED USAID PROJECTS IN RUSSIA
GAO/NSIAD-95-156
USAID Projects in Russia
Abbreviations
=============================================================== ABBREV
ARC - American Russian Center, University of Alaska-Anchorage
BCG - Boston Consulting Group
CNFA - Citizens Network for Foreign Affairs
GKI - State Committee of the Russian Federation for the Management
of State Property
IBTCI - International Business & Technical Consultants, Inc.
NIS - New Independent States
PADCO - Planning and Development Collaborative International
PIER - Partners in Economic Reform
PIET - Partners for International Education and Training
RCG/HB - RCG/Haggler Bailly
RPC - Russian Privatization Center
TVG - Tri Valley Growers
USAID - U.S. Agency for International Development
Letter
=============================================================== LETTER
B-260365
August 3, 1995
The Honorable Benjamin Gilman
Chairman
The Honorable Lee Hamilton
Ranking Minority Member
Committee on International Relations
House of Representatives
The Honorable Sam Nunn
Ranking Minority Member
Permanent Subcommittee on Investigations
Committee on Governmental Affairs
United States Senate
This report responds to your request that we evaluate assistance
projects in Russia managed by the U.S. Agency for International
Development (USAID). Specifically, we determined whether (1)
individual USAID projects were meeting their objectives and
contributing to systemic reforms, (2) the projects had common
characteristics that contributed to their successful or unsuccessful
outcomes, and (3) USAID was adequately managing its projects in
Russia. In conducting our study, we reviewed
10 judgmentally selected projects with obligations of $64.6 million
as case studies and used audits and evaluations performed by the
USAID Inspector General.
BACKGROUND
------------------------------------------------------------ Letter :1
The United States began providing limited assistance to the Soviet
Union in December 1990 to support the reform effort and then
increased assistance after the Soviet Union dissolved in December
1991. In October 1992, the Freedom for Russia and Emerging Eurasian
Democracies and Open Markets Support Act of 1992 (P.L. 102-511),
commonly known as the Freedom Support Act, was enacted. It further
increased assistance to the former Soviet Union and established a
multiagency approach for providing assistance. It also called for
the designation of a coordinator within the Department of State whose
responsibilities would include designing an assistance and economic
strategy and ensuring program and policy coordination among federal
agencies in carrying out the act's policies.
The Freedom Support Act sets forth the broad policy outline for
helping former Soviet Union countries implement both political and
economic reforms. It also authorized a bilateral assistance program
that is being implemented primarily by USAID. In January 1994, the
State Department Coordinator approved the first overall U.S.
assistance strategy for the former Soviet Union, and in May 1994, the
Coordinator approved the strategy specifically for Russia. This
strategy has three core objectives: (1) help the transition to a
market economy, (2) support the transition to a democratic political
system, and (3) ease the human cost associated with the transition.
As of December 1994, USAID had obligated $1.4 billion and spent $539
million for programs and projects in Russia since fiscal year 1990.
(See app. I.)
USAID's assistance to Russia has focused on 13 sectors, such as
health care and housing, that support the three U.S. objectives.
Hundreds of U.S. contractors and grantees are responsible for
implementing individual projects in the 13 sectors.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :2
Projects have had mixed results in meeting their objectives. Some of
the USAID projects we reviewed fully met most or all of their
objectives, were contributing to systemic reform, and were
sustainable. Others did not have all or some of these attributes of
success.
The more successful projects had several characteristics in common:
(1) all levels of the Russian government provided broad and strong
support, (2) U.S. contractors had a long-term physical presence in
Russia, and (3) these projects had a specific sustainability
objective. Also, successful projects generally complemented or
supported ongoing Russian initiatives. Projects that did not meet
all their objectives lacked one or more of these characteristics,
were poorly designed and implemented, and often had little or no
impact on problems.
USAID did not adequately manage some projects it funded. The
devolution of management and monitoring responsibility from USAID's
Washington office to its Moscow office delayed decision-making and
created confusion among contractors. Furthermore, USAID's management
information systems were inadequate, and it did not adequately
monitor or coordinate some projects. USAID has taken steps to
overcome these problems, including terminating some unsuccessful
projects, refining its assistance strategy, and undertaking efforts
to improve project monitoring and evaluation.
In commenting on this report, USAID said that the difficult operating
environment in which it worked during the first 2 years of the
program in Russia cannot be overstated. We agree that USAID faced
numerous operating obstacles in getting this program underway, and
our observations on how well the projects performed should be seen in
that context.
PROJECT RESULTS HAVE BEEN MIXED
------------------------------------------------------------ Letter :3
The 10 projects we reviewed showed mixed results in meeting their
objectives.
Two projects--coal industry restructuring and housing sector
reform--met or exceeded their objectives.
Five projects--voucher privatization, officer resettlement, small
business development, district heating, and agribusiness
partnerships--met some but not other objectives.
Three projects--health care, commercial real estate, and
environmental policy\1 --met few or none of their objectives.
--------------------
\1 The environmental policy project we reviewed was still in the
early implementation phase. USAID awarded the initial contract in
September 1993 to establish offices and technical support, but did
not sign the agreements to implement the actual project work until
April and September 1994. The projects will extend through September
1997.
PROJECTS THAT CONTRIBUTED TO
SYSTEMIC REFORM
---------------------------------------------------------- Letter :3.1
Three of the 10 projects we examined were contributing significantly
to systemic reform\2 --that is, they were making fundamental
structural changes. These projects were effecting change because
they had sustainability--benefits that extend beyond the project's
life span--built into their design and they focused on issues on a
national or regional scale.
--------------------
\2 A project can contribute to systemic reform without meeting all of
its objectives. Two projects--coal industry and housing sector
reform--contributed to systemic reform and met their objectives. The
privatization project contributed to systemic reform but did not meet
all its objectives.
HOUSING SECTOR PROJECT
-------------------------------------------------------- Letter :3.1.1
The housing sector project helped Russian ministries and agencies
implement 38 laws, regulations, and decrees to reform housing
policies and practices. The Urban Institute, which implemented the
project, also completed a series of pilot projects related to housing
maintenance, mortgage lending, rent reform, and property rights.
Many of the activities affect the entire country or could be
replicated in additional cities. The project helped create new
institutions, strengthened existing ones, and distributed procedural
guides and manuals to local governments as a way to sustain the
reforms.
PRIVATIZATION PROJECT
-------------------------------------------------------- Letter :3.1.2
The contract for implementing the voucher privatization project
called for Deloitte & Touche to establish 35 voucher clearing centers
in cities throughout Russia. This project encountered some
difficulty in meeting its initial time frames and establishing all
the centers, but overall the project was successful. The active
centers handled 70 million voucher transactions as part of Russia's
unprecedented privatization program, and over half the centers
participate in ongoing capital market activities.
COAL RESTRUCTURING
PROJECT
-------------------------------------------------------- Letter :3.1.3
Partners in Economic Reform (PIER), which implemented the coal
restructuring project, has facilitated movement toward the
transformation of the entire coal sector. PIER helped build a
consensus for reform among the Russian government, mine labor unions,
and mine management. It was also instrumental in facilitating a
World Bank review that could lead to a restructuring loan. To
sustain the project's contribution to systemic reform, PIER helped
establish long-term business relations between the Russian and U.S.
coal industries, formed a consortium of U.S. coal-related business,
and is involved in social safety net and new job-creation activities.
Finally, PIER helped facilitate the sale of U.S. equipment in
Russia.
PROJECTS WITH RELATIVELY
LIMITED CONTRIBUTION TO
SYSTEMIC REFORM
---------------------------------------------------------- Letter :3.2
Seven projects we examined did not contribute significantly to
systemic reform because they either did not meet their objectives,
were narrowly focused, or lacked sustainability.
SMALL BUSINESS
DEVELOPMENT PROJECT
-------------------------------------------------------- Letter :3.2.1
The University of Alaska sought to help develop small businesses by
establishing American Russian Centers in four cities across the
Russian Far East. The centers' purpose was to help train
entrepreneurs, help form new businesses, and build lasting business
ties between the region and the United States. To become
self-supporting after USAID stopped funding, the centers planned to
develop partnerships with counterpart institutions. However, the
centers have so far been unable to attract alternative funding.
ENVIRONMENTAL POLICY AND
TECHNOLOGY PROJECT
-------------------------------------------------------- Letter :3.2.2
CH2M Hill International Services, Inc., signed a contract in
September 1993 for an environmental policy development and technology
project. The contractor had difficulty filling critical staff
positions in Russia and providing required work plans for the
activities. Of the work plans due in November 1994, one was approved
in May 1995, while the other was still being revised as of June 1995.
HEALTH CARE TRAINING
PROJECT
-------------------------------------------------------- Letter :3.2.3
The project to provide health care financing training in the United
States to Russian health professionals was implemented by Partners
for International Education and Training (PIET), several training
institutions, and USAID. Although PIET and the institutions provided
the training as required, the Russian participants did not have the
authority, expertise, or resources to make systemic changes. In
addition, changes in Russia's health reform plans have made the
training irrelevant.
COMMERCIAL REAL ESTATE
PROJECT
-------------------------------------------------------- Letter :3.2.4
A commercial real estate project, implemented by International
Business & Technical Consultants, Inc. (IBTCI), was intended to
create a standard approach for increasing the availability of
commercial real estate in six Russian cities. The project design
called for a pilot project/roll-out concept, but IBTCI did not roll
out the pilot in any of the cities and used a different approach in
each city. Also, Russian officials said the project had little or no
effect on the availability of commercial real estate in their cities.
DISTRICT HEATING PROJECT
-------------------------------------------------------- Letter :3.2.5
The district heating project, which USAID recommended we review, was
implemented by RCG/Haggler Bailly and met its objectives primarily by
conducting energy audits and training as well as providing energy
efficiency equipment to two Russian cities. However, as of February
1995, some of the equipment in one of the cities had not been
installed. Russian officials said the equipment may never be
installed because Russian authorities never certified it. USAID had
not monitored the use of the equipment or followed up on the impact
of the studies produced for the project. Consequently, we found no
indication that the project contributed to systemic reform in the
energy efficiency area.
AGRIBUSINESS PARTNERSHIPS
PROJECT
-------------------------------------------------------- Letter :3.2.6
The agribusiness partnerships project, implemented by Tri Valley
Growers, helped two U.S. companies establish joint ventures in two
Russian cities. Although the involvement of U.S. companies
increased the probability that the business ventures would be
sustained, the limited scope of the partnerships makes it unlikely
that they will have a significant effect on reforming Russia's
agricultural sector. USAID has discontinued the entire agribusiness
partnerships project in Russia.
OFFICER RESETTLEMENT
PILOT PROJECT
-------------------------------------------------------- Letter :3.2.7
The Russian officer resettlement pilot project was not intended to be
sustainable after its completion, but instead was motivated by the
United States' desire to encourage the withdrawal of Russian troops
from the Baltic countries.\3 The $6-million pilot project objective
was to construct 450 housing units to resettle demobilized officers
by July 1994. As of February 1995, 422 units were either occupied or
available for use, so in that sense the project was successful.
--------------------
\3 The Baltic countries are Latvia, Lithuania, and Estonia. The
United States never officially recognized them as part of the former
Soviet Union.
COMMON THEMES TO SUCCESSFUL
PROJECTS
------------------------------------------------------------ Letter :4
Successful projects (1) had strong support and involvement at all
levels of the Russian government, (2) had a long-term physical
presence by U.S. contractors in Russia, and (3) were designed to
achieve maximum results by supporting Russian initiatives, having a
broad scope, and including elements that made them sustainable. A
critical element to a project's success was the degree to which
Russian officials were committed to reform in the particular sector.
STRONG SUPPORT AND
INVOLVEMENT AT ALL RUSSIAN
LEVELS
---------------------------------------------------------- Letter :4.1
Russians at both the federal and local levels demonstrated a strong
commitment to the projects that were contributing to systemic reform.
The Russian government also provided financial or in-kind support,
and Russian nationals held leadership roles in the projects. For
example, PIER's approach to implementing the coal project included
working with officials in the Ministry of Fuel and Energy, Fund of
Social Guarantees, and the federal coal company; academic
institutions; oblast'\4 and city officials in the two targeted
regions; local mine management; and representatives of two labor
unions. Russian nationals served as codirectors, and PIER staff
received free apartments and office space.
To accomplish the housing sector reform project, the Urban Institute
worked closely with officials in the Ministries of Finance and
Economy and the State Committee on Architecture and Construction at
the federal level, the Moscow city government, various maintenance
firms, banks, and grass-roots condominium associations. Although
office space in Moscow is expensive and scarce, the Institute
received free office space. In addition, Russian nationals played a
key role on the Institute's staff.
In contrast, many less successful projects lacked the buy-in of
Russians at either the local or federal level and had little Russian
involvement or contribution. For example, the State Committee of the
Russian Federation for the Management of State Property (GKI),\5
Russia's federal agency overseeing the privatization effort, was
instrumental in designing the voucher clearing and commercial real
estate projects. However, in some cities, local officials were not
involved in designing the projects and had little interest in them;
as a result, these projects were not fully successful.
The officer resettlement project established housing in several
cities, but not in Novosibirsk, where city officials reneged on a
previous administration's commitment to provide needed infrastructure
support. Because officials at the federal and oblast' levels were
not involved in the initial agreements, they had no authority to
require the new city administration to fulfill the contract, nor were
they willing to provide additional funding for the project.
The district heating project was not completed in Yekaterinburg
because local officials did not allow monitoring equipment to be
installed. They said the proper Russian authorities had not
certified the equipment.
--------------------
\4 Oblasts, krays, and republics are regional administrative units
similar to states in the United States.
\5 GKI is a Russian acronym.
LONG-TERM PRESENCE BY U.S.
CONTRACTORS IN RUSSIA
---------------------------------------------------------- Letter :4.2
The successful projects usually had long-term advisers living in
Russia, which enabled the advisers to build trust, learn about local
conditions and plan accordingly, monitor progress closely, and
correct problems as they occurred. In addition, successful projects
involved contractors that had appropriate experience to carry out the
project. For example, the Urban Institute has had two long-term
advisers living in Moscow since 1992 who maintained close contact
with Russians involved in housing reforms. PIER's project director
had lived in Moscow for 3 years. Other members of its American staff
had lived in Kemerovo and Vorkuta, the key cities of the major coal
mining oblasts, since 1993 and 1994, respectively. The two field
staff have years of experience as coal mine engineers. Russian
officials at all levels (1) praised PIER's staff; (2) described
PIER's assistance as timely, well-targeted, and beneficial; and (3)
wanted the project to continue and expand.
Contractors implementing many of the less successful projects did not
have staff living in the Russian cities being assisted. For example,
neither IBTCI nor RCG/Haggler Bailly had permanently assigned
American staff in the cities being assisted. IBTCI's consultants
would fly in, make rapid diagnoses, deal with problems quickly, and
then leave. Many U.S. officials, Russians, and contractors said
that relying on "fly-through" consultants rather than permanent staff
was an ineffective approach.
DESIGNED TO MAXIMIZE RESULTS
---------------------------------------------------------- Letter :4.3
Successful projects--the housing reform, voucher privatization, and
coal industry restructuring--were designed to be sustainable, have a
widespread effect, and support existing initiatives. Each project
supported ongoing Russian efforts at widespread reform, considered
local conditions, and contained elements to sustain the effects of
the project beyond its life span.
In contrast, some projects were not designed to maximize their
potential impact. For example, the project design required
RCG/Haggler Bailly to provide energy efficiency equipment and audits
but did not include methods to replicate the project in other cities,
or extend monitoring efforts to determine how the equipment or
studies were used. The USAID Inspector General reported that other
projects did not include any follow-up steps to ensure that the
assistance provided was used.\6 In addition, projects focusing on
health care training and commercial real estate leasing did not
consider local needs and conditions and thus had limited impact.
Several projects did not adequately identify outcomes or measurable
results. For example, the Tri Valley Growers' contract with USAID
did not stipulate how many agribusiness partnerships were to be
established. The design of the coal project also did not adequately
identify outcomes or measurable results, but PIER developed an
effective project nonetheless. The USAID Inspector General found
similar problems when reviewing many projects in the region.\7
--------------------
\6 Audit of the Bureau for Europe's Technical Assistance Contracts
(Report No. 8-180-93-05, June 30, 1993); Audit of the Distribution
of Emergency Medical Supplies to the New Independent States Under
USAID Cooperative Agreement With the People-to-People Health
Foundation "Project Hope" (Report No. 8-110-94-006, Mar. 17, 1994);
and Audit of Activities to Improve Crop Storage Systems in the New
Independent States (Report No. 8-110-94-014, Aug. 31, 1994).
\7 Audit of the Bureau for Europe's Technical Assistance Contracts
(Report No. 8-180-93-05, June 30, 1993); Audit of the ENI Bureau's
Monitoring, Reporting and Evaluation System (Report No.
8-000-95-002, Nov. 28, 1994); and Audit of Selected Privatization
and Restructuring Activities in Russia (Report No. 8-118-95-007,
Mar. 10, 1995).
RUSSIAN INVOLVEMENT AND
COMMITMENT
---------------------------------------------------------- Letter :4.4
It is widely acknowledged that the Russian people themselves will
determine the ultimate success or failure of political and economic
reforms. Without their involvement and commitment to change, outside
assistance will have a limited effect. For example, the support and
involvement of Russian federal agencies, such as GKI in the
privatization effort and the ministries related to housing, ensure
that projects in those sectors are likely to have a wide and
sustained effect. The coal project's impact depends on Russia's
commitment to restructure the coal industry.
In several sectors, a Russian commitment to reform remains elusive.
Powerful factions in the Russian legislative branch strongly oppose
land reform, and the Ministry of Health has not demonstrated a
commitment to health care reform. This lack of commitment raises
concerns that projects in the agriculture and health sectors will not
have widespread benefits. USAID is now working with the Ministry of
Environmental Protection and Natural Resources, but the level of
support from other important federal ministries, including the
Ministry of Finance, is still questionable.
Other domestic conditions will also influence a project's success.
Russia's commitment to breaking up monopolies and its ability to
attract capital for modernizing outdated equipment, restructuring
existing state enterprises, and starting new businesses will affect
the pace and scope of Russia's transformation to a market economy.
Moreover, projects such as introducing mortgage lending will depend
on macroeconomic policy and land reforms. Russia is counting on
foreign capital to help move the transition process forward, but such
factors as the unstable economic situation, a poor and uncertain tax
structure, an undeveloped financial market infrastructure, and an
increased crime rate make foreign investors hesitant to invest.
USAID MANAGEMENT PERFORMANCE
------------------------------------------------------------ Letter :5
USAID responded quickly to assist Russia in undertaking its political
and economic reforms, as called for in the Freedom Support Act.
However, to respond quickly, USAID made certain exceptions to its
normal procedures and processes. Although USAID provided a quick and
flexible response to a fluid, unpredictable situation,\8 we
identified several management problems in addition to design problems
that occurred, in part, because of the quick response. USAID
officials agreed that management problems occurred, but they said the
risks associated with not responding quickly were high.
--------------------
\8 Congress granted USAID special authority to provide a quick
response. Section 201 of the Freedom Support Act of 1992 amended the
Foreign Assistance Act by adding chapter 11, section 498B, that
waived provisions of law in providing assistance to the former Soviet
Union.
LACK OF MONITORING AND
EVALUATION
---------------------------------------------------------- Letter :5.1
The large size of USAID's program, the vast geographic area receiving
assistance, and staff limitations have prevented adequate monitoring
in some cases. We found that USAID officials were unaware of
positive and negative aspects of the projects implemented by IBTCI,
RCG/Haggler Bailly, and PIER. USAID officials had not visited some
projects, and USAID did not have representatives located outside
Moscow. USAID expected its Russian staff to conduct field
monitoring, but the Russian nationals lacked the necessary training.
USAID officials said they considered but rejected the idea of
establishing field offices outside Moscow.
Without adequate staff, USAID relied mainly on contractors' written
and oral reports to monitor the projects, but some contractors did
not report all problems. The USAID Inspector General also found
shortcomings in the reporting process: contractors were not required
to report on their progress toward specific objectives or
indicators.\9
Moreover, USAID did not enforce some of its reporting requirements.
For example, Deloitte & Touche did not provide the required lists of
equipment purchased with USAID funds and brought into the country,
and USAID did not enforce the requirement.
Although the State Department allowed USAID/Moscow to increase U.S.
direct-hire personnel and personal services contractors from 27 in
fiscal year 1993 to 66 in fiscal year 1995, USAID officials said even
more staff were needed to adequately monitor the program. However,
State would not allow the USAID mission to grow further because,
among other reasons, the USAID assistance program is scheduled to end
by the end of the decade.
In some cases, USAID had not determined the relative success or
failure of projects so that it could apply lessons learned to other
efforts. It did not conduct the required periodic
assessments/evaluations of the coal and agribusiness projects. The
omnibus contracts\10 do not require an evaluation of the individual
tasks, but instead evaluations are to be done at the end of the
contracts, according to USAID officials. The omnibus contracts for
USAID's private sector initiatives alone have obligated approximately
$200 million and are not scheduled to terminate until 1996, too late
to apply lessons learned. In addition, an evaluation that was
conducted was not accurate. A contractor evaluated the district
heating project in June 1993 and gave it high marks. Our 1995 review
of the project found major shortcomings, such as equipment still in
boxes after being delivered in 1993, even though the evaluation
report said the equipment had been installed and was being used. The
USAID Inspector General also found that evaluations had not been
conducted and that the quality and impact of some project evaluations
were questionable.\11
--------------------
\9 Audit of Selected Privatization and Restructuring Activities in
Russia (Report No. 8-118-95-007, Mar. 10, 1995) and Audit of the
Distribution of Emergency Medical Supplies to the New Independent
States Under USAID Cooperative Agreement With the People-to-People
Health Foundation "Project Hope" (Report No. 8-110-94-006, Mar. 17,
1994).
\10 USAID's Europe and the New Independent States Bureau often
procured U.S. technical assistance through multipurpose contracts,
commonly referred to as "omnibus" contracts. These contracts
provided for the performance of activities, many of which needed to
be further defined. USAID used the omnibus contracts to retain the
services of U.S. companies to mobilize, either in-house or through
subcontractors, the resources and expertise needed to identify and
implement project activities. The description of work in these
contracts was very general, but required subsequent development of
"task orders" and "work plans" to further define the activities the
contractor was to perform.
\11 Audit of the ENI Bureau's Cooperative Agreement With World
Learning, Inc. for Support to Non-Governmental Organizations in the
New Independent States of the Former Soviet Union
(Report No. 8-110-95-008, Mar. 10, 1995).
PROBLEMS ASSOCIATED WITH
DELEGATED AUTHORITY
---------------------------------------------------------- Letter :5.2
The devolution of management and monitoring responsibility from
USAID's Washington office to a rapidly growing Moscow office has not
been smooth, and several problems have developed as a result. First,
as USAID's Moscow office assumed more management responsibility,
contractors had to begin dealing with another layer of management
review. This caused delays and confusion among some contractors.
Second, there were tensions between the Washington and Moscow offices
because of differences regarding program implementation. For
example, the offices disagreed about which reformers and Russian
government agencies to work with. Third, the USAID/Moscow office
lacked some essential documents to enable officials to carry out
their duties. We found that key contract and financial documents
were not available in Moscow, a problem also reported by the USAID
Inspector General.\12
The State Department Coordinator opposed giving greater authority to
USAID/Moscow because he believed USAID/Washington needed to maintain
a more prominent role. He said that because assistance to Russia is
an important foreign policy issue, key decisions should not be
delegated to the field. State and USAID/Washington officials said
they needed quick access to important project data for reporting
purposes, but quick access to data could not be ensured when projects
were managed by the USAID/Moscow office.
--------------------
\12 Audit of the Bureau of Europe's Technical Assistance Contracts
(Report No. 8-180-93-05, June 30, 1993) and Audit of the ENI
Bureau's Monitoring, Reporting and Evaluation System (Report
No. 8-000-95-002, Nov. 28, 1994).
LACK OF ADEQUATE INFORMATION
SYSTEMS
---------------------------------------------------------- Letter :5.3
USAID has not yet developed a good management information system for
its Russia program. The USAID Inspector General reported that USAID
lacked an information system with baseline data, targets, time
frames, and quantifiable indicators by which to measure program
progress and results.\13 USAID's Bureau for Europe and the New
Independent States was exempted from a new agencywide management
system because the program was intended to be short term and regional
rather than long term and country-specific. USAID officials said the
pressure to provide assistance quickly meant forgoing the traditional
project design process, which included developing progress
indicators.
--------------------
\13 Audit of the ENI Bureau's Monitoring, Reporting and Evaluation
System (Report No. 8-000-95-002, Nov. 28, 1994).
POOR FIELD COORDINATION
---------------------------------------------------------- Letter :5.4
Part of USAID's assistance strategies was to focus on areas where
reformers were willing to make changes. USAID believed this would
help create a synergy that could stimulate the overall impact of the
projects.
Some contractors were not aware of each others' activities. USAID's
management information system did not list contractors by region, and
USAID sometimes did not tell new contractors about other contractors'
activities. In some cities, contractors contacted each other on
their own and started coordinating their efforts. However, this was
being done on an ad hoc basis. In Vladivostok and Yekaterinburg,
U.S. Consuls General facilitated contractor coordination. The USAID
Inspector General found that many projects with similar goals were
not linked to one another.\14 Poor coordination reduced the
opportunity to achieve synergy and targeted impact and gave some
Russians the impression U.S. assistance was fragmented and
uncoordinated.\15
--------------------
\14 Audit of ENI's Strategy for Managing Its Privatization and
Restructuring Activities in Russia (Report No. 8-118-95-009, Mar.
17, 1995).
\15 The Russian government has no agency to coordinate assistance
efforts within the country.
RECOMMENDATIONS
------------------------------------------------------------ Letter :6
We recommend that the USAID Administrator focus assistance efforts on
projects that (1) will contribute to systemic reforms; (2) are
designed to be sustainable; (3) are supported by all levels of
Russian government; and (4) whenever possible, use American
contractors with an in-country presence.
AGENCY COMMENTS AND OUR
EVALUATION
------------------------------------------------------------ Letter :7
In commenting on a draft of this report, USAID said the three
projects that we had deemed to have not met their primary objectives
did produce some positive benefits or it was too early to tell the
impact the projects would have. USAID also said it was aware of the
problems that have occurred and has taken steps to correct them or
terminate activities that could not be fixed. USAID pointed to a new
computerized monitoring system that is expected to produce its first
report in November 1995.
USAID agreed with our recommendation regarding the focus of its
assistance projects and said it was taking steps to implement it. In
addition, USAID said it was taking corrective action to address the
management problems we identified. However, it stressed that its
assistance has had a positive impact and occurred in a difficult
operating environment. USAID indicated that it had made progress in
setting up its own monitoring, reporting, and evaluation system. It
should be pointed out, however, that in November 1994, the USAID
Inspector General reported that the system was still far from able to
measure program results.\16
USAID said that our report would have provided a more balanced and
accurate view of the systemic impact and sustainability of a
project's activities if we had considered the activity in the context
of the whole program. USAID stated that, in nearly every case, the
individual projects we focused on were part of a larger project or
program that would have substantial impact on reforming Russia's
economy. USAID is correct that the projects we examined were usually
one component of a larger sector program; however, USAID is incorrect
in its assertion that we evaluated projects in isolation and without
considering the context of the whole program. The overriding
objective of USAID's program in Russia is to contribute to reforming
both the political and economic systems. This is also the objective
of the assistance program for each sector and, with few exceptions,
of each project that supports a sector program. Our examination
focused on the individual building blocks that support sector
programs and ultimately support the reform effort in Russia. In some
cases, we found that the individual building blocks will not
contribute to systemic reform in the sector or in Russia overall.
Even though this does not mean that an entire program, of which a
less-than-successful project is a part, will fail in its systemic
reform objective, it does mean that an unsuccessful project is not
contributing to a program's success.
We also disagree with USAID's assertion that significant systemic
reform has resulted from USAID activities in all sectors. For
example, the agribusiness partnerships project, including components
reviewed by the USAID Inspector General, comprises most of the USAID
funding going to the sector but is not expected to contribute
significantly to systemic reforms. Only a limited degree of systemic
reform has occurred in other sectors as well, including the health
care and the environmental sectors. We believe that a sector
evaluation, although useful in its own right, would not have allowed
us to draw conclusions about the role and contribution of individual
projects.
USAID provided other comments that we incorporated into the report
where appropriate. The full text of USAID's comments is reprinted in
appendix IV.
--------------------
\16 Audit of the ENI Bureau's Monitoring, Reporting and Evaluation
System (Report No. 8-000-95-002, Nov. 28, 1994).
SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :8
We judgmentally selected 10 individual projects from 6 sectors to
review as case studies. We selected projects based on their
geographic distribution, focusing on regions where several projects
were concentrated. We also considered the level of obligations and
expenditures; the type of assistance provided (e.g., training,
technical assistance, and product delivery); and the type of
contracting vehicle (e.g., cooperative agreements, grants, and
contracts). We generally did not review projects examined by the
USAID Inspector General, although we analyzed its work to assess
whether common themes emerged. (See app. II for a list of the 10
projects we studied and USAID Inspector General reports we reviewed.)
We analyzed USAID and project documents and interviewed USAID and
other U.S. government officials, U.S. contractors, Russian
counterparts, and beneficiaries. We visited project sites in Western
Russia, Siberia, and the Russian Far East in November 1994 and
February 1995.
Appendix III provides a detailed analysis of the 10 projects in our
case study.
We conducted our work from September 1994 to April 1995 in accordance
with generally accepted government auditing standards.
---------------------------------------------------------- Letter :8.1
As agreed with your offices, unless you publicly announce this
report's contents earlier, we plan no further distribution of this
report until 30 days after its issue date. At that time, we will
send copies to the Secretary of State, the Administrator of USAID,
and other interested congressional committees. Copies will also be
made available to others on request.
Please contact me at (202) 512-4128 if you or your staff have any
questions concerning this report. Major contributors to this report
are listed in appendix V.
Harold J. Johnson, Director
International Affairs Issues
OBLIGATIONS AND EXPENDITURES FOR
ASSISTANCE TO RUSSIA
=========================================================== Appendix I
In his January 1995 annual report, the State Department Coordinator
reported about $2.9 billion in obligations and $1.8 billion in
expenditures for Russia through December 1994. (See table I.1.)
Between fiscal years 1990 and 1994, the U.S. Agency for
International Development (USAID) allocated assistance to the New
Independent States (NIS) as a region. During that time, most
projects spanned different countries and USAID did not track how much
money was obligated or expended by country. Thus, USAID country
attributions are estimates and should be treated as such. In fiscal
year 1995, USAID began keeping country accounts.
Table I.1
Obligations and Expenditures for
Assistance to Russia (Fiscal Years 1990
through 1994)
(Dollars in millions)
Agency Obligations Expenditures
-------------------------------- ------------ ------------
USAID direct $1,230.0 $440.4
USAID transfers and allocations 171.1 98.1
to other agencies
============================================================
USAID subtotal 1,401.1 538.5
Department of Defense 291.0 72.9
Department of Agriculture 1,192.0 1,192.0
============================================================
Total $2,884.1 $1,803.4
------------------------------------------------------------
Source: Department of State.
PROJECTS AND USAID INSPECTOR
GENERAL REPORTS GAO REVIEWED
========================================================== Appendix II
We reviewed 10 projects in depth as part of our review. In addition,
we reviewed various reports that USAID's Inspector General has issued
on management issues and projects in Central and East Europe and the
former Soviet Union. The 10 projects and the USAID Inspector General
reports we reviewed are listed in table II.1.
Table II.1
Projects GAO Reviewed
(Dollars in millions)
Start End
Project/implementing partner date date Obligations
------------------------------ ------ ------ ------------
Housing policy reform/Urban Sept. Aug. $5.8
Institute 92 94
Voucher privatization/ Feb. June 4.1
Deloitte & Touche 93 94
Coal project/Partners in June Aug. 8.0\a
Economic Reform 92 95
Small business development/ Apr. May 95 5.1
American Russian Center, 93
University of Alaska
Environmental policy & Apr. Sept. 26.4\b
technology/ 94 97
CH2M Hill International
Services
District heating/RCG Haggler Feb. Dec. 1.3
Bailly 92 93
Health care training/Partners Fall Fall 0.7
for International Education 93 93
and Training
Commercial real estate/ Oct. Dec. 2.0
International Business & 93 94
Technical Consultants, Inc.
Agribusiness partnerships/Tri Feb. Sept. 5.2\a
Valley Growers 93 97\c
Russian officer resettlement/ July July 6.0
Planning and Development 93 94\e
Collaborative International\d
$64.6
------------------------------------------------------------
Note: Obligations are as of March 1995.
\a Amount could be applied to other countries in the NIS as well as
Russia.
\b Total estimated amount of three contracts.
\c USAID stopped accepting proposals for partnerships in Russia in
September 1994.
\d Primary implementing partner.
\e Project had not been completed as of April 1995.
The following are the USAID Inspector General reports GAO reviewed.
Audit of the Bureau for Europe's Technical Assistance Contracts
(Report No. 8-180-93-05, June 30, 1993).
Audit of the ENI Bureau's Monitoring, Reporting and Evaluation
System (Report No. 8-000-95-002, Nov. 28, 1994).
Audit of ENI's Strategy for Managing Its Privatization and
Restructuring Activities in Russia (Report No. 8-118-95-009,
Mar. 17, 1995).
Audit of Selected Privatization and Restructuring Activities in
Russia (Report No. 8-118-95-007, Mar. 10, 1995).
Audit of the Reestablishment of Vaccine Production Activity Under
the New Independent States Health Care Improvement Project No.
110-0004 (Report No. 8-110-94-004, Feb. 25, 1994).
Audit of the Medical Partnerships in Russia and Health Information
Clearing House Activities Under the New Independent States
Health Care Improvement Project (Report No. 8-110-94-005, Feb.
28, 1994).
Audit of the Distribution of Emergency Medical Supplies to the New
Independent States Under USAID Cooperative Agreement With the
People-to-People Health Foundation "Project Hope" (Report No.
8-110-94-006, Mar. 17, 1994).
Audit of the Vulnerable Groups Assistance Program in Russia Under
Project No. 8-110-0001 (Report No. 8-110-93-08, Sept. 24,
1993).
Audit of Activities to Improve Crop Storage Systems in the New
Independent States (Report No. 8-110-94-014, Aug. 31, 1994).
Audit of the ENI Bureau's Cooperative Agreement With World
Learning, Inc., for Support to Non-Governmental Organizations in
the New Independent States of the Former Soviet Union (Report
No. 8-110-95-008, Mar. 10, 1995).
Audit of the Department of Commerce's Special American Business
Internship Training Program in the New Independent States
(Report No. 8-110-93-10, Sept. 24, 1993).
Audit of the Department of Commerce's Consortia of American
Businesses in the New Independent States Program (Report No.
8-110-93-11, Sept. 24, 1993).
Audit of the Nuclear Regulatory Commission's Technical Assistance
Activities in Russia (Report No. 8-118-94-012, June 28, 1994).
Audit of the Department of Energy's Nuclear Safety Technical
Assistance Activities in Russia and Ukraine (Report No.
8-110-95-001, Oct. 7, 1994).
PROJECT SUMMARIES
========================================================= Appendix III
The following provides a detailed analysis of each project that we
reviewed. We selected one project in each of the following areas:
(1) housing policy reform, (2) voucher privatization, (3) coal, (4)
small business development, (5) environmental policy and technology,
(6) heating, (7) health care, (8) commercial real estate, (9)
agribusiness partnerships, and (10) officer resettlement. Each
summary contains information on the problems in the sector, USAID's
project objectives for the selected contract, and the project
approach used by USAID or the contractor. We also provide our
assessment of the contractor's performance, the impact on systemic
reform, and USAID's management of the contract. The projects are
presented on the basis of their capacity to contribute to systemic
reform.
HOUSING POLICY REFORM--URBAN
INSTITUTE
------------------------------------------------------- Appendix III:1
The Urban Institute housing project was successful. It supported
reforms already underway, used an experienced contractor with staff
in country, installed local nationals in high-level positions,
focused its efforts on both the federal and local levels, and
contained elements that made it sustainable. Therefore, this project
will likely have sustained benefits as legislation is implemented and
new Russian institutions expand the pilot projects into other areas.
SECTOR PROBLEM
----------------------------------------------------- Appendix III:1.1
Russia's housing sector \1
has been beset by housing shortages, production inefficiencies,
maintenance problems, and deterioration. This situation occurred
primarily because the state had a monopoly on housing. For example,
it (1) used standardized apartment buildings constructed by
state-owned companies, (2) controlled apartment construction and
maintenance, (3) financed all state housing from state assets, (4)
almost totally subsidized housing and maintenance, (5) guaranteed
low-cost housing, and (6) distributed housing through waiting lists.
In addition, because the Soviet government had not raised rents since
1928, rents covered less than 5 percent of the cost of operating the
apartments in 1990. The problem was exacerbated when the Russian
Federation government stopped paying for maintenance cost of
apartments and they fell into disrepair. In addition, the Federation
devolved the housing assets and responsibilities to municipalities as
a way of relieving itself of the burdens associated with managing the
apartments.
Russia initiated housing reforms in 1991 when it allowed its citizens
to privatize their apartments at little to no cost. This action set
the stage for establishing a private housing sector.
--------------------
\1 This information was obtained from (1) USAID's Housing Sector
Reform Project for the NIS and (2) the Urban Institute's report, The
Russian Housing Market in Transition.
PROJECT OBJECTIVES
----------------------------------------------------- Appendix III:1.2
USAID signed its first contract for housing sector reform with the
Urban Institute in September 1992 for $5.8 million. This 2-year
contract required the Institute to provide draft legislative and
financial advisers to help Russia develop market-oriented housing
programs and legislation. Other Institute advisers were expected to
conduct pilot projects on (1) rent reforms and housing allowances for
the poor, (2) privatized housing maintenance, (3) condominium
associations, and (4) mortgage lending. It was also expected to
provide targeted training to those implementing reforms and develop
local institutions to sustain and expand the reforms. Specific
objectives and milestones were incorporated into the project design.
PROJECT APPROACH
----------------------------------------------------- Appendix III:1.3
A USAID team that included Institute representatives met with their
Russians counterparts in early 1992 to determine their reform
priorities. From 12 to 15 meetings at both the national and
municipal levels in Moscow were needed to clarify Russian reform
priorities. To help focus Russian priorities, the team used a "menu"
of reforms based on experience in housing reforms in Hungary and
developing countries, and then focused on one or two priorities to
demonstrate results quickly and build confidence.
The Russian priorities were formalized through agreements signed in
March 1992 with USAID, the City of Moscow, the State Committee on
Architecture and Construction, the Ministry of Finance, and the
Ministry of Economy. The team sought joint agreements with the three
ministries agencies to (1) ensure it would not become captive to any
one ministry, (2) ensure broad-based agreement on reform priorities,
and (3) reduce governmental impediments to reform. The Russian
counterparts showed their support for the project by providing the
Institute with free office space, which is highly unusual due to the
scarcity of office space in Moscow.
The team's strategy was to work at the national level to help draft
legislation that would shape and codify reforms. In addition, it
planned a series of local demonstration projects to determine the
effectiveness of the designs in the Russian context. The team
augmented these efforts by providing training in Russia and the
United States. A key strategy was to take advantage of the Russian
reforms already underway and try to create "win-win" situations for
the government and its citizens.
The Institute's staffing policies were also important to its
approach. It provided two long-term resident advisers, including the
Program Director, who were located in Moscow. The Director said
using advisers who were permanently located in Russia rather than
"fly-through" consultants helped establish trust with their Russian
counterparts and enabled them to respond immediately. The Institute
also employed five Russian housing experts. Short-term U.S.
advisers were used on an as-needed basis. The Director said that
using local Russians in key positions was critical to establishing
trust with the Institute's Russian counterparts. The large Russian
staff also was less expensive than U.S. consultants.
CONTRACTOR PERFORMANCE
----------------------------------------------------- Appendix III:1.4
The Institute achieved its objectives of helping to develop housing
legislation. According to the Russian Federation Housing Director,
the Institute's assistance was critical in drafting the 38 laws,
decrees, and regulations that have been implemented. These included
laws and regulations on property rights, housing finance, rent
reform, housing allowances for the poor, privatized maintenance,
condominiums, and mortgage lending. The Institute is now the
government's principal housing adviser.
The Institute also achieved its objectives of establishing pilot
projects in four areas: rent reform and housing allowances for the
poor, privatized maintenance, condominiums, and mortgage lending.
RENT REFORM AND HOUSING
ALLOWANCES
----------------------------------------------------- Appendix III:1.5
The Institute helped the City of Moscow develop a program that would
raise rents over a 5-year period until they covered all the costs of
operating the apartments. To reduce resistance to rent increases, it
tied maintenance improvements to the increases so citizens would see
an immediate improvement in their housing conditions. In 1994, the
Federation initiated a national 5-year program to increase tenant
payments to cover the full operating costs. The Institute also
helped the Federation structure a program in which the municipalities
began providing housing allowances to the poor.
PRIVATIZED MAINTENANCE
----------------------------------------------------- Appendix III:1.6
The Institute helped introduce competitive private maintenance for
municipal housing. It conducted training sessions, organized the
competition to select private firms, and conducted a study tour to
the United States so officials could see private maintenance
activities. In March 1993, three private maintenance firms assumed
management of 2,000 apartments in Moscow, and in October 1993,
Moscow's mayor extended the program to all areas of the city. By
1994, over 60,000 apartments were under privatized maintenance, far
surpassing the project's goal of 2,000.
CONDOMINIUM ASSOCIATIONS
----------------------------------------------------- Appendix III:1.7
The Institute's goal for the condominium pilot was to lay the legal
and procedural groundwork by 1994. However, it surpassed this goal
and helped to create 24 functioning condominium associations in
Ryazan'. The regulations it helped develop were instrumental in
registering the first condominium association in Moscow.
HOME MORTGAGES
----------------------------------------------------- Appendix III:1.8
To address mortgage-lending problems, the Institute developed
mortgage-lending facilities at several banks; limited lending has
begun. For example, the Institute helped Mosbusinessbank, Russia's
third largest commercial bank, to make home mortgage loans and
provided assistance in all phases of operations, including legal
documentation, underwriting, loan servicing, mortgage loan instrument
development, and risk management. The Institute then expanded its
work to eight other banks and provided the necessary materials to
other banks to expand and sustain mortgage lending. However,
hyperinflation has precluded lending to most Russians.
RESULTS
----------------------------------------------------- Appendix III:1.9
The Institute's critical assistance helped transform Russian
priorities into workable legislation and pilot projects. Although
the Russians are responsible for the pace of reforms, the Institute
has helped effect systemic changes in Russia's housing sector. It
helped pass far-ranging laws that have codified reforms. The program
to raise rents and provide subsidies for the poor, which is being
implemented across the country, is a fundamental change for the
government and its citizens.
The project has a strong sustainability component. Over the next
several years, it plans to institutionalize the reforms by expanding
the number of demonstration projects and developing private
maintenance organizations, condominium associations, and mortgage
banks. In addition, it created procedures manuals, explanatory
guides, and other necessary documentation on implementing rent
increases, beginning privatized maintenance, creating condominium
associations, and developing mortgage lending. The Institute has
distributed more than 25,000 of these documents, mostly to local
governments.
The project has won high praise from USAID and Russian officials.
The USAID Mission Director in Moscow called the project one of the
most successful ones he had ever seen. A USAID official in
Washington said that, for the money, no USAID project has had more
macroeconomic impact. The Russian Federation Housing Director noted
the Urban Institute's tremendous influence on the government, and
Russian citizens working in maintenance, condominium associations,
and mortgage lending also praised the project.
Despite the program's progress, most Russians have yet to benefit
from the reforms. This is because the reforms are relatively recent,
are tremendously complex, and face opposition by antireformists; they
are also being implemented in a country with no tradition of
market-based decisions. Private land ownership rights are still
generally uncertain, housing and construction mortgages are generally
unavailable, additional laws and regulations are needed, and most
apartment buildings are still maintained by state organizations. In
addition, factors beyond the housing sector, such as macroeconomic
and political instability, slow the transformation to a fully
developed privatized housing sector.
USAID MANAGEMENT
---------------------------------------------------- Appendix III:1.10
USAID successfully managed the contract. It determined the Russians'
reform priorities, incorporated these into its reform plan, and
listed these in its contract. USAID selected a contractor with
experience in both the sector and region and is effectively
monitoring the reforms through regular contacts with the contractor
and Russian counterparts. Both USAID/Washington and USAID/Moscow
agreed on the housing strategy. USAID also had the contractor
develop measurable goals in its annual work plan. USAID then
measured the contractor's progress by comparing its task orders to
the deliverables.
VOUCHER PRIVATIZATION--DELOITTE
& TOUCHE
------------------------------------------------------- Appendix III:2
As part of Russia's privatization effort, Deloitte & Touche
established a national system of centers to process millions of
vouchers that Russians received and used in the privatization
process.
Overall, the Deloitte & Touche voucher privatization project was
successful, with a few exceptions.
The project focused on national reforms, but some areas had lower
Russian participation than expected.
Deloitte & Touche kept USAID and the State Committee of the Russian
Federation for the Management of State Property (GKI) informed
of project progress but did not meet some of its reporting
requirements.
Deloitte & Touche met its amended objective of setting up 30
centers, but many were underused.
Several factors contributed to the overall success of the project.
The Russian GKI helped focus assistance efforts and identified
problems when USAID had minimal field presence. Further, the omnibus
contract system allowed the contractor to institute a rapid roll-out
as well as adjust the scope of work when warranted. In addition,
using existing Russian agencies and using staff and equipment for
follow-on activities increased the project's effects and
sustainability.
SECTOR PROBLEM
----------------------------------------------------- Appendix III:2.1
Because the state controlled Russian enterprises, which were
generally large and monopolistic, the private sector was virtually
nonexistent. The legal and regulatory framework to create the new
system was not in place; few citizens had entrepreneurial experience
or exposure to western management, accounting, and marketing
concepts; and no capital market infrastructure existed.\2
In August 1992, President Yeltsin announced plans to privatize
Russia's large and medium-size state-owned enterprises. Within
weeks, distribution of privatization vouchers began, with each
Russian citizen eligible to receive one voucher. The sale of the
enterprises was expected to reduce the need for massive state
subsidies, begin to reduce inefficiencies, and eventually lead to
higher productivity and innovation as shareholders demand profits.
Voucher privatization was the initial step in the overall
privatization process and was used to transfer ownership from the
state to private individuals. Unlike the approaches used in some
Central European countries, Russia chose to privatize enterprises
before restructuring them.\3 The process is therefore not complete:
restructuring must still take place before the enterprises can
function in a market economy. This may be difficult because
management and workers received a majority of shares and can resist
taking the painful steps necessary for restructuring.
--------------------
\2 We obtained this information mainly from USAID's project
memorandum for private sector initiatives, GKI's annual report, and
the World Bank document, Russia: Creating Private Enterprises and
Efficient Markets.
\3 Enterprise restructuring typically involves organization and
management changes, choice of product markets, development of
marketing and distribution capabilities, and reductions in staff.
PROJECT OBJECTIVES
----------------------------------------------------- Appendix III:2.2
The voucher-clearing centers allowed individual Russians and
investment fund managers to more easily buy shares via electronic
transfers in enterprises located in remote areas. Without the
centers, people would have had to physically transport vouchers to
other parts of the country. There was also a fear that regions would
not let outsiders, including foreigners, buy shares in highly visible
enterprises, thereby allowing insiders and local bureaucrats to
control the process.
The specific objectives of the project required Deloitte & Touche to
establish 35 functioning centers in various Russian cities to verify,
process, and cancel voucher receipts. The project was carried out
under two separate contracts at a total cost of $4.1 million. The
initial contract (as amended) required the contractor to establish 20
centers by the end of 1993, and a task order under the omnibus
contract required 15 more before March 1994. This would give
citizens enough time to process their vouchers before the
privatization program ended in July 1994. USAID and GKI, the Russian
agency overseeing the national privatization effort, hoped that many
of the centers would develop into institutions, such as registrars
and depositories, in the capital market infrastructure. The number
of vouchers the centers were to process was not defined.
PROJECT APPROACH
----------------------------------------------------- Appendix III:2.3
USAID worked closely with GKI on project design, which called for
Deloitte & Touche to develop 4 pilot centers and then establish 16
more after successfully setting up the pilot sites. To provide
broader geographic coverage, USAID and GKI decided to extend the
project and have Deloitte & Touche set up 15 more centers.
Consultants from another USAID contractor, the Harvard Institute for
International Development, worked with GKI to design and monitor the
project.
Deloitte & Touche established a permanent office in Moscow in June
1990 and opened a separate office for this project in early 1993. It
worked closely with GKI in Moscow and GKI's local offices in various
Russian cities to identify appropriate cities for the centers and
suitable partners. Deloitte & Touche then imported computer
equipment, established accounting systems, and installed the software
and telecommunications systems needed to facilitate voucher
transactions. Teams of Deloitte & Touche staff then traveled to the
cities to train center staff, install and test the equipment, and
test the software and telecommunications systems. The contractor
hired Russians to assist with these efforts and usually supplemented
the work of Russian agencies, typically banks, already working in
related fields.
CONTRACTOR PERFORMANCE
----------------------------------------------------- Appendix III:2.4
Under the first contract, Deloitte established all 20 centers before
its deadline. Under the second contract, USAID and GKI reduced the
number of centers from 15 to 10 and extended the deadline by 3-1/2
months because of implementation delays. The delays took place
because of problems with equipment procurement and Russian government
customs clearance; difficulties locating viable agencies to act as
centers; and problems at the local level. For example, some centers
collapsed when their leadership changed or chose not to participate
on a national scale for local political reasons. Both parts of the
project were completed under budget.
Of the 30 centers Deloitte & Touche set up, only 23 were used, and
many of these experienced relatively little activity. The lack of
use was attributed to delays in setting up some centers; limited
public awareness (centers were not responsible for advertising their
services); limited local interest in voucher auctions in other areas;
Russian reluctance to use electronic transfers; and a lack of
compatibility between the project goals and individual center goals.
Deloitte & Touche was responsive to GKI requests for project changes.
The task order was revised once it became clear that all 15 centers
would not be needed. In some cases, Deloitte & Touche went beyond
the required tasks at GKI's request.
Deloitte & Touche generally kept USAID and GKI informed with monthly
reports on progress and problems. However, some reports were not
filed as required, and Deloitte & Touche did not provide an adequate
inventory of the equipment it procured, which would have ensured the
accountability and tracking as required by the company's contracts.
RESULTS
----------------------------------------------------- Appendix III:2.5
The project is considered a success although not cost-effective. A
functioning national system was created in a short time, and
according to GKI, it handled over 70 million vouchers, nearly half
the vouchers processed in the program. People were able to buy
shares in enterprises located in remote areas. GKI noted that over
half of the centers have evolved into institutions that are now
active in capital market activities, such as share registrars and
depositories. Our visits to three centers verified that center staff
and equipment are being used in follow-on activities. These centers
intend to become self-financing on a fee-for-service basis when USAID
assistance ends.
The scope of voucher privatization in Russia was unprecedented in
scale and speed. According to Russian and U.S. officials, USAID's
support of GKI and other Russian institutions involved in
privatization activities was crucial to this phase of the program.
The Russian Privatization Center estimated that 14,000 large and
medium-sized enterprises were privatized by July 1994; they employed
over 60 percent of the industrial workforce. Nevertheless, the
overall effect of the privatization program on Russia has yet to be
determined. Enterprise restructuring has only begun, monopolies
still exist, and inadequate tax legislation makes foreigners
reluctant to provide badly needed capital investment.
USAID MANAGEMENT
----------------------------------------------------- Appendix III:2.6
USAID used an omnibus contract to plan and implement projects
quickly. It allowed USAID to respond quickly to emerging needs
through task orders that included specific objectives for narrowly
focused, short-term projects. USAID officials said this gave them
the flexibility to change directions quickly, move money into areas
and projects making rapid progress in reform, and adjust projects to
meet emerging needs. Omnibus contracts also allowed USAID to
obligate a large amount of funding. Deloitte & Touche has an omnibus
contract for $41.5 million, with subcontractors performing some of
the work. However, the individual task orders lacked an evaluation
requirement. USAID officials said an evaluation is planned only at
the end of the omnibus contract, which ends June 30, 1996.
We identified several USAID management problems. For example, USAID
did not design the project with quantifiable indicators to measure
progress. Although Deloitte & Touche set up 30 voucher clearing
centers, the project design did not specify the amount of activity
expected at each center or on a national scale. USAID/Moscow had
limited information on the project, such as key documents on Deloitte
& Touche's initial contract or task order, and accurate financial
data for the project. USAID officials said key documents had not
been transferred to Moscow when management moved to Moscow from
Washington. Also, the physical distances involved, the geographic
distribution of project activities, and the lack of staff to visit
the sites left USAID uninformed about some Deloitte & Touche
activities. USAID officials said they relied heavily on GKI and
Harvard consultants to help monitor the project. Finally, USAID did
not require Deloitte & Touche to provide adequate inventory data on
the $1.1-million worth of equipment purchased with USAID funds. Not
having this data hindered USAID from using surplus equipment in other
projects as planned.
THE COAL PROJECT--PARTNERS IN
ECONOMIC REFORM
------------------------------------------------------- Appendix III:3
The coal project is achieving its primary objective of facilitating
the restructuring of Russia's coal industry and is opening the
industry to American technology and companies. The Russian
beneficiaries expressed appreciation for the assistance and found it
useful. Due to the size and cost of the restructuring, however, the
Russian government must complete the effort. If the World Bank
approves a $500-million sector loan, this project will have played a
key role in restructuring the coal sector. Although the project is
meeting its objective, USAID did not provide adequate oversight, did
not fully understand the beneficiaries' needs or opinions about USAID
assistance, and did not know the extent to which the project was
meeting its goals.
SECTOR PROBLEM
----------------------------------------------------- Appendix III:3.1
Coal is an important component of Russia's economy. However,
Russia's coal sector suffers from declines in production and serious
environmental and safety liabilities, in large part because of the
centralized structures, subsidized pricing, Soviet-style management,
and state allocation system. To solve these problems, the coal
industry needs to be restructured.\4
The process of restructuring is both a problem and a solution because
it creates new challenges. The major areas that need to be addressed
in restructuring Russia's coal industry and transitioning from a
centrally planned to a market economy include reducing the numbers of
mines and miners as well as the amounts of coal produced and
government subsidies. In addition, the coal monopoly must be broken
up, mines must be privatized, and new relationships and agreements
must be established between management and labor. Efforts to
restructure the coal industry are complicated because the
state-subsidized coal mines provide many social services and may be
the only source of energy or employment in the areas where they are
located.
Coal industry restructuring will take a heavy toll on miners and
their families as the industry streamlines its operations, mines are
closed, and miners lose their jobs. These same miners, who could
lose their jobs as a result of the Yeltsin reform program, were
instrumental in bringing Yeltsin to power in 1991. The mining
community in Russia is still considered a politically powerful force.
President Yeltsin took a major step toward restructuring Russia's
coal industry in July 1993 when he freed coal prices. Since that
time, the industry has made some progress. For example,
approximately 72,000 of the 914,000 coal miners and others employed
by the coal sector in 1992 left the industry between January 1993 and
June 1994. In addition, coal production decreased by approximately
41 percent from 1988 to 1994. The government also reduced subsidies
to the coal industry by approximately 20 percent in real terms (i.e.,
taking inflation into account) between 1993 and 1994. Finally, the
Russian coal industry closed 2 of its approximately 273 mines, was in
the process of closing 14 more mines in 1994, and is preparing to
close 40 more in the future.
--------------------
\4 This information was obtained primarily from coal project
documents and officials; PlanEcon reports; the World Bank's report,
Russian Federation Restructuring the Coal Industry: Putting People
First (Dec. 1994); and the U.S. Department of the Interior's
report, A Cost Comparison of Selected U.S. and Russian Coal Mines
(Sept. 1994).
PROJECT OBJECTIVE
----------------------------------------------------- Appendix III:3.2
As part of USAID's broader effort to assist Russia's energy sector,
USAID signed a cooperative agreement with the Partners In Economic
Reform (PIER), a private, nonprofit organization established to
assist the coal industries in Russia, Ukraine, and Kazakstan.\5
USAID signed the agreement with PIER for $6.9 million in June 1992
and has increased funding since then to $8 million. The project's
main objective is to facilitate the transformation of the centrally
planned and controlled coal mining industry to an industry capable of
operating in a market economy. The cooperative agreement did not
specify any measurable goals or deliverables.
--------------------
\5 The scope of our evaluation is limited to PIER's activities in
Russia.
PROJECT APPROACH
----------------------------------------------------- Appendix III:3.3
In 1989, U.S. coal representatives visited some of the coal regions
of the Soviet Union where miners were starting to form independent
unions, and between 1989 and 1991, groups of independent miners met
with leaders of the U.S. coal industry labor and management in the
United States. In June 1991, a memorandum of understanding,
pertaining to continued assistance, was signed by U.S. and Russian
coal industry representatives.
During 1991, circumstances changed drastically. Boris Yeltsin was
elected President of Russia in June, the communists mounted a failed
coup attempt in August, and the Soviet Union dissolved in December.
These changes opened the door for a broad U.S. technical assistance
program in Russia. As part of this effort, the State Department
announced the coal project in a January 23, 1992, press release and
signed a cooperative agreement on June 25, 1992.
The coal project gained early acceptance because PIER targeted the
project at a problem (i.e., coal industry restructuring) that the
Russians had already identified and were struggling to address. In
addition, PIER established good working relationships. For example,
PIER established a coordinating office in Moscow and cooperation and
development centers in the Russian cities of Kemerovo and Vorkuta.\6
An American director heads the coordination office, and an American
director and a Russian codirector head the cooperation and
development centers. In addition, because the American staff lived
in Russia, they were able to develop and maintain long-term
relationships with the Russian government, coal industry management,
and labor unions. The Russians further demonstrated their support by
donating rare office space for the coordination offices and
cooperation and development centers and donating apartments for the
American directors in Kemerovo and Vorkuta.
PIER staff worked closely with representatives from the Russian
government, coal industry management, and labor to (1) reduce the
number of mines and miners, (2) develop new sources of employment in
coal-producing regions to absorb displaced laborers, and (3) develop
a social safety net for those miners left unemployed during and after
the transition. PIER has cultivated cooperative efforts between
government, management, and labor to address problems associated with
coal industry restructuring. In addition to these efforts, PIER
staff has helped build commercial links between the Russian and
American coal and coal-related industries.
--------------------
\6 PIER has implemented the coal project in three countries--Russia,
Kazakstan, and Ukraine--under one cooperative agreement. PIER also
established coordinating offices and cooperation and development
centers in Ukraine and Kazakstan.
CONTRACTOR PERFORMANCE
----------------------------------------------------- Appendix III:3.4
PIER made progress in facilitating the restructuring of Russia's coal
industry and opening the Russian market to U.S. mining technology.
Specifically, PIER
worked closely with the World Bank to evaluate Russia's coal
industry and develop a restructuring plan;
conducted detailed studies of employment, unemployment, and social
programs; government subsidies; labor demand; a social safety
net, job creation, and mine planning; and enterprise debt in the
Russian coal industry;
established a coal-bed methane recovery center;
mediated discussions between U.S. and Russian officials on
equipment certification in an effort to open the Russian market
to U.S. high-tech safety equipment;
established a program to facilitate U.S. private sector
investment;\7
hosted approximately 150 representatives of the Russian government
and coal industry in the United States where they participated
in meetings and negotiations with World Bank officials, training
seminars, and meetings with U.S. coal industry representatives;
provided training material and conducted seminars in Russia
concerning mine safety, labor-management relations, mining and
mine management in a market economy, and small business
development in Russia;
implemented a transition assistance program focused on developing a
viable social safety net and creating new jobs; and
provided $200,000 worth of U.S.-manufactured mine health and safety
equipment to Russian miners.
The Russian beneficiaries (i.e., government, labor, and management)
we contacted in Russia stated that the coal project was
well-targeted, timely, and beneficial. In addition, they all wanted
the project to be continued and expanded.
--------------------
\7 This program, the PIER-Coalition of American Business in the New
Independent States in Coal Extraction, Processing, Transportation,
and Utilization, is intended to promote joint venture and commercial
activities between U.S. and NIS private sector counterparts in coal
and related industries.
RESULTS
----------------------------------------------------- Appendix III:3.5
PIER has made several contributions to systemic reform. One of the
clearest contributions is its work in facilitating a $500-million
World Bank loan. By providing U.S. coal industry experts, PIER
facilitated the World Bank's work in Russia; contributed extensive
analysis of the coal industry's problems; built consensus among
Russian government, management, and labor representatives; and
brought representative Russian delegations to the United States to
negotiate with the World Bank. The World Bank acknowledged PIER in
its 1994 report for contributing to the Bank's work in Russia.\8
PIER helped establish relationships between Russian and U.S. coal
mining and equipment-manufacturing firms. According to the
beneficiaries, these relationships will help Russia attract capital
investors and gain greater access to U.S. expertise and technology
so that it can begin to produce coal efficiently and compete in a
market economy. PIER also facilitated the sale of millions of
dollars of non-USAID-funded U.S. mining equipment in Russia. PIER
formed a consortium of U.S. industry representatives to help create
a viable private coal industrial sector. The consortium is to assist
coal managers and technicians in operating in a market economy,
identify ways that private U.S. firms can participate in
restructuring the coal industry, establish NIS-U.S. joint ventures,
and promote the consortium's services so it can become
self-sustaining.
Finally, PIER worked with Russia's Fund for Social Guarantees to
implement a transition assistance program focused on developing a
viable social safety net and creating new jobs. PIER also brought in
U.S. experts to provide small business education to miners and
helped mining communities develop business proposals that can be
presented to the Russian-American Enterprise Fund, Russian banks, and
other sources for eventual financing.
--------------------
\8 According to a World Bank official, to receive a loan from the
Bank, the Russian government must meet the Bank's conditions, one of
which requires developing a restructuring plan. The Russian
government provided the Bank with an initial restructuring plan and
is currently refining the plan to address the Bank's concerns.
USAID MANAGEMENT
----------------------------------------------------- Appendix III:3.6
USAID started to implement the coal project before it had established
a USAID mission in Moscow; consequently, the USAID project officer in
Washington managed the project. Since the coal project was
established through a cooperative agreement,\9 without quantifiable
indicators, PIER designed and implemented the project without direct
oversight and control by USAID. PIER provided the required monthly
program performance reports, annual work plans, and annual progress
reports to USAID,\10 which then reviewed them. PIER's staff
communicated regularly with USAID and felt they had a good reciprocal
working relationship.
Despite some success with the project, USAID did not meet its
monitoring and evaluation requirements. Although the USAID staff
should have regularly monitored the project, they visited the Russian
project sites three times between June 1992 and February 1995. Two
of the visits occurred after we began our review. USAID officials
said a lack of staff prohibited more frequent visits. In addition,
USAID did not conduct the annual assessments or midterm evaluation as
required and thus lacked an objective basis for evaluating PIER's
activities and accomplishments. This, coupled with a lack of
quantifiable indicators, hindered USAID's ability to independently
determine the project's impact on coal sector restructuring.
--------------------
\9 Cooperative agreements, similar to grants, are awarded to
nonprofit organizations whose activities are consistent with USAID
objectives. USAID is not expected to exercise a substantial degree
of operational control, but is responsible for approving the
implementation plans, budgets, and subgrants of the organizations'
programs.
\10 PIER also provided USAID with periodic financial reports, and an
independent accounting firm audited PIER's financial statements in
1992, 1993, and 1994.
SMALL BUSINESS
DEVELOPMENT--UNIVERSITY OF
ALASKA
------------------------------------------------------- Appendix III:4
The University of Alaska met most of its project objectives while
encouraging systemic reform, but to date the project has not become
self- sustaining. The American staff live in Russia and have built
trust with Russian officials and institutions, and Russians support
the project with personnel and in-kind contributions.
SECTOR PROBLEM
----------------------------------------------------- Appendix III:4.1
New enterprises are a major source of new jobs for most economies.
However, the development of new enterprises in Russia has been
hampered by years of central control of the economy, excessive rules
and procedures for establishing a business, and the lack of
entrepreneurial skills.
PROJECT OBJECTIVE
----------------------------------------------------- Appendix III:4.2
To help promote the growth of small, private businesses and alleviate
unemployment, the United States supported the creation of
multipurpose business development centers in several Russian cities.
These centers provided training and advisory services to small
businesses and worked with local governments to create a hospitable
environment for private business growth. USAID's goal is that the
centers eventually be operated by trained Russians on a
self-financing, fee-for-service basis.
The American Russian Center (ARC), established by the University of
Alaska in Anchorage through an USAID cooperative agreement, was one
of the first contractors in USAID's program to establish new
businesses. The program's two phases, conducted over 2 years, cost
about $5.1 million. The agreement called for ARC to provide small
business training, develop Russian business activities in specific
geographic areas, and develop business ties between the Russian Far
East and the United States.
ARC's initial objectives were to establish a headquarters at the
University of Alaska and two field centers, as well as to train a
specific number of people. A subsequent work plan called for ARC to
establish two more centers while expanding its program in the two
original centers. Specific objectives included increasing (1) the
number of Russians trained in modern business methods, (2) the number
of viable Russian small businesses, (3) access to both U.S. and
Russian technologies, (4) U.S.-Russian business ties through ARC
field and business information centers, and (5) U.S. business
activity in the Russian Far East. Creating Russian institutions that
would be sustainable after USAID assistance ended was also an
objective.
PROJECT APPROACH
----------------------------------------------------- Appendix III:4.3
From its headquarters at the University of Alaska, ARC worked closely
with Russian partners to establish business training centers in four
Russian cities. In each city, ARC had a local educational or
academic institution as a partner. This partnership was reflected in
the American and Russian codirectors and staff at each center and
in-kind contributions such as free office space from the
institutions.
ARC's American staff have had a long-term commitment to Russia.
Full-time staff spoke Russian fluently, lived in the cities where the
centers were located, and had business experience in the region.
They were complemented by short-term American teachers who taught a
1- or 2-month course as well as by itinerant teachers who taught a
1-week or weekend course in one city and then moved to another city.
These courses were taught with interpreters.
ARC's core program was an evening course that taught such skills as
accounting, marketing, and management that were necessary to write a
business plan. This course lasted 1 or 2 months, depending on the
center, the time of year, and the targeted clientele. It was
supplemented by short seminars in the host cities and extension
seminars in outlying cities and was targeted at specific business
sectors, such as bankers lending to small businesses. Russians and
Americans, both resident and visiting, taught the courses and
seminars. The centers also provided business counseling for Russians
trying to set up their own small businesses. The training centers
charged a relatively low fee for its courses and seminars.
Participants who excelled in the training center programs were
invited for advanced training in Anchorage. They were selected, in
large part, through the business plans they wrote during their core
course. In Anchorage, they attended a 5-week course that explored
topics from the earlier training in more depth, and toured stores,
manufacturing facilities, and offices in the Anchorage area. The
5-week course was followed by 2 weeks of either internships in local
small businesses or more extensive business tours tailored to the
participants' interests.
CONTRACTOR PERFORMANCE
----------------------------------------------------- Appendix III:4.4
ARC successfully fulfilled its first year's work plan targets, and
then received $3 million for a second year's activities (fiscal year
1994) from USAID after a March 1994 evaluation of the initial
$2.1-million project. USAID also stipulated that, in fiscal year
1995, ARC must match USAID's funding. ARC established business
training centers in four Russian cities: Yuzhno-Sakhalinsk, Yakutsk,
Khabarovsk, and Magadan. It established the Yuzhno-Sakhalinsk and
Yakutsk centers in the fall of 1993 and Khabarovsk and Magadan
centers in the fall of 1994. In May 1995, USAID agreed to provide
ARC with an additional $1.5 million, even though ARC had not raised
any matching funds.
Between the fall of 1993 and January 1995, ARC's Yuzhno-Sakhalinsk
and Yakutsk Business Training Centers offered four cycles of evening
courses, lasting 1 or 2 months, that trained 211 Russians--thereby
exceeding the first year's work plan goal of 200 participants. The
two centers also provided individual business counseling to 300
Russians; the work plan's goal was 200. In addition, the two centers
offered 7 extension seminars to 103 Russians in outlying cities. The
training centers in Khabarovsk and Magadan had only recently
completed their first evening courses.
ARC sponsored 19 technical assistance seminars, meeting the first
year's work plan goal of 15 to 20 seminars. Four seminars on banking
drew
180 Russian bankers, and 8 seminars on hair salon management drew
250 women from throughout the Russian Far East. Forty construction
managers from Yakutsk participated in training on cold weather
construction methods. This seminar led to the government of Yakutsk
testing American-manufactured plastic piping to replace its existing
steel piping.
Between the fall of 1993 and January 1995, 71 Russians completed the
advanced business training courses at the University of Alaska. This
exceeded the first year's work plan goal of 50. In total, ARC
trained 1,646 Russians in its USAID-financed programs through January
1995.\11
On a more systemic level, ARC developed a database of U.S. and
Russian businesses in the Russian Far East and provided assistance or
information to U.S. and Russian businesses working throughout the
region.
ARC generally coordinated its activities with other U.S. government
programs located in cities of the Russian Far East, but there were a
few exceptions, particularly when contractors worked in separate
sectors. For example, in Khabarovsk, where ARC established a center
in late 1994, the local American codirector did not know the local
environment project director until we visited. ARC officials in
Anchorage were, however, working with CH2M Hill staff to link the
projects.
--------------------
\11 ARC also has a U.S. Information Agency grant that it uses to
sponsor exchanges.
RESULTS
----------------------------------------------------- Appendix III:4.5
The ARC project will contribute to systemic reform on a regional
basis if it can become financially self-supporting. USAID recognizes
that creating small businesses in the region will push the Russian
government to be more responsive and further develop the area's
nascent capitalist economy.
The centers help Russians who come to Anchorage from their relatively
isolated cities\12 to meet each other and develop business contacts
with other Russians as well as Americans. By drawing entrepreneurs
from cities throughout the Russian Far East, ARC has helped build a
network of private small businesses that can generate business for
one another and for the region.
Once the USAID funding ends, ARC's partnerships with Russian
institutions will be the key ingredient to sustaining its work. The
directors of the Russian institutions plan to continue the program.
For example, the Rector of the Far Eastern State Transport Academy,
ARC's partner in Khabarovsk, plans to establish a permanent school
based on the activities of the project. Kray and oblast' government
officials are also highly supportive of ARC. However, despite the
Russians' desire to continue the program, most of these institutions
currently lack the means to support an entire local ARC operation.
Further, in an April 1994 evaluation report, USAID questioned whether
ARC would be able to support itself. Other donors have not yet
stepped forward to replace USAID. According to ARC's director, the
problem lies in the newness of the project and the project's focus on
Alaska and its businesses. The ARC director said the project plans
to include more business internships and tours in the rest of the
United States. The director believes that this expanded scope will
increase ARC funding because large U.S. institutions and enterprises
have the funds and business interests in Russia to provide long-term
support. If USAID assistance had ended with fiscal year 1994 funds,
the U.S. side of ARC would have been curtailed and U.S. personnel
in Russia would have been withdrawn.
--------------------
\12 Except for Khabarovsk, each of the cities was closed to
foreigners before the collapse of the Soviet Union.
USAID MANAGEMENT
----------------------------------------------------- Appendix III:4.6
USAID officials played a significant role in designing the project.
Because the University of Alaska had no previous USAID contract
experience, USAID sent an official from Moscow to Anchorage to help
with the project's design, which has proven to be effective.
Under a cooperative agreement, USAID has relatively limited
management and monitoring responsibility. ARC provided good progress
reports to USAID. USAID/Moscow has adequately managed and monitored
the project. USAID/Washington has maintained a duplicate document
set so that it can respond to U.S. inquiries.
When it was considering further funding for ARC, USAID sent an
evaluation team from Moscow to Yakutsk, Yuzhno-Sakhalinsk,
Khabarovsk, and Anchorage in March 1994. The team's report became
the basis for USAID's continued funding of ARC. Within the amendment
that provided the fiscal year 1994 funding, USAID included a clause
stating that USAID would provide $1.5 million in fiscal year 1995 if
ARC raised $1.5 million in matching funds. However, in May 1995,
USAID agreed to provide the $1.5 million even though ARC had not
raised the matching funds.
ENVIRONMENTAL POLICY AND
TECHNOLOGY--
CH2M HILL
------------------------------------------------------- Appendix III:5
CH2M Hill is an integral part of USAID's $35-million environmental
policy and technology program. In September 1993, USAID awarded CH2M
Hill a contract to serve as the program's core contractor and provide
the technical support for its environmental activities in Russia,
Kazakstan, and Ukraine. In April 1994, an initial delivery order was
signed to provide support for activities in Novokuznetsk and the
Russian Far East. Detailed delivery orders were signed for these
activities in September 1994. CH2M Hill also serves as the
contractor or subcontractor on various components of the program.
Although the environmental policy and technology project is still
ongoing, USAID officials said its progress so far has been
disappointing. Progress has been slow because CH2M Hill did not fill
critical staff positions in Russia in a timely manner, and it relied
on staff located in the United States to manage the projects. The
expanded scope of the Far East component further contributed to the
delay. Further, USAID field staff lacked authority and information
to expedite project implementation. The projects' expected
contributions to systemic reform and long-term benefits are not
likely to be significant.
SECTOR PROBLEM
----------------------------------------------------- Appendix III:5.1
Severe environmental degradation threatens the physical health and
socioeconomic well-being of people throughout Russia and deters
economic and political restructuring efforts. Environmental problems
range from nuclear safety issues; to pervasive mismanagement of
natural resources; to some of the worst air, water, and land
pollution problems in the world. The breadth and magnitude of the
economic, health, and ecological costs are difficult to quantify,
although remediation activities alone are expected to cost
billions.\13
Environmental problems are exacerbated by many factors, including
inattention to environmental consequences, a lack of economic and
political incentives to use resources efficiently, the inability of
nongovernmental agencies to participate in environmental decision-
making, and the inability of governmental institutions to effectively
regulate state-owned monopolies and curb illegal economic activities.
--------------------
\13 This information was obtained primarily from USAID's February
1993 Project Memorandum and the U.S. Government's Environmental
Strategy for Russia, prepared by USAID.
PROJECT OBJECTIVES
----------------------------------------------------- Appendix III:5.2
Our analysis focused on CH2M Hill's performance as the core
contractor and two projects where it serves as the primary
contractor--the Multiple Pollution Source Management project in
Novokuznetsk and the Sustainable Natural Resources Management and
Biodiversity Protection project in the Russian Far East. Both
projects are to run from September 1994 to September 1997.
The objectives of the $7.4-million core contract are to coordinate
all activities under the core contract, monitor and evaluate the
activities and deliverables, and provide support functions as needed.
The objectives of the Multiple Pollution Source Management project
are to reduce pollution-related health risks and promote
environmentally sustainable economic development; improve public
health; reduce pollutant emissions from industries and cities; assist
industries in restructuring in an environmentally sound and
sustainable manner; and strengthen institutions and train individuals
to continue improvements initiated during the project. The
$13.4-million delivery order included $6.3 million for the
Novokuznetsk project along with two other projects.
The Sustainable Natural Resources Management project was expanded
from a narrowly focused $3-million, 3-year project focused on fire
prevention and control to a $16.7-million, 5-year project focused on
sustainable forest management and biodiversity protection. This
expansion responded to the Gore-Chernomyrdin Commission's
recommendations.\14 Specific project objectives are to promote
sustainable forest management in the Khabarovskiy and Primorskiy
Territories and protect endangered species and critical habitats in
the Khrebet Sikhote-Alin' mountain region. To address these
objectives, the contract specifies 25 tasks for CH2M Hill and
multiple subcontractors. USAID approved a $9.4-million delivery
order for CH2M Hill to implement and coordinate these activities.
--------------------
\14 The Gore-Chernomyrdin Commission was created in 1993 to overcome
trade barriers in the energy sector, but now includes business
development, space, environment, science and technology, health, and
defense diversification.
PROJECT APPROACH
----------------------------------------------------- Appendix III:5.3
CH2M Hill worked mainly with local and oblast' government officials
to design and manage the programs. CH2M Hill consultants spent short
periods of time in Russia to design the project proposals and then
returned to the United States to complete the project design.
Although USAID and CH2M Hill established rapport with local and
oblast' authorities in the affected cities, the Ministry of
Environmental Protection and Natural Resources was only involved in
the initial selection of project activities and their locations.
Subcontractors, U.S. nongovernmental organizations, and other
federal agencies helped implement parts of the project.
The project approach includes providing technical assistance,
demonstration projects, training seminars, and limited commodities.
Several components in both projects continued efforts initiated by
the U.S. Environmental Protection Agency, World Bank, and the City
of Pittsburgh.
CH2M Hill staff in Washington manage the project, and a regional
director and site managers in Russia handle the day-to-day activities
and coordinate with other implementers. CH2M Hill plans to hire and
train Russian employees who can eventually manage the activities
without assistance from its U.S. office.
CONTRACTOR PERFORMANCE
----------------------------------------------------- Appendix III:5.4
Project progress to date has been mixed. CH2M Hill met the
requirements of its core contract by establishing field offices,
monitoring project implementation, and providing support functions to
its field staff. Even though it has made some progress toward
addressing the Novokuznetsk project objectives, it has been slow to
implement the Far East project. CH2M Hill has missed critical
milestones for both projects.
In Novokuznetsk, CH2M Hill established an air pollution database for
the 180 heating plants in the city and developed a strategic plan to
address particulate pollution from the heating plants. It also
upgraded the city's air pollution program, trained Russian
counterparts in environmental auditing, and completed environmental
audits of two large steel mills. CH2M Hill is currently assessing
local water monitoring activities and has recommended laboratory
improvements to more accurately measure the quality of drinking
water. CH2M Hill is working with the Novokuznetsk Development Fund
and local government officials to develop a strategic plan. However,
CH2M Hill has not provided an acceptable work plan, which was due on
November 30, 1994. The current work is based on the delivery order
specifications.
In the Far East project, CH2M Hill has been even slower getting
started and, according to USAID and Russian officials, had produced
almost no quantifiable results as of February 1995. Several factors
have hindered the project's implementation, including its increased
complexity; the size of the geographic area; and the large numbers of
governmental officials, local interest groups, and subcontractors
involved. The project covers 2 large regions and will involve at
least 16 implementing organizations, including 2 U.S. federal
agencies, subcontractors, and U.S. nongovernmental organizations.
Several problems have delayed the effective implementation of both
CH2M Hill projects. One problem was that CH2M Hill experienced
problems filling critical staffing positions in Washington and Moscow
and at the field office level. Although the core contract was
awarded in September 1993, the regional director did not arrive in
Moscow until February 1994. Other positions in Moscow funded in
September 1994 delivery orders were still being filled as of January
1995. The contract to implement field support functions in
Novokuznetsk and the Far East was awarded in April 1994, but on-site
managers did not arrive until September and October 1994,
respectively. The Far East project manager position was authorized
in September 1994, but the manager did not move to Russia until
February 1995. CH2M Hill officials had difficulty finding qualified
staff who were willing to relocate to these areas because of the
acute environmental problems and remote locations. USAID and CH2M
Hill officials agreed that the on-site presence is essential for
making progress. USAID/Moscow officials said staffing delays and
delays in producing an acceptable work plan have hurt the credibility
of the program in the region.
CH2M Hill also had difficulty developing acceptable work plans that
define how and when the scope of work will be implemented. CH2M Hill
was required to submit the work plans for both the Novokuznetsk and
the Far East projects within 60 days after signing the contract on
September 30, 1994. USAID approved the work plan for the Far East
project on May 8, 1995, but the work plan for the Novokuznetsk
project was still being revised as of June 1, 1995. According to an
USAID official, the work plans originally submitted were incomplete
and lacked specific indicators or other factors necessary to evaluate
the activities.
Additionally, USAID officials said CH2M Hill had done a poor job of
providing them with the appropriate reporting documents for these
activities. USAID expressed concern over CH2M Hill's failure to
provide timely delivery of tracking materials, such as monthly
summaries of financial status by project, monthly presentations of
progress on select tasks, and weekly briefings on overall project
progress. According to USAID officials, CH2M Hill addressed their
concerns and has recently improved its reporting.
RESULTS
----------------------------------------------------- Appendix III:5.5
As of February 1995, the CH2M Hill projects had contributed little to
systemic reforms, and they will not generally be sustainable without
outside funding support. This limited contribution is due largely to
the vast environmental needs in Russia and the massive amounts of
capital investment needed to modernize or purchase equipment for
restructuring Russia's environmental sector. Also, USAID and CH2M
Hill officials said that Russian monitoring and enforcement
procedures will be extremely difficult to change and are not
addressed in these projects. Finally, the Ministry of Environmental
Protection and Natural Resources was not involved in designing the
project, thus reducing the likelihood the project could be duplicated
on a wider scale. USAID officials said the project will attempt to
address systemic reform through efforts to maintain and restock the
forestry base.
Some components of the Novokuznetsk project are likely to address
environmental sector restructuring. CH2M Hill expects to work with
Novokuznetsk's industry, citizens, and local government to develop a
strategic plan and provide recommendations for creating an
environmentally safe city by 2010. However, these recommendations
could require large capital investments. For example, CH2M Hill
recently conducted industrial audits for two steel companies. After
spending
6 weeks and using 7 U.S. advisers and 25 Russian counterparts to
conduct the audits, company officials said the audits did not provide
any new information on major pollution sources. Further, the
companies do not have the funding to make the recommended
improvements and will have difficulty obtaining it. According to one
steel mill executive, the environmental audit allowed the mill to
fulfill a condition for a World Bank loan. The Novokuznetsk project
places a considerable emphasis on the contractor delivering studies
and does not establish any indicators to measure progress in reducing
actual pollution.
Some components of the Far East project are designed to address the
region's need to maintain and restock its important forestry base.
Efforts are planned to (1) strengthen polices and develop an adequate
environmental regulatory structure, (2) create economic and political
incentives to use resources efficiently, (3) increase the
participation of nongovernmental agencies in environmental
decision-making, (4) promote U.S.-Russian partnerships, (5) promote
the export of timber products made by Russian workers, and (6)
conserve biodiversity.
USAID MANAGEMENT
----------------------------------------------------- Appendix III:5.6
USAID's decision to use a core contract and delivery orders has
caused delays and excessive paperwork reviews for both CH2M Hill and
USAID staff. Under this system, USAID must prepare delivery orders
and CH2M Hill must submit detailed work plans for each project
component within
60 days. The decision to expand the Far East program has also
delayed project design and implementation. The expansion covers a
larger geographic region and greatly increased the scope of work,
including the number of activities and subcontractors involved.
The division of responsibility between USAID/Washington and
USAID/Moscow has affected the agency's ability to manage the project.
USAID/Washington maintains overall management authority, but has
given USAID/Moscow increased monitoring and program responsibility.
However, USAID/Moscow officials said they still had minimal authority
to manage the project or make changes. USAID/Washington must approve
all program decisions, including minor ones, such as country
clearances for visitors and all purchases exceeding $500. In April
1995, USAID/Moscow submitted an initial request, which remained under
review as of June 1, 1995, for delegation of authority to the field.
USAID has had difficulty monitoring the projects. USAID staff said
they have not regularly visited the project sites because of the
difficulty of traveling to the sites and the lack of adequate staff.
The USAID/Moscow project officer keeps apprised of the project
activities primarily by talking to project staff over the telephone
or in informal meetings and by reviewing reports by the contractor or
visiting teams.
DISTRICT HEATING--
RCG/HAGGLER BAILLY
------------------------------------------------------- Appendix III:6
The district heating project is one component of USAID's Energy
Efficiency and Market Reform Project for the NIS. The project began
in January 1992 and is considered the first economic development
effort undertaken by the United States in the region. With $5.3
million in funding, the project was designed to improve district
heating systems in six countries: Armenia, Belarus, Kazakstan,
Kyrgyzstan, Russia, and Ukraine.
Although the contractor, RCG/Haggler Bailly (RCG/HB),\15 met most of
its objectives, we found no indication that the project was having a
significant impact on the sector. Most of the Russian work was
concentrated in two Russian cities, Yekaterinburg and Kostroma, and
the project was not completed in Yekaterinburg. Because USAID did
not adequately monitor the project, it was unaware of (1) problems
that prevented the completion of the project and (2) any long-term
benefits, if any, to the beneficiaries. An evaluation conducted by a
consultant did not identify obvious problems, and USAID did not
address the recommendations in this evaluation.
--------------------
\15 RCG/Haggler Bailly, one of three contractors implementing the
district heating project in the NIS, is an international management
consulting firm that specializes in energy and the environment.
SECTOR PROBLEM
----------------------------------------------------- Appendix III:6.1
Fuel and energy are an important part of Russia's economy. The
subsidies provided by the former Soviet government to Russian energy
consumers, both residential and industrial, created artificially low
prices and promoted the inefficient use of highly polluting energy.
Since the dissolution of the Soviet Union, Russia has implemented
several policies, including increasing or freeing coal, oil, and gas
prices, to reform its energy sector. Although still below world
market levels, the cost of domestic oil and oil products in Russia
doubled in 1993 and 1994. Such increases in energy prices have a
significant influence on inflation and social conditions. As energy
prices increase, consumers must find ways to use energy more
efficiently. \16
--------------------
\16 The information in this section was obtained primarily from
papers presented at the International Energy Agency's October 1994
Symposium on Russia's Energy Strategy and the Agency's report,
Russian Energy Prices, Taxes, and Costs, 1993.
PROJECT OBJECTIVES
----------------------------------------------------- Appendix III:6.2
In February 1992, RCG/HB was awarded a contract for $550,000 to
complete the project in Russia. The project was amended in August
1992, increasing the total cost to $1.3 million. The project had
five objectives: (1) foster improved management of energy use in
heating plants by identifying and implementing cost-effective "low
cost-no cost" energy efficiency improvements; (2) transfer energy
auditing and management techniques, including financial and economic
analysis techniques; (3) provide equipment support to implement
low-cost options, improve monitoring and energy management, and
identify additional energy efficiency opportunities; (4) support the
World Bank's efforts to reform Russia's energy pricing policies; and
(5) promote the emergence of an energy efficiency industry in Russia.
PROJECT APPROACH
----------------------------------------------------- Appendix III:6.3
RCG/HB and USAID worked with representatives from the Russian
Ministry of Fuel and Power, the Commission for Humanitarian and
Technical Assistance of the Russian Federation, nongovernmental
organizations concerned with energy efficiency and conservation,
municipal governments, and industrial enterprises.
The two primary Russian cities selected for the project were
Yekaterinburg and Kostroma. In these cities, extensive energy audits
were conducted of the district heating facilities, and three sites
(i.e., hospitals, apartment buildings, and heating plants) in each
city were selected as demonstration sites for U.S. energy efficiency
equipment, including flow meters, temperature sensors, and
thermostatic control valves. The value of the equipment supplied to
the demonstration sites was approximately $172,000. The project
sites were intended to demonstrate the savings in using no-cost or
low-cost technologies and also serve to promote American-made
equipment.
In addition, RCG/HB conducted energy audit training seminars and
provided energy audit equipment to technicians in Yekaterinburg,
Kostroma, Irkutsk, Moscow, Murmansk, and St. Petersburg. To
complete its work, RCG/HB contacted more than 250 U.S. equipment
manufacturers to determine their interest in conducting business in
Russia. The 12 companies that responded participated (at their own
expense) in "wrap-up" seminars in four Russian cities when the
project ended. The information obtained at these seminars was
published in a lessons learned document.
CONTRACTOR PERFORMANCE
----------------------------------------------------- Appendix III:6.4
RCG/HB completed most of the objectives stipulated in its contract.
The products delivered to complete the objectives included energy
audits in two cities, energy audit training and distribution of
energy audit equipment, a study of natural gas pricing in Russia, and
an energy efficiency industry development effort. It also produced a
video about the project that was shown on Russian television.
RCG/HB was also required to identify, purchase, and install low-cost
energy efficiency equipment manufactured by U.S. companies. RCG/HB
purchased this equipment; however, due to problems with local
conditions, some of the equipment was not installed in Yekaterinburg.
An RCG/HB official said that in June 1993, a Russian subcontractor
assured RCG/HB that it would install the equipment in Yekaterinburg
by the end of 1993. We visited three sites in Yekaterinburg in
February 1995 and found all the equipment at one site was still in
shipping containers. Russian officials said the equipment at the
other two sites only began operating in September 1994 and January
1995, respectively. According to an RCG/HB official, the company had
not paid the subcontractor and would not pay until the installation
was completed. However, USAID had already paid for the equipment,
valued at $8,000.
Officials in Yekaterinburg stated that the equipment had not been
installed in 1993 for two reasons. First, in two cases, the sites (a
hospital and an apartment building) were under construction and the
construction plans had to be altered to accommodate the equipment.
Second, at the other installation site (a district heating facility),
the equipment had not been installed, and most likely will not be
installed, because the proper Russian authorities had not certified
it. Officials in Yekaterinburg stated that it would be illegal to
install and operate the equipment before it was certified. They
explained that although the equipment can be used for demonstration
purposes at consumer locations (e.g., apartment buildings), a public
utility cannot use the equipment and the information (e.g., energy
consumption data) it produces as a basis for charging customers.
Similar equipment was installed in Kostroma, according to USAID and
RCG/HB officials, even though it had not been certified. USAID
officials told us that city officials were willing to install the
equipment because they realized the potential benefits.
RESULTS
----------------------------------------------------- Appendix III:6.5
We found no indication that the project had contributed to systemic
reform in the area of energy efficiency. Most of the work was
concentrated in two cities, and the project was not completed in
either city. In addition, USAID did not adequately monitor the
project and could not be certain of any long-term benefits.
USAID MANAGEMENT
----------------------------------------------------- Appendix III:6.6
USAID used an independent consultant, Management Systems
International, to evaluate the NIS district heating project,
including RCG/HB's work in Russia.\17 The evaluation, published in
July 1993, reported no serious problems and declared the project a
success. Specifically, the study indicated that total equipment
costs for the four cities in Russia amounted to $418,000 and would
produce an annual savings of $1.4 million. It also noted that the
equipment would reduce pollution. Furthermore, as a result of the
energy efficiency industry development effort, 12 U.S. companies had
sent representatives to the various countries to participate in
seminars held at the end of the project.
We found that the consultant's evaluation was deficient. The
evaluation did not mention the equipment installation problems in
Yekaterinburg or the need to have foreign equipment certified by the
Russian government. Instead, the evaluation stated that "by April
1993, all of the equipment was installed and operating." In addition,
USAID did not specifically direct Management Systems International to
assess the products RCG/HB was required to produce, such as the
natural gas pricing study for Russia or the lessons learned from the
energy efficiency industry development effort. The evaluation did
not discuss the quality of either of these products. USAID officials
stated that the natural gas pricing study had been completed in a
collaborative effort with the World Bank, which used it in its work
pertaining to loans made to Russia's natural gas sector. However,
the consultant's report contradicted USAID's statement by noting that
the World Bank did not make a serious attempt to involve RCG/HB in
its work in Russia.
Finally, the evaluation did not discuss the training seminars
conducted by RCG/HB in Irkutsk, Moscow, Murmansk, and St.
Petersburg, or the energy audit kit instrumentation supplied to
technicians in these cities. The continued use of these deliverables
is an important factor to consider when evaluating the success and
sustained benefits of this project.
USAID officials were not aware of the problems we identified in
Yekaterinburg nor the shortcomings of the evaluation. They stated
that in June 1993, an official from USAID/Washington visited all the
NIS sites except Yekaterinburg. They stated that equipment had been
installed at the sites visited. A local national employee from the
USAID mission in Moscow also visited Yekaterinburg in June 1993 but
did not report any problems at that site. USAID officials discovered
the problems we found when they accompanied us to Yekaterinburg.
USAID said it would take corrective action.
Also, USAID has no mechanism to monitor various outcomes of the
project, including (1) the success of U.S. industry in entering the
NIS market, (2) policy reforms written into law, and (3) the rate of
adoption of new technologies. Although USAID said that the installed
equipment would produce annual savings at project sites, it did not
record these savings during the 1993 or 1994 heating seasons.
Furthermore, USAID had not determined the savings generated by either
the energy audit kits provided to technicians in six cities or the
energy audits conducted in Yekaterinburg or Kostroma.
--------------------
\17 USAID paid $18,385 for this evaluation.
HEALTH CARE TRAINING--PARTNERS
FOR INTERNATIONAL EDUCATION AND
TRAINING
------------------------------------------------------- Appendix III:7
USAID initiated the NIS Exchange and Training program in the spring
of 1993 to train NIS leaders about free-market economies and
democratic governance. USAID hoped that training the participants in
the United States would provide them with the technical skills and
attitudes required to create similar policies, programs, and
institutions in their own nations. We reviewed the health care
training provided to Russians in late 1993.
Our analysis indicated that the health care training had little
likelihood of contributing to systemic reform and that USAID now
considers the training to be irrelevant after Russia changed its
direction for health care reform. The training's primary
objective--to facilitate Russia's transformation to a democratic
free-market system--was unrealistic for a 2-week training course.
USAID did not follow up with participants to determine the training's
impact on systemic reforms. Although USAID officials said that most
participants have been involved in follow-on projects, only 25
percent are slated for follow-on activities planned in 1995 and 1996.
SECTOR PROBLEMS
----------------------------------------------------- Appendix III:7.1
According to USAID, Russia's health care industry has a number of
problems. These problems include the virtual collapse of the
pharmaceutical and medical supply industry, poor quality of care due
to training and technical gaps, serious funding shortfalls, and a
centralized system devoid of incentives for efficiency and cost
control. Although Russian policies have produced an educated
workforce with more doctors per capita than the United States, the
workforce lacks many of the basic skills and institutions necessary
to function in a democratic, free-market context.
PROJECT OBJECTIVES
----------------------------------------------------- Appendix III:7.2
USAID contracted in June 1993 with its worldwide training support
contractor, Partners for International Education and Training (PIET),
to conduct training in the United States for 200 NIS leaders and
professionals at a cost of $2.6 million. The training objectives
were to
facilitate the region's rapid and sustainable transformation from
authoritarian, centrally controlled regimes to pluralistic,
democratic countries with free-market economies;
provide participants with new skills and knowledge to contribute to
economic and social development;
promote the value of democratic decision-making;
provide an understanding of U.S. programs; and
lead to long-term relationships with U.S. institutions.
USAID also hoped that participants would share their new skills and
perceptions with their counterparts.
PROJECT APPROACH
----------------------------------------------------- Appendix III:7.3
Under the PIET contract, USAID missions identified the training
topics and selected the participants. USAID/Moscow selected the
participants based on their positions in oblast' health care systems
and their planned inclusion in follow-on projects. According to
USAID, participants went to the United States before participating in
follow-on projects so they would be more receptive to reforms. The
training project encouraged missions to link training, if
appropriate, to ongoing or planned developmental assistance by USAID
and others.
After course topics and participants were selected, PIET was expected
to arrange training courses in the United States and provide
administrative and logistical support for international travel,
living expenses, medical insurance, tuition, books, and other needs.
PIET was also expected to (1) ensure that training programs at U.S.
training institutions were functioning properly, (2) monitor the
participants' progress, (3) provide USAID status reports, and (4)
evaluate each training program.
CONTRACTOR PERFORMANCE
----------------------------------------------------- Appendix III:7.4
PIET subcontracted with Management Sciences for Health to provide the
training in the United States; 42 Russians were trained in health
finance and 20 were trained in pharmaceutical management. USAID
subsequently contracted with the Academy for Educational Development
to make training arrangements.
PIET met its contractual requirements by providing training,
transportation, and logistics, according to USAID officials. The
participants we spoke with in Russia praised PIET's support and
assistance as well as the quality of the training they received in
the United States. Our review of sample course assessments showed
that other participants generally gave high marks to the training.
For example, in the evaluation conducted by the USAID mission, most
participants were satisfied with the course and believed it was
applicable to their work conditions.
PIET also met its monitoring and reporting requirements. PIET
maintained contact with the training institutions, called a random
sample of participants once a week, contacted the trainers on an as
needed basis, and helped participants with general adjustment
problems. PIET also provided all the required reports, including
regular status reports and course assessments. USAID/Washington
officials were satisfied with the quality of PIET's support and
monitoring during the training.
RESULTS
----------------------------------------------------- Appendix III:7.5
USAID was unable to provide any evidence that the training will help
Russia's democratic or economic transformation. Although the
training may have met some secondary goals, without follow-on
activities, fulfilling these objectives will not likely result in
systemic reform. USAID/Washington officials agreed that the training
could not meet all of the contract's objectives. They said that a
2-week training course could only "facilitate" reaching these
objectives but not actually attain them. Further, they provided the
training quickly as a political imperative to respond to the opening
in the NIS, and they recognized training alone has limited
usefulness.
USAID was unable to substantiate that any of the 62 participants
contributed to any reforms, partly because the participants lacked
the authority, expertise, or resources to influence reforms.
However, the participants who had taken PIET health-related courses
said the training helped them understand U.S. programs and they had
shared their training with others.
USAID officials in Moscow and Washington said that training alone
would not influence systemic change and that subsequent training was
better integrated into follow-on activities. They said that the main
purpose of the PIET training was to make participants receptive to
follow-on reform projects, which USAID thought would occur. However,
USAID later dropped plans for follow-on activities in Central Russia
because the oblasts were not reform-minded and the contractor
reported that only 25 percent of the Siberian participants would
participate in follow-on activities.
The Russians did not see health reform as a priority when the early
training took place, and Russia has only recently begun to consider
the direction of reforms, according to USAID/Washington officials.
Further, this early training is now irrelevant because it was based
on Russian policy directions that were later discarded as unworkable.
USAID MANAGEMENT
----------------------------------------------------- Appendix III:7.6
The mission was forced to move much more quickly than it desired
because it was under extreme congressional pressure to quickly
establish the training program, according to USAID officials. As a
result, the health care training was initiated before it could be
integrated into follow-on projects more likely to facilitate systemic
change. Further, because the Russians were unclear about what
reforms they wanted, the mission had trouble targeting the training.
The Russian officials began exploring reform options with the mission
in December 1994.
USAID/Moscow officials assessed the training after participants
returned to Moscow; however, they have had no contact with the
participants since then. They did not know which participants, if
any, would be involved in any of the follow-on activities planned in
Siberia.
COMMERCIAL REAL
ESTATE--INTERNATIONAL BUSINESS
& TECHNICAL CONSULTANTS, INC.
------------------------------------------------------- Appendix III:8
The International Business & Technical Consultants, Inc. (IBTCI)
project did not achieve its goal to increase the availability of
commercial real estate to small enterprises, although it did provide
potentially useful technical assistance in three cities. The project
did not contribute to systemic reform and was not sustainable. IBTCI
did not replicate the pilot project--a project objective--in part
because the roll-out cities were poorly chosen. IBTCI was
responsible for choosing appropriate cities, but its short-term
consultants lacked sufficient knowledge of Russia and local
conditions to determine what cities would have cooperative officials
and could benefit from the project.
SECTOR PROBLEM
----------------------------------------------------- Appendix III:8.1
Much of Russia's commercial real estate is still owned by the
government. Rather than divesting its ownership rights, the central
government has decentralized those rights to local governments, both
regional and municipal. Although this practice is quite common among
other countries in transition, Russia is different because local
governments (1) have a virtual monopoly on commercial real estate and
(2) have not moved toward commercial real estate leasing using market
mechanisms. Highly inefficient users occupy valuable commercial
space, contributing very little to local budgets, while private
sector development is blocked by the unavailability of property.\18
--------------------
\18 Harding, April L., Commercial Real Estate Market Development, The
World Bank, Discussion Paper Number 109 (May 10, 1995).
PROJECT OBJECTIVES
----------------------------------------------------- Appendix III:8.2
USAID and GKI recognized this problem and signed a task order with
IBTCI to develop a solution. The $2-million task order for the rapid
diagnosis pilot project and roll-out project was part of IBTCI's
$13.3-million omnibus contract. The initial deadline of May 1994 was
extended to December 1994, but without any increase in the cost or
level of effort required. The general purpose of the task order was
to significantly increase the availability of commercial property in
Russian cities.
The specific goals of the task order were to examine the causes of
limited access to retail space, implement a pilot project in a
selected city, and then replicate the pilot project in other oblasts.
IBTCI was to (1) deliver a report on the root causes of and solutions
to the problem of commercial property access for one city; (2) design
an implementation plan to address these issues, including procedural,
legal, administrative, financial, policy, and other measures; (3)
replicate the pilot project in at least five other oblasts; and (4)
produce and nationally distribute publicity and instructional
materials for local state property committees, local authorities, and
entrepreneurs on how to increase the availability of retail property.
PROJECT APPROACH
----------------------------------------------------- Appendix III:8.3
IBTCI used a subcontractor, Boston Consulting Group (BCG), to perform
the rapid diagnosis phase and conduct the pilot project in the City
of Perm'.\19 The goals of the pilot project were to (1) design and
test a method for increasing the amount of commercial real estate
available to small and start-up businesses and (2) identify any
constraints or impediments that might exist. The pilot was intended
to serve as a model for instituting the program in five other Russian
cities. IBTCI instituted the roll-out in Irkutsk, Tver', Novgorod,
Yekaterinburg, and Vladivostok.
Because it had previously worked in Perm', BCG used staff who already
had a relationship with municipal officials when it began the
diagnosis and pilot phases of the project. In contrast, IBTCI relied
on consultants who made short visits to the other cities to research,
plan, and implement the roll-out.
During the rapid diagnosis phase in Perm', BCG identified three
feasible ways of improving access to commercial space: convert
residential premises to commercial use, develop a secondary real
estate market, and optimize the leasing process. BCG, IBTCI, USAID,
and GKI selected the leasing optimization method because they thought
some concrete results were possible during the study period, even
though it was predicted that this alternative would have limited
support and low potential impact. Lease optimization means, among
other things, (1) moving toward market-determined rents, (2) removing
bureaucratic discretion in space allocation, and (3) creating
incentives to sublease unused space.
--------------------
\19 Perm', with a population of 1.5 million, was selected for the
pilot because of its intermediate size and presumed representative
nature of municipalities within the Russian Federation.
CONTRACTOR PERFORMANCE
----------------------------------------------------- Appendix III:8.4
BCG conducted the rapid diagnosis and pilot phase of the project from
November 1993 to March 1994 in Perm'. BCG devised a two-track
auction system for making municipality-controlled real estate
available to private businesses. The first track was an auction for
the right to lease specific commercial real estate properties (i.e.,
a one-time premium). The second track was an auction for the rental
rate at which a property would be leased. The purpose of this system
was to introduce market mechanisms into the allocation and pricing of
commercial real estate. Under this system, bidding for the right to
lease and the rent to be paid replaced government bureaucrats with
market mechanisms.
The results of the first auction, which occurred on March 1, 1994,
were not promising. In the first track auction, three properties
were available. The right to lease them was sold for each property.
In the second track auction, 15 properties, all basements, were
available. Bids were made on only 3 of the 15, and each received
only one bid. The rental rate for the three properties did not
exceed the rent that started the bidding.
The results of a second round of auctions, which occurred in May
1994, were also disappointing. In the first track auction, 10
properties were available, but the right to lease was sold for only 4
properties, although several parties bid on them. In the second
track auction, three properties were available, but only one received
a bid, and that was the starting bid. These results were not
perceived to have significantly increased the amount of commercial
real estate available in Perm'.
IBTCI started work on the roll-out in mid-February, before the Perm'
pilot was completed or its results evaluated. IBTCI soon found that
none of the five cities chosen for the roll-out had conditions that
approximated, let alone duplicated, those in Perm'. The roll-out
cities seem to have been chosen more for their geographic and
population distributions than for any existing economic, political,
and regulatory conditions that might make the Perm' model
replicable.\20 Because of these differences, IBTCI had to deviate
from the Perm' model and basically develop five new pilot projects;
nonetheless, it still experienced problems.
Irkutsk officials were not cooperative with IBTCI and declared the
information needed to assess the commercial real estate situation a
state secret. Local officials were not ready to participate in the
project.
In Tver', an auction system had been functioning between June 1992
and December 1993. The original investment tender process used in
the auction was challenged in court and hopelessly compromised.
IBTCI introduced a new tender process in Tver', but the new system's
effectiveness has not yet been demonstrated.
The Novgorod officials opposed conducting right-to-lease auctions
because they feared losing future revenue and the city had
experienced poor results from a similar auction in November 1993.
IBTCI focused on establishing a market for municipal, oblast', and
private commercial space by creating a real estate listing center,
developing a secondary market, and encouraging officials to allow
increased and legalized subletting. The listing center's
effectiveness has not yet been demonstrated.
In Yekaterinburg, an effective auction system has been in place since
1992. City officials were not interested in IBTCI's original task of
increasing the use of commercial leases. Instead, they wanted
assistance in how to use retail and commercial space efficiently and
increase the city's revenues from property leases. Although IBTCI
submitted some analyses and recommendations addressing their
concerns, city officials told us that IBTCI came to town on different
occasions, spent little time there, did not speak to the appropriate
local officials, and presented an academic report that was of little
use to them.
Vladivostok city officials were interested in privatizing commercial
real estate, but were unable to devise a method that would use
mortgages to provide substantial revenue for the city. IBTCI devised
a mortgage instrument that allowed the city to continue receiving
income by holding the mortgages and allowed small business owners to
bid for a property and provide as little as 5 percent of the final
cost as a down payment. The city auctioned one property in August
1994 for under $7,000, but local officials doubted they would use an
auction again because the city did not not have any more excess
property.
By the time IBTCI had completed its work in the five cities, none of
the cities had participated in any activities that remotely resembled
the Perm' model. As a result, the objective of replicating the pilot
project in other cities was not achieved.
There were various reasons for IBTCI's inability to replicate the
Perm' model. First, tensions between IBTCI and BCG caused some
problems. BCG performed both the rapid diagnosis and the pilot
phases, but IBTCI determined that BCG's approach was not adequate.
Russian officials monitoring the project were aware of tensions
between IBTCI and BCG early in the project, but IBTCI was obligated
to fulfill the contract and replicate the model. The tensions
between IBTCI and BCG resulted in little continuity of personnel from
the pilot to the roll-out phases.
Second, a provision of Russia's 1994 State Privatization Program Act
and its implementing regulations caused problems in the Perm' pilot.
The provision gave lessees who obtained their leases competitively
(i.e., at an auction) the right to buy the property at the end of the
leases. The implementing regulation set an extremely low selling
price for such privatized properties. IBTCI said that the act's
provision and the implementing regulation stopped the Perm' model
because city officials did not want to lose revenue from leases and
did not want to be forced to sell leased property for extremely low
prices. Even though reports identified the problem as early as
January 1994, USAID, GKI, and the Russian Privatization Center took
no effective action to address the issue.
Third, although officials at the federal level agreed earlier that
the project should be done and that the Perm' model was viable, local
officials in the roll-out cities did not agree with the Perm' model
or its usefulness in their cities.
Fourth, although the consultants used by IBTCI for this project had
some experience in Russia and some spoke Russian, Russian authorities
questioned the level of some IBTCI consultants' professional
experience. In addition, the consultants did not have enough
knowledge of the Russian localities and local politics to choose
roll-out cities well. IBTCI staff did not reside in the cities
during the roll-out. Instead, they would fly in, do a few days work,
then leave. Thus, they were unable to identify what cities would be
the best candidates for replicating the Perm' model. Even BCG had
problems carrying out a successful pilot, despite its knowledge of
and relationships in Perm'.
--------------------
\20 Novgorod and Tver' were included because they are relatively
small cities, under 250,000 in population. Irkutsk and Vladivostok
represented medium-sized cities, with populations between 250,000 and
750,000. Yekaterinburg was to represent large cities, population
over 750,000, and be a match for Perm'.
RESULTS
----------------------------------------------------- Appendix III:8.5
The project was not sustainable and did not contribute to systemic
reform. Although IBTCI's final report provided solutions to specific
problems, the project did not implement the pilot or develop a method
that could be replicated in other cities. USAID officials and the
Russians who were in charge of disseminating the report did not know
whether or where the IBTCI "solutions" had been applied in any but
the six cities. City officials we interviewed in Yekaterinburg and
Vladivostok were not using the concepts of the project.
GKI and the Russian Privatization Center had originally proposed the
project, which suppported a federal initiative. However, an existing
GKI act and its implementing regulation potentially forced local
governments to sell leased property at low prices to anyone who
bought the lease at an auction. This regulation contributed to the
poor results of the project.
USAID MANAGEMENT
----------------------------------------------------- Appendix III:8.6
USAID managed this project--from Washington with limited help from
USAID/Moscow-- jointly with GKI and the Russian Privatization Center;
it relied on consultants from the Harvard Institute for International
Development and the Center to help monitor the project. Nonetheless,
USAID did not monitor the project adequately. Even though IBTCI
filed the required reports, these reports failed to describe how much
the roll-out deviated from the Perm' model. Center officials said
they first became aware that the pilot was not being implemented in
other cities in late May 1994, long after the roll-out could be
redirected to other cities.
When problems were known in some cases, USAID did not take any
corrective action. For example, a Harvard consultant who visited
some sites raised questions about the cities selected for the
roll-out, but USAID took no corrective action. Similarly, the
problem with the State Privatization Program Act and its implementing
regulations was mentioned in reports in January 1994, but no action
was taken to resolve it. Finally, USAID officials said they were
aware of the tensions between BCG and IBTCI, but simply told IBTCI to
work the problem out themselves. USAID/Moscow officials said they
did not have enough staff to intervene when problems arose, visit the
project sites, and talk with beneficiaries about how the project was
progressing.
AGRIBUSINESS PARTNERSHIPS
PROJECT--TRI VALLEY GROWERS
------------------------------------------------------- Appendix III:9
The lack of quantifiable objectives or time frames in the Tri Valley
Growers' (TVG)\21 project design makes it difficult to measure the
project's success. TVG helped to facilitate the work of two
agribusiness partnerships in Russia; nevertheless, USAID concluded
that TVG did not perform adequately. It is too early to determine
the long-term economic viability of the partnerships; however, the
involvement of U.S. companies increases the likelihood the
partnerships will be maintained. The partnerships will probably not
have any measurable effect on Russia's agricultural sector because of
their limited size and number.
--------------------
\21 Tri Valley Growers is an agribusiness association based in
California. Its members are involved in value-added fruit and
vegetable preserving, processing, packaging, distribution, and
marketing.
SECTOR PROBLEM
----------------------------------------------------- Appendix III:9.1
Agriculture plays an important role in the Russian economy. Although
estimates vary, Russia has approximately 27,000 large state and
collective farms, which cultivate approximately 90 percent of
Russia's arable land. Approximately 270,000 private farms cultivate
5 percent of the arable land. The remaining 5 percent is made up of
private garden plots. The total farming population comprises about
26 percent of the country's population. Subsidies and income
transfers to the agricultural sector represent 25 percent of Russia's
public expenditures. Some of these subsides could be expected to be
eliminated if the agricultural sector were privatized.\22
Russian agriculture is a low-productivity sector. For example, milk
cows and potato and grain crops yield about half of western levels,
and labor productivity is probably as low as one-tenth. In addition
to low productivity, Russia has been plagued by losses of up to 50
percent in its storage and handling systems. Finally, Russia's food
processing system suffers from poor management and a lack of quality
produce, additives, ingredients, and packaging materials.
Although the Russian government has begun reforming the agricultural
sector, the actual transformation of farms and agribusiness
enterprises into market-oriented, productive entities is moving
slowly. In 1992, it reorganized state and collective farms and
agricultural input and output distribution enterprises into joint
stock companies. However, most farms have not altered their
operations to increase productivity and competitiveness.
--------------------
\22 This information was obtained primarily from the Rural
Development Institute's report, Agrarian Reform In Russia (May 1993);
a report produced by Chemonics International for USAID, Agriculture
Post-Privatization In Russia (Jan. 1994); and USAID project
documents.
PROJECT OBJECTIVES
----------------------------------------------------- Appendix III:9.2
In August 1992, USAID developed the agribusiness partnerships project
as the cornerstone of its Food System Restructuring Project. The
agribusiness partnerships project was designed to create efficient
systems for providing inputs to agriculture and for processing and
distributing agricultural products. USAID intended to catalyze NIS
private sector activity by facilitating the involvement of private
U.S. agribusinesses. Between January and May 1993, USAID signed
cooperative agreements with three agribusiness cooperatives:
Citizens Network for Foreign Affairs (CNFA), TVG, and Agricultural
Cooperative Development International. We reviewed USAID's
cooperative agreement with TVG,\23 which had obligated $5.2 million
for the region.\24
To achieve the project's objective, TVG was to facilitate
partnerships between American and NIS private agribusiness-related
enterprises. However, the agreement did not specify the number of
partnerships or the related time frames.
--------------------
\23 At the time of our review, USAID's Inspector General was
reviewing the cooperative agreement with CNFA.
\24 The combined budget for the three cooperative agreements under
the agribusiness partnerships project was $59.3 million. As of
February 1995, $52.5 million had been obligated.
PROJECT APPROACH
----------------------------------------------------- Appendix III:9.3
TVG established an office in Moscow staffed by one American director
and three Russian nationals. This office was supported by several
TVG headquarters staff in California. The American director did not
have an agribusiness background but was responsible for managing the
office, identifying potential Russian and American agribusiness
partners, reviewing partnership proposals, and submitting the
proposals to USAID for final subgrant approval.
According to TVG officials, TVG identified potential Russian
agribusiness partners through a network of contacts at the Ministry
of Agriculture, Association of Individual Farms and Agricultural
Cooperatives of Russia, World Bank, European Bank for Reconstruction
and Development, Peace Corps, investment funds, and regional and
local administrations. To identify American partners, TVG canvassed
its members in the United States, advertised in agricultural
publications, contacted agribusinesses via telephone, and looked for
firms already operating in Russia.
Once identified, TVG worked with the American and Russian partners to
develop proposals for USAID's approval. After receiving USAID
approval, TVG awarded subgrants to U.S. agribusinesses working in
Russia primarily to provide technical assistance and agricultural
training to help create efficient food systems. The American
agribusiness partner was required to provide at least 2.5 times the
level of funding provided by USAID, to ensure its commitment to the
partnership and the long-term economic viability and sustainability
of the joint venture.
The items purchased with the USAID subgrants are referred to as
"additionality" components, or those components that might otherwise
not be included in the joint venture without USAID funding.
Additionality components include additional training and facility
expansion.
CONTRACTOR PERFORMANCE
----------------------------------------------------- Appendix III:9.4
TVG established six partnerships in five NIS countries, with two in
Russia. As of March 1995, one additional partnership in Russia was
awaiting USAID approval. The first partnership established in Russia
was with Petoseed Company, Inc., and is located in Krasnodar.\25
Petoseed produces vegetable seeds that will be sold in the NIS and
internationally. During the 1994 growing season, Petoseed produced
11,000 pounds of seed in Russia. The second TVG Russian partnership
involves CTC Foods Company, which is building a potato processing
facility in Pushchino. If finished, the facility will produce dried
potato flakes that will be sold primarily to hospitals and schools.
The two American agribusiness partners exceeded the required level of
partnership funding in both partnerships. Contributions by USAID,
U.S. agribusiness partners, and Russian beneficiaries to the TVG
partnerships in Russia are shown in table III.1.
Table III.1
Agribusiness Partnerships Established by
Tri Valley Growers in Russia
Su
bg
ra Russian
nt Total U.S. firm firm Grant U.S. firm
ee partnership USAID grant share share\a expended expended
-- ----------- ----------- ----------- ----------- ----------- -----------
Pe $1,085,975 $309,250 $776,725 0 $44,054 $181,338
t
o
s
e
e
d
CT 4,300,000 800,000 2,200,000 $1,300,000 428,249 1,624,789
C
F
o
o
d
s
================================================================================
To $5,385,975 $1,109,250 $2,976,725 $1,300,000 $472,303 $1,806,127
t
a
l
--------------------------------------------------------------------------------
\a The Russian partners' contributions are generally in-kind
contributions.
Source: TVG statistics as of May 25, 1995.
TVG had difficulty identifying partnerships. TVG staff had
difficulty beginning work in Russia because of poor telecommunication
and office facilities, the chaotic Russian business environment, the
limited number of American firms willing to invest in Russia, limited
funding, and a small staff. According to a TVG official, Petoseed
and CTC Foods contacted TVG to participate in the project. However,
both companies were already working in Russia before USAID had
established the agribusiness project and located Russian partners on
their own. He said they would have invested in Russia without USAID
involvement. An official at the Association of Individual Farms and
Agricultural Cooperatives of Russia told us that the Association
tried to work with TVG to identify Russian partners but received only
"empty promises."
Although USAID never specified the number of partnerships that it
wanted to establish within a given time frame, it concluded that TVG
had not performed adequately. Between May and December 1993, USAID
expressed concern about the number of partnership proposals TVG was
submitting and the quality of the proposals. A February 1995 USAID
review of the agribusiness project stated that TVG required more
support by USAID staff and was less committed to the project than
CNFA. TVG closed its office in Moscow in August 1994 and has stopped
the Russia part of its program because USAID terminated the
agribusiness partnerships project in Russia. Nevertheless, USAID's
review noted that the partnerships to which TVG had made subgrants
were doing well. However, a TVG official told us in May 1995 that
because of financial problems, CTC Foods may not be able to continue
its work in Russia. Consequently, the processing facility in
Pushchino may never be constructed. According to a USAID official,
TVG's Moldova office now monitors the Russian subgrants.
--------------------
\25 Petoseed also established seed production facilities in Ukraine
and Moldova through TVG.
RESULTS
----------------------------------------------------- Appendix III:9.5
The agribusiness partnerships developed by TVG in Russia have not
been operating long enough to adequately judge their impact.
However, due to their limited scope, it is unlikely that the
partnerships will have a significant effect on reforming Russia's
agricultural sector.
USAID MANAGEMENT
----------------------------------------------------- Appendix III:9.6
USAID/Washington designed the agribusiness partnerships project in
1992, before the USAID/Moscow mission was opened. USAID/Washington
and USAID/Moscow split the oversight responsibilities: Washington
was primarily responsible for TVG's compliance with the cooperative
agreement and Moscow was responsible for subgrant proposal
evaluation. Final grant approval was a joint Moscow/Washington
effort.
Although USAID/Moscow raised continued concerns about the
agribusiness partnerships project's ability to influence systemic
reform, the project proceeded. USAID/Moscow officials called for a
review of the project as early as November 1993, and they developed a
statement of work for an evaluation team. However, USAID/Washington
told USAID/Moscow to "forget the assessment and get on with the job."
Consequently, no assessment was conducted. According to USAID
officials, an evaluation is planned for June 1995.
According to USAID officials, the agency wanted to implement the
project quickly and demonstrate results. TVG's Moscow director
stated she was pressured to submit proposals quickly because USAID
was being pressured by Congress. However, both CNFA and TVG
officials complained that USAID's subgrant approval process was
arduous and lengthy. They said it took several months for USAID to
accept or reject a proposal and added that USAID/Washington caused
most of the delay. USAID/Washington officials said the delays were
caused by the time required to research legal issues, conduct
environmental audits, and work through the Washington bureaucracy.
The cooperative agreement with TVG called for quarterly program
performance reports and annual progress reports. An independent
accounting firm was to audit TVG's financial statements. Although
USAID officials said that TVG met all of its reporting requirements,
our review indicated that TVG had not submitted annual reports.
According to USAID officials responsible for the project in Russia,
USAID staff visited only half the project sites established by TVG,
CNFA, and Agricultural Cooperative Development International between
May 1993 and November 1994.
USAID was required to annually assess the performance and program
direction of the cooperative agreement and contract for an
independent external evaluation. As of March 1995, it had done
neither. However, CNFA completed an evaluation of the agribusiness
partnerships project in August 1994 at USAID/Moscow's request. It
reported that the project had not started agribusiness partnerships
quickly, had not made a significant contribution to sectoral reform,
and had little to show for the "additionality" purchased with USAID
funds. CNFA's internal evaluation did not address TVG's performance.
USAID completed an internal review in February 1995, but the review
did not cover the additionality components. The review stated that
it was unrealistic to expect the overall project to have a
significant, measurable impact on the food system in the NIS.
USAID has discontinued the agribusiness partnerships project in
Russia and, as of September 1994, stopped accepting proposals for
Russian agribusiness partnerships. In addition, USAID has decided
not to obligate any additional funds for the project. Agency
officials stated that the project itself cannot adequately address
the obstacles of reforming the agricultural sector and indicated that
other projects, such as the Russian-American Enterprise Fund, were
better vehicles for financing joint ventures.
OFFICER RESETTLEMENT HOUSING
PILOT
------------------------------------------------------ Appendix III:10
The Russian officer resettlement pilot project has been successful in
providing the required housing units, although not within the
original time frames. The project's secondary objectives--to provide
job skills training for demobilized officers and to help facilitate
housing sector reform--were only partially met. By implementing a
pilot program, USAID was able to test the viability of a housing
construction project and apply lessons learned to the $160-million
follow-on project.
Planning and Development Collaborative International (PADCO), the
contractor tasked to provide construction management services, was
successful in part because it (1) had experience in working on
housing sector reform in Russia, (2) established a physical presence
in Moscow and in the field, (3) obtained at least some buy-in and
involvement from the local Russian government, and (4) employed
Russian staff to oversee construction activities.
SECTOR PROBLEM
---------------------------------------------------- Appendix III:10.1
The Russian Ministry of Defense has traditionally provided qualifying
retired and demobilized military officers with a dwelling unit or
plot of land and some job skills training. After the Soviet Union
dissolved, between 200,000 and 350,000 officers needed housing;
approximately 42,000 were located in the Baltic Republics of Estonia,
Latvia, and Lithuania. However, since the dissolution, the Russian
government has lacked the housing stock to resettle all the
demobilized military officers. Further, Russia's severe economic
problems, housing shortages, and lack of "social guarantees" for
these retired officers has delayed troop withdrawals.
PROJECT OBJECTIVE
---------------------------------------------------- Appendix III:10.2
President Clinton announced the Russian officer resettlement program
at the Vancouver Summit in April 1993. Later, in July 1993, he
stated at the G-7 Heads of State meeting in Tokyo that the program
should encourage rapid withdrawal of demobilized Russian officers
from the Baltic Republics. The Russian Officer Resettlement
Initiative is being conducted in two phases--a $6-million pilot and a
$160-million follow-on project. The pilot's primary objective was to
construct 450 housing units by July 1994 for the resettlement of
demobilized Russian military officers. The follow-on project was to
provide up to 5,000 units (2,500 constructed and 2,500 voucher
certificates) by November 30, 1996, for officers demobilized in the
Baltics or other countries outside Russia. The pilot project's
secondary objectives were to provide job skills training, experiment
with new housing technologies, assist private firms in housing
development and construction, and expand the scope of housing
choices.
PROJECT APPROACH
---------------------------------------------------- Appendix III:10.3
To implement the pilot project, USAID contracted with PADCO for
construction management services. It also awarded fixed-price
contracts to five Russian builders and one private voluntary agency
to construct housing units in five cities. Finally, it provided a
grant to the International Catholic Migration Commission for
training.
According to project officials, PADCO assisted the project design
team that included officials from USAID/Washington and USAID/Moscow.
This team visited potential project sites, evaluated projects, and
negotiated construction contracts. PADCO was responsible for
managing the construction activities and monitoring contractor
performance. U.S. officials said the design team created the pilot
with only minimal input from the Ministry of Defense or the Ministry
of Construction. USAID officials added that the design team
conducted its own field assessment to select participating cities and
worked almost exclusively with the local authorities in these cities.
The local authorities were to provide infrastructure services such as
heating, water, and road access for the housing units.
USAID relied on the Russian Ministry of Defense to select the
officers to receive the housing. The initial design for the pilot
program did not stipulate where the officers should come from, but as
a result of the Tokyo G-7 meetings in July 1993, USAID established
criteria that gave priority to demobilized officers living in the
Baltics.
The criteria also included housing for officers from other areas
outside Russia because two cities were reluctant to provide
infrastructure for officers exclusively from the Baltics. Under this
criteria, officers demobilized in other areas would be included.
USAID's compromise with these cities allowed some demobilized
officers from their own jurisdictions to receive housing. In Nizhniy
Novgorod, half the officers could come from its jurisdiction, while
in Volgograd, 40 percent of the officers could come from its
jurisdiction. USAID and PADCO officials said beneficiary composition
would also be an issue in the follow-on project.
PADCO's project staff established a long-term presence in Moscow and
traveled regularly to the various building sites. It also hired and
trained Russian construction specialists to supervise the
construction in each city. USAID officials said PADCO's experience
in Russian housing issues helped facilitate this project.
CONTRACTOR PERFORMANCE
---------------------------------------------------- Appendix III:10.4
PADCO and Russian contractors generally met the program's primary
objective of providing housing units, although not within established
time frames. As of July 1994, only 94 (21 percent) of the 452 units
were completed, although as of February 15, 1995, the project had
provided 422 units (93 percent) through a combination of construction
and voucher certificate activities. (See table III.2 and figs.
III.1 and III.2.)
Table III.2
Status of Russian Officer Resettlement
Pilot Project as of February 15, 1995
Pending
Originally Added/ Completed/ completion Transferre
Location planned Terminated allotted issued /approval d to owner
-------- ---------- ---------- ---------- ---------- ---------- ----------
Construc
tion
Lipetsk 40 40
Nizhniy 128 128 45
Novgorod
1
Nizhniy 50 50 0
Novgorod
2
Novosibi 180 180 0
rsk
Tula 1 14 14 12
Tula 2 16 16 9
Tula 3 30 30 0
Volgogra 32 32 29
d 1
Volgogra 80 80 75
d 2
Volgogra 72 72 0
d 3
================================================================================
Subtotal 460 270 182 342 30 170
Voucher
certifi
cates
Novgorod 40 40 2 38
Pskov 40 40 2 38
================================================================================
Subtotal 80 80 4 76
================================================================================
Total 460 270 262 422 34 246
--------------------------------------------------------------------------------
Figure III.1: Housing Units
Built in Volgograd for Russian
Officers
(See figure in printed
edition.)
Figure III.2: Duplexes Built
in Tula for Russian Officers
(See figure in printed
edition.)
Of the 10 project sites in 5 cities, USAID terminated 3: one because
newly elected local officials refused to meet the previous
administration's commitments to provide infrastructure support to the
housing units and 2 because contractors defaulted on their building
commitments.
In Novosibirsk, USAID and PADCO officials said federal and oblast'
officials were not involved in the initial agreements. Therefore,
they had no authority to require the local administration to abide by
the contract, and they would not allocate additional funds for the
infrastructure.
In Lipetsk, the contractor was a private voluntary agency that
subcontracted with a local Russian construction firm to execute the
work. When the subcontractor defaulted, the agency was unable to
find a replacement to complete the work. At the Nizhniy Novgorod
50-unit project site, project officials said the Russian contractor
ran into financial problems and stopped work, claiming that the
$8,500 per unit allowed in the contract was not enough to cover
costs. Although the city offered several incentives, including a
$300,000 letter of credit and land for additional construction, USAID
and PADCO officials said the contractor was unwilling to spend his
own funds and the contract was terminated by USAID.
USAID officials said the contractor at the 128-unit Nizhniy Novgorod
site was concerned that $8,500 per unit was not enough to cover the
cost of construction. The contractor had completed almost 70 percent
of construction when increased construction costs, caused by rapidly
rising inflation (9 percent a month) and the devaluation of the
ruble, forced him to stop work. According to USAID officials,
because the contractor had done a good job, used his own funds from
other projects, and was well-connected with city and oblast'
officials, he negotiated an agreement so that the oblast' and USAID
would cover the increased costs of the 128 units. To ensure the
project's completion, USAID and the oblast' administration each
provided an additional $700,000, thus increasing the per unit price
to $19,500.
Because contracts were terminated months after they were awarded,
USAID developed a method to meet the housing requirements quickly.
It awarded a contract to the Urban Institute to implement a voucher
certificate program, which allowed officers to purchase existing
local housing in a participating area or housing under construction.
Because of increased construction costs, the inclusion of land, and
infrastructure costs, the vouchers were increased from the $8,500 per
unit in the construction program to a maximum of $25,000 per unit.
According to USAID officials, using voucher certificates allowed the
pilot program to provide housing units much quicker than through
direct construction. As of January 30, 1995, 80 vouchers had been
disseminated to the officers, and 76 (95 percent) of the them had
been used to purchase units, which were turned over to the officers.
The International Catholic Migration Commission's efforts to address
one of the project's secondary goals of job skills training has shown
limited results, according to USAID housing officials. As of
December 1994, it had arranged training for 46 beneficiaries who
attended business courses in Pskov, Novgorod, and Volgograd. The
USAID official said construction delays and the subsequent delays in
officers relocating to their respective cities affected start-up
activities. Further, the official said the Commission did not
adequately identify the officers' training needs and failed to
recognize that many of them were not interested in training.
Project officials said only minimal progress was made in addressing
other secondary goals, such as demonstrating new housing
technologies, expanding customer choices, and implementing more
stringent quality control standards. For example, in Tula,
contractors constructed 14-, 16-, and 30-unit duplexes, which took as
long or longer to build than traditional high rise structures. (See
fig. III.2.) PADCO field representatives worked with local builders
to ensure that quality control measures were introduced and achieved.
RESULTS
---------------------------------------------------- Appendix III:10.5
The officer resettlement pilot project accomplished its objective of
providing housing to demobilized officers. The project was not
designed to address systemic reform or to be sustainable, and it did
not do so. PADCO officials said the attempts to sustain the effects
of the project's secondary objectives were short-lived, although the
lessons imparted by PADCO--new housing technologies, housing choices,
and quality control measures-- may have some positive effect on the
building industry and contractors. USAID and Urban Institute
officials said the lessons learned from implementing the voucher
certificate activity by the banks, realtors, and local governments
may be used to facilitate the local governments' transition to a
private housing market.
As a result of the pilot project, USAID incorporated the lessons
learned as it designed the $160-million follow-on initiative to
provide 5,000 units to officers from the Baltics. The primary
changes included (1) obtaining total support, involvement, and buy-in
from all three levels of Russian government; (2) using the voucher
certificate program to expedite the relocation of 2,500 officers; (3)
stipulating that a maximum of 10 percent of the demobilized officers
could come from local jurisdictions; (4) using a U.S. construction
management firm as the prime construction contractor and subcontract
to the individual builders; (5) using only experienced,
well-connected Russian builders; (6) selecting partially completed
buildings and sites with existing infrastructure; (7) using a
traditional Russian housing design; and (8) providing a more
realistic per unit cost ($25,000 versus $8,500). According to USAID
officials, these changes are expected to allow the follow-on project
to proceed more quickly and efficiently than the pilot.
USAID MANAGEMENT
---------------------------------------------------- Appendix III:10.6
USAID/Moscow had management responsibility for the project and
generally did a good job of managing, monitoring, and overseeing it
and coordinating with USAID/Washington. PADCO officials said
USAID/Moscow actively assisted the contractors in reaching acceptable
compromises with government officials and contractors. The
USAID/Moscow project team reviewed project status reports, visited
project sites, and held regular meetings with contractors. Finally,
AID terminated work when problems could not be overcome.
(See figure in printed edition.)Appendix IV
COMMENTS FROM THE U.S. AGENCY FOR
INTERNATIONAL DEVELOPMENT
========================================================= Appendix III
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
The following are GAO's comments on USAID's letter dated June 1,
1995.
GAO'S COMMENTS
1. We have incorporated these comments into the report where
appropriate.
2. Although we noted project shortcomings, we also recognized the
contribution Deloitte & Touche made toward the privatization process
and considered the project a success. Moreover, we recognized
USAID's positive contribution to the overall privatization effort.
3. We conducted a detailed review of Tri Valley Growers'
performance, one of the three contractors responsible for
implementing the agribusiness partnerships project, to determine
whether this expenditure of funds had any sustainable impact. We
concluded that it did not. Although we did not draw any conclusions
about the agribusiness partnerships project as a whole, our analysis
casts doubt on whether the project can have a systemic impact if the
individual partnerships are not having an impact. (See comments 29
and 30 for additional discussion.)
4. It is too early to know whether USAID's prediction concerning the
outcome of ongoing activities in the energy sector will result in
significant sector reform. Many of these projects are just starting
and must overcome many obstacles. For example, in our September 1994
report on nuclear safety, we reported that there are no guarantees
that the international assistance effort will result in safer
reactors or expedite the closure of the riskiest reactors.\1 In fact,
in the absence of a commitment to close down the reactors, the
assistance may encourage their continued operation. We noted that
donor countries face formidable challenges in promoting the closure
of the Soviet-designed reactors because the countries operating them
depend on the nuclear power to meet their needs for domestic energy
and export income.
5. We agree that the new evaluation system is promising in that it
should provide an improved basis to evaluate USAID's programs in the
NIS. However, since the first report is not due until November 1995,
it is too early to know whether the system will fulfill its promise.
The value of the system will depend on the indicators selected, the
reliability of the data, and the subjective judgments of USAID
officials preparing the reports. For the system to have credibility,
USAID will have to be able to identify shortcomings as well as
successes.
6. We have modified the report to reflect this information.
7. We were able to reconcile obligations and expenditures in the
USAID financial report with other USAID documentation. Accordingly,
we have deleted the examples from the report.
8. Our draft report included information on the work of the Consuls
General. We have modified our report to update the information on
increased site visits.
9. Although market forces played a role in the limited use of some
of the centers, the lack of local support as well as other factors
also caused the low activity levels at some centers. More
importantly, it is questionable whether USAID should spend funds on
activities without a market unless it has a strategy to create demand
for the product it is financing.
10. We visited only one site (in Siberia during February) because of
the limited amount of time we had in country. Vorkuta can be reached
by plane in the winter. USAID can visit the sites at other times
during the year. We believe that three site visits--two occurred
after we began our audit--in 31 months is inadequate for monitoring
purposes. Day-to-day contacts with PIER staff are important;
however, they do not substitute for site monitoring or provide USAID
with an objective basis for evaluating the project's success.
11. The accomplishments noted in our report are those that have had
the greatest impact. PIER did not provide us with any statistics
that indicated increased mine safety or productivity. Moveover, a
September 1994 study produced by the U.S. Department of the Interior
indicates that mine safety in Russia is actually getting worse. The
beneficiaries we spoke with indicated that they were implementing new
mining methods introduced by PIER; however, they did not mention any
measurable increases in productivity. In addition, although
productivity and efficiency are important, overall production for the
coal sector is still too high. Finally, PIER's Moscow director
stated that this project has had the greatest impact in the areas of
restructuring, private sector involvement, and social safety net
development.
12. Our report does not state that an interim evaluation is imminent
and may lead to activities being redirected. Our report states that
"USAID management admitted that no annual assessments or midterm
evaluations were conducted," even when required by the cooperative
agreement.
13. USAID's new procedures did not affect the program during the
time frames we reviewed. Also, the work plan example should be taken
in the context that several iterations of the plan have been
submitted and revised since November 1994.
14. Our draft report did not recommend that more authority be
delegated.
15. Our report was modified to show that the Ministry of
Environmental Protection and Natural Resources was involved in the
initial selection of project activities. However, the Ministry did
not participate in designing the projects as USAID suggests. USAID
acknowledges the almost immediate shift of its relationship from the
central to the local government once the projects were selected. We
remain concerned over the lack of federal involvement, especially
regarding the provision of resources and the limited potential for
replicability. We believe that without outside funding or support
from the federal level, sustainability and replication will be
difficult.
16. As indicated in the report, we found that as of February 1995,
the Far East project had contributed little to systemic reform and is
unsustainable without outside funding. The report discusses the
project's attempt to address systemic reform through efforts to
maintain and restock the forestry base.
17. As indicated in the report, the deliverables identified in the
delivery orders generally cited reports and studies as the results.
We are unable to verify USAID's statements regarding the project's
results.
18. Although we recognize that this project was only one of many in
the energy sector, we found that the project is unlikely to
contribute to systemic reform because of its design and the lack of
monitoring and follow-up by USAID.
19. USAID suggested that we select this project in part because
USAID represented it as a success, based on an independent
evaluation. We visited only Yekaterinburg for two primary reasons:
USAID suggested that it was a good site to visit and it was one of
only two cities where equipment was installed and extensive energy
audits and training were provided. However, as we reported, the
equipment had not been installed. USAID's assertion that our
conclusion is based almost entirely on the site visit is wrong. Our
conclusion is also based on discussions with representatives from
RCG/Haggler Bailly, Joseph Technology, Honeywell, and USAID officials
responsible for the project and our review of numerous documents on
the entire project.
20. We did not make statements or draw conclusions about other
projects in the program or about the overall program. We noted that
there was no indication that the project we reviewed had contributed
to systemic reform. The energy efficiency audits and demonstration
sites can only have an impact on systemic reform if USAID ensures
that (1) equipment is installed; (2) equipment and training is used;
(3) the recommendations in the energy audits are implemented; (4) the
results of the project are monitored, recorded, and publicized; (5)
appropriate personnel have access to the demonstration sites; and (6)
problems such as lack of equipment certification are corrected. At
the time we conducted our fieldwork, USAID was not ensuring any of
these elements because the project did not include any mechanisms for
long-term monitoring or replicating the project. USAID, in its
comments, acknowledged that the project alone is unlikely to have an
impact on systemic reform.
21. We agree that the dollar value of the uninstalled equipment
constitutes a relatively small percentage of all the equipment
purchased. The fact that USAID was unaware of the problems in
Yekaterinburg and had not monitored whether any cost savings had been
achieved and did not know whether any systemic improvements had
resulted from the equipment, energy audits, and training provided is
the issue.
22. The consultant's evaluation indicated that all of the equipment
was installed in April 1993. The documents we reviewed indicate that
the equipment was provided in April and May 1993. If the equipment
did not arrive until June 1993 as USAID suggests, USAID should have
known the evaluation had problems when it indicated that all the
equipment was installed 1 or 2 months before the equipment ever
arrived.
23. We contacted individuals who were identified as participants by
USAID/Moscow.
24. Our conclusion that the training was irrelevant was not based on
our discussions with the participants of the PIET training courses.
It was based on statements by various USAID officials, including the
USAID/Moscow Mission Director and the Chief of USAID/Washington's
Europe and the Newly Independent States/Health and Population Office.
For example, USAID officials told us that a 2-week training course
without follow-on activities could not be expected to result in any
systemic reform. In addition, as we also noted in the report,
because USAID had not conducted any long-term monitoring of the
participants, it had no evidence that any of the participants
instituted systemic changes based on the training. The opinions and
views of the participants we interviewed were used to provide insight
as to why no systemic changes had occurred.
25. Contrary to USAID's comments, we referred to the course
evaluations in our draft report. We stated that ". . . most
participants were satisfied with the course and believed it was
applicable to their work conditions." However, our assessment was not
concerned with whether the participants were satisfied with the
course but with whether the goals of the PIET contract and the
Freedom Support Act were fulfilled.
26. We question USAID's assertion about the actual positive impact
of the training and follow-on assistance. First, at the time of our
review, follow-on assistance was not planned to begin for another 6
months and no assessment had been completed to confirm or deny
USAID's assertion. Second, despite repeated USAID statements that
the Siberian participants were involved in follow-on projects, USAID
project officials did not know how many were actually involved. When
the contractor compiled this data for us, the USAID project official
in charge of the program was surprised that 75 percent of the
participants in Siberia were not involved with the planned follow-on
activities. Finally, USAID health officials we spoke with were
unanimous in their assessment that the follow-on project's progress
to date has been a disappointment.
27. Contrary to USAID's assertion, we met with the Russian
Privatization Center (RPC) official responsible for the project.
This official questioned the competence of some of the consultants.
We also believe that the characterization of this project as a
qualified success is an overstatement. Project task orders were
never modified, thus the focus of the project remained to increase
the availability of commercial real estate. However, after project
completion, large amounts of commercial real estate continued to be
leased in the selected cities under conditions that encouraged
inefficient use, and the municipalities failed to maximize revenues.
Furthermore, we noted that the Center, in one of the memorandums
quoted by USAID, questioned the reliability of IBTCI representations
and indicated that USAID and the Center believed that the Perm' model
was being rolled out to the other cities. The memorandum stated the
following:
At our most recent meeting, on June 2, Jay Kalotra [the project
director] presented a preliminary draft of the wrap-up memo for
the project. At that time, I reminded Jay several times that
deviations from the Perm' model would have to be rigorously
defended to both USAID and the senior management of the RPC.
Jay's response was that IBTCI deviated from the Perm' model in
large part because the model was ill-suited to the chosen
roll-out cities. To a considerable extent, this may be true.
However, this obviously does not exonerate IBTCI, since their
task was to find cities where a pure roll-out could be
performed. [underscoring supplied]
In its most recent memos, IBTCI suggests that they told us at
the outset they would adopt a broader approach than BCG took in
Perm'. While it is true that IBTCI states some very ambitious
goals, it is disingenuous to suggest that we agreed to replace
the Perm' model with something else. . . . [underscoring
supplied]
. . . However, as noted above, even three weeks ago IBTCI was
maintaining the pretense of Perm'-style results. And it was
only when we actually visited the project sites that we could
see the extent to which deviations had occurred. [underscoring
supplied]
IBTCI was clearly responsible for selecting the cities. GKI relied
on the contractor, but GKI's approval did not relieve the contractor
of this responsibility. Because the contractor did not attempt to
use the Perm' model during the roll-out phase, it is impossible to
determine if the Perm' model could be applicable to the other cities.
28. Our report draft specifically stated that the IBTCI team spoke
Russian and had meaningful business experience. The issue was
IBTCI's awareness of local conditions in order to replicate the Perm'
model. Although the project manager on the roll-out was also the
comanager for the pilot, none of the city team leaders had worked on
the Perm' pilot.
29. Our analysis casts doubt on whether the entire project can have
a systemic impact if the individual partnerships are not having an
impact.
30. We did not assess the activities of CNFA because the USAID
Inspector General was evaluating CNFA's work. Discussions with the
USAID Inspector General's office indicate that some of CNFA
partnerships are experiencing problems and consequently are not
contributing to systemic reform. In addition, CNFA's August 1994
evaluation of the agribusiness partnerships project indicates that
the overall project had not achieved the desired results. Also,
USAID's February 1995 review of the agribusiness partnerships states
the following:
It is not realistic to expect the agribusiness partnerships
program to have a significant, measurable impact on overall food
systems in the NIS. The limited number of partnerships being
supported suggests such national level impacts are unlikely
during the life of the activity, if ever.
USAID/Moscow staff stated that the agribusiness partnerships project
could not by itself influence systemic reform.
31. Although TVG has established three partnerships in Russia, only
one (Petoseed) is functioning. According to TVG staff, CTC Foods has
run into financial problems; consequently, its potato processing
facility may never be constructed. Finally, the third partnership
(Big Sky Foods Trading, Inc.) has only recently been approved, and it
is too early to determine whether this project will be successful.
32. USAID staff in Moscow and Washington characterized the project
as discontinued because no more partnerships can be introduced and no
more funds will be obligated.
33. USAID/Moscow staff said TVG had not performed adequately and had
not identified appropriate partnerships. The documents we reviewed
also indicated that TVG was not performing well.
34. Our report includes examples of the causes for delays in the
approval process, including the need to deal with legal issues.
35. We modified the report to reflect this foreign policy goal.
However, the primary objective for the pilot project announced in
April 1993 did not focus on relocating officers from the Baltics.
The announcement made in July 1993 focused the program on the removal
of officers from the Baltics.
36. USAID is incorrect in stating that the oblast' was involved in
signing the memorandum for Novosibirsk. It was only signed by USAID
and the municipality of Novosibirsk.
--------------------
\1 Nuclear Safety: International Assistance Efforts to Make
Soviet-Designed Reactors Safer (GAO/RCED-94-234, Sept. 29, 1994).
MAJOR CONTRIBUTORS TO THIS REPORT
=========================================================== Appendix V
NATIONAL SECURITY AND
INTERNATIONAL AFFAIRS DIVISION,
WASHINGTON, D.C.
Louis H. Zanardi
Eugene D. Beye
Edward J. George, Jr.
EUROPEAN OFFICE
Peter J. Bylsma
David M. Bruno
Jodi McDade Prosser