    Foreign Assistance: Assessment of Selected USAID Projects in Russia
    (Letter Report, 08/03/95, GAO/NSIAD-95-156).

    Pursuant to a congressional request, GAO reviewed the Agency for
    International Development's (AID) assistance projects in Russia,
    focusing on whether: (1) individual AID projects met their objectives
    and contributed to systemic reforms; (2) the projects had common
    characteristics that contributed to their successful or unsuccessful
    outcomes; and (3) AID adequately managed the Russian projects.

    GAO found that: (1) some of the projects reviewed fully met or exceeded
    their objectives, while other projects met few or none of their
    objectives; (2) 3 AID projects contributed to fundamental structural
    changes in Russia because they had sustainability built into their
    design and they focused on national or regional issues; (3) the
    successful projects had broad and strong support from all levels of the
    Russian government, U.S. contractors with long-term physical presence in
    Russia, a broad scope to maximize benefits, and specific sustainability
    objectives, and complemented or supported Russian initiatives; (4)
    Russian officials' commitment to reform in certain sectors was critical
    to project success; (5) the unsuccessful projects were poorly designed
    and implemented and often had little or no impact on problems; (6) AID
    made certain exceptions to its normal procedures and processes in its
    desire to respond quickly to assist Russia; and (7) AID failed to
    adequately manage some projects because of problems in delegating
    management and monitoring responsibility to the Moscow AID office,
    inadequate staff, and inadequate management information systems.

    --------------------------- Indexing Terms -----------------------------

     REPORTNUM:  NSIAD-95-156
         TITLE:  Foreign Assistance: Assessment of Selected USAID Projects
                 in Russia
          DATE:  08/03/95
       SUBJECT:  Federal aid to foreign countries
                 Foreign economic assistance
                 Project monitoring
                 Foreign governments
                 Human resources utilization
                 Housing programs
                 Health care programs
                 Coal mining
                 Agricultural programs
                 Management information systems
    IDENTIFIER:  Russia

    **************************************************************************
    * This file contains an ASCII representation of the text of a GAO        *
    * report.  Delineations within the text indicating chapter titles,       *
    * headings, and bullets are preserved.  Major divisions and subdivisions *
    * of the text, such as Chapters, Sections, and Appendixes, are           *
    * identified by double and single lines.  The numbers on the right end   *
    * of these lines indicate the position of each of the subsections in the *
    * document outline.  These numbers do NOT correspond with the page       *
    * numbers of the printed product.                                        *
    *                                                                        *
    * No attempt has been made to display graphic images, although figure    *
    * captions are reproduced. Tables are included, but may not resemble     *
    * those in the printed version.                                          *
    *                                                                        *
    * A printed copy of this report may be obtained from the GAO Document    *
    * Distribution Facility by calling (202) 512-6000, by faxing your        *
    * request to (301) 258-4066, or by writing to P.O. Box 6015,             *
    * Gaithersburg, MD 20884-6015. We are unable to accept electronic orders *
    * for printed documents at this time.                                    *
    **************************************************************************

    Cover
    ================================================================ COVER

    Report to Congressional Requesters

    August 1995

    FOREIGN ASSISTANCE - ASSESSMENT OF
    SELECTED USAID PROJECTS IN RUSSIA

    GAO/NSIAD-95-156

    USAID Projects in Russia

    Abbreviations
    =============================================================== ABBREV

      ARC - American Russian Center, University of Alaska-Anchorage
      BCG - Boston Consulting Group
      CNFA - Citizens Network for Foreign Affairs
      GKI - State Committee of the Russian Federation for the Management
         of State Property
      IBTCI - International Business & Technical Consultants, Inc.
      NIS - New Independent States
      PADCO - Planning and Development Collaborative International
      PIER - Partners in Economic Reform
      PIET - Partners for International Education and Training
      RCG/HB - RCG/Haggler Bailly
      RPC - Russian Privatization Center
      TVG - Tri Valley Growers
      USAID - U.S.  Agency for International Development

    Letter
    =============================================================== LETTER

    B-260365

    August 3, 1995

    The Honorable Benjamin Gilman
    Chairman
    The Honorable Lee Hamilton
    Ranking Minority Member
    Committee on International Relations
    House of Representatives

    The Honorable Sam Nunn
    Ranking Minority Member
    Permanent Subcommittee on Investigations
    Committee on Governmental Affairs
    United States Senate

    This report responds to your request that we evaluate assistance
    projects in Russia managed by the U.S.  Agency for International
    Development (USAID).  Specifically, we determined whether (1)
    individual USAID projects were meeting their objectives and
    contributing to systemic reforms, (2) the projects had common
    characteristics that contributed to their successful or unsuccessful
    outcomes, and (3) USAID was adequately managing its projects in
    Russia.  In conducting our study, we reviewed
    10 judgmentally selected projects with obligations of $64.6 million
    as case studies and used audits and evaluations performed by the
    USAID Inspector General.

       BACKGROUND
    ------------------------------------------------------------ Letter :1

    The United States began providing limited assistance to the Soviet
    Union in December 1990 to support the reform effort and then
    increased assistance after the Soviet Union dissolved in December
    1991.  In October 1992, the Freedom for Russia and Emerging Eurasian
    Democracies and Open Markets Support Act of 1992 (P.L.  102-511),
    commonly known as the Freedom Support Act, was enacted.  It further
    increased assistance to the former Soviet Union and established a
    multiagency approach for providing assistance.  It also called for
    the designation of a coordinator within the Department of State whose
    responsibilities would include designing an assistance and economic
    strategy and ensuring program and policy coordination among federal
    agencies in carrying out the act's policies.

    The Freedom Support Act sets forth the broad policy outline for
    helping former Soviet Union countries implement both political and
    economic reforms.  It also authorized a bilateral assistance program
    that is being implemented primarily by USAID.  In January 1994, the
    State Department Coordinator approved the first overall U.S.
    assistance strategy for the former Soviet Union, and in May 1994, the
    Coordinator approved the strategy specifically for Russia.  This
    strategy has three core objectives:  (1) help the transition to a
    market economy, (2) support the transition to a democratic political
    system, and (3) ease the human cost associated with the transition.
    As of December 1994, USAID had obligated $1.4 billion and spent $539
    million for programs and projects in Russia since fiscal year 1990.
    (See app.  I.)

    USAID's assistance to Russia has focused on 13 sectors, such as
    health care and housing, that support the three U.S.  objectives.
    Hundreds of U.S.  contractors and grantees are responsible for
    implementing individual projects in the 13 sectors.

       RESULTS IN BRIEF
    ------------------------------------------------------------ Letter :2

    Projects have had mixed results in meeting their objectives.  Some of
    the USAID projects we reviewed fully met most or all of their
    objectives, were contributing to systemic reform, and were
    sustainable.  Others did not have all or some of these attributes of
    success.

    The more successful projects had several characteristics in common:
    (1) all levels of the Russian government provided broad and strong
    support, (2) U.S.  contractors had a long-term physical presence in
    Russia, and (3) these projects had a specific sustainability
    objective.  Also, successful projects generally complemented or
    supported ongoing Russian initiatives.  Projects that did not meet
    all their objectives lacked one or more of these characteristics,
    were poorly designed and implemented, and often had little or no
    impact on problems.

    USAID did not adequately manage some projects it funded.  The
    devolution of management and monitoring responsibility from USAID's
    Washington office to its Moscow office delayed decision-making and
    created confusion among contractors.  Furthermore, USAID's management
    information systems were inadequate, and it did not adequately
    monitor or coordinate some projects.  USAID has taken steps to
    overcome these problems, including terminating some unsuccessful
    projects, refining its assistance strategy, and undertaking efforts
    to improve project monitoring and evaluation.

    In commenting on this report, USAID said that the difficult operating
    environment in which it worked during the first 2 years of the
    program in Russia cannot be overstated.  We agree that USAID faced
    numerous operating obstacles in getting this program underway, and
    our observations on how well the projects performed should be seen in
    that context.

       PROJECT RESULTS HAVE BEEN MIXED
    ------------------------------------------------------------ Letter :3

    The 10 projects we reviewed showed mixed results in meeting their
    objectives.

      Two projects--coal industry restructuring and housing sector
         reform--met or exceeded their objectives.

      Five projects--voucher privatization, officer resettlement, small
         business development, district heating, and agribusiness
         partnerships--met some but not other objectives.

      Three projects--health care, commercial real estate, and
         environmental policy\1 --met few or none of their objectives.

    --------------------
    \1 The environmental policy project we reviewed was still in the
    early implementation phase.  USAID awarded the initial contract in
    September 1993 to establish offices and technical support, but did
    not sign the agreements to implement the actual project work until
    April and September 1994.  The projects will extend through September
    1997.

          PROJECTS THAT CONTRIBUTED TO
          SYSTEMIC REFORM
    ---------------------------------------------------------- Letter :3.1

    Three of the 10 projects we examined were contributing significantly
    to systemic reform\2 --that is, they were making fundamental
    structural changes.  These projects were effecting change because
    they had sustainability--benefits that extend beyond the project's
    life span--built into their design and they focused on issues on a
    national or regional scale.

    --------------------
    \2 A project can contribute to systemic reform without meeting all of
    its objectives.  Two projects--coal industry and housing sector
    reform--contributed to systemic reform and met their objectives.  The
    privatization project contributed to systemic reform but did not meet
    all its objectives.

             HOUSING SECTOR PROJECT
    -------------------------------------------------------- Letter :3.1.1

    The housing sector project helped Russian ministries and agencies
    implement 38 laws, regulations, and decrees to reform housing
    policies and practices.  The Urban Institute, which implemented the
    project, also completed a series of pilot projects related to housing
    maintenance, mortgage lending, rent reform, and property rights.
    Many of the activities affect the entire country or could be
    replicated in additional cities.  The project helped create new
    institutions, strengthened existing ones, and distributed procedural
    guides and manuals to local governments as a way to sustain the
    reforms.

             PRIVATIZATION PROJECT
    -------------------------------------------------------- Letter :3.1.2

    The contract for implementing the voucher privatization project
    called for Deloitte & Touche to establish 35 voucher clearing centers
    in cities throughout Russia.  This project encountered some
    difficulty in meeting its initial time frames and establishing all
    the centers, but overall the project was successful.  The active
    centers handled 70 million voucher transactions as part of Russia's
    unprecedented privatization program, and over half the centers
    participate in ongoing capital market activities.

             COAL RESTRUCTURING
             PROJECT
    -------------------------------------------------------- Letter :3.1.3

    Partners in Economic Reform (PIER), which implemented the coal
    restructuring project, has facilitated movement toward the
    transformation of the entire coal sector.  PIER helped build a
    consensus for reform among the Russian government, mine labor unions,
    and mine management.  It was also instrumental in facilitating a
    World Bank review that could lead to a restructuring loan.  To
    sustain the project's contribution to systemic reform, PIER helped
    establish long-term business relations between the Russian and U.S.
    coal industries, formed a consortium of U.S.  coal-related business,
    and is involved in social safety net and new job-creation activities.
    Finally, PIER helped facilitate the sale of U.S.  equipment in
    Russia.

          PROJECTS WITH RELATIVELY
          LIMITED CONTRIBUTION TO
          SYSTEMIC REFORM
    ---------------------------------------------------------- Letter :3.2

    Seven projects we examined did not contribute significantly to
    systemic reform because they either did not meet their objectives,
    were narrowly focused, or lacked sustainability.

             SMALL BUSINESS
             DEVELOPMENT PROJECT
    -------------------------------------------------------- Letter :3.2.1

    The University of Alaska sought to help develop small businesses by
    establishing American Russian Centers in four cities across the
    Russian Far East.  The centers' purpose was to help train
    entrepreneurs, help form new businesses, and build lasting business
    ties between the region and the United States.  To become
    self-supporting after USAID stopped funding, the centers planned to
    develop partnerships with counterpart institutions.  However, the
    centers have so far been unable to attract alternative funding.

             ENVIRONMENTAL POLICY AND
             TECHNOLOGY PROJECT
    -------------------------------------------------------- Letter :3.2.2

    CH2M Hill International Services, Inc., signed a contract in
    September 1993 for an environmental policy development and technology
    project.  The contractor had difficulty filling critical staff
    positions in Russia and providing required work plans for the
    activities.  Of the work plans due in November 1994, one was approved
    in May 1995, while the other was still being revised as of June 1995.

             HEALTH CARE TRAINING
             PROJECT
    -------------------------------------------------------- Letter :3.2.3

    The project to provide health care financing training in the United
    States to Russian health professionals was implemented by Partners
    for International Education and Training (PIET), several training
    institutions, and USAID.  Although PIET and the institutions provided
    the training as required, the Russian participants did not have the
    authority, expertise, or resources to make systemic changes.  In
    addition, changes in Russia's health reform plans have made the
    training irrelevant.

             COMMERCIAL REAL ESTATE
             PROJECT
    -------------------------------------------------------- Letter :3.2.4

    A commercial real estate project, implemented by International
    Business & Technical Consultants, Inc.  (IBTCI), was intended to
    create a standard approach for increasing the availability of
    commercial real estate in six Russian cities.  The project design
    called for a pilot project/roll-out concept, but IBTCI did not roll
    out the pilot in any of the cities and used a different approach in
    each city.  Also, Russian officials said the project had little or no
    effect on the availability of commercial real estate in their cities.

             DISTRICT HEATING PROJECT
    -------------------------------------------------------- Letter :3.2.5

    The district heating project, which USAID recommended we review, was
    implemented by RCG/Haggler Bailly and met its objectives primarily by
    conducting energy audits and training as well as providing energy
    efficiency equipment to two Russian cities.  However, as of February
    1995, some of the equipment in one of the cities had not been
    installed.  Russian officials said the equipment may never be
    installed because Russian authorities never certified it.  USAID had
    not monitored the use of the equipment or followed up on the impact
    of the studies produced for the project.  Consequently, we found no
    indication that the project contributed to systemic reform in the
    energy efficiency area.

             AGRIBUSINESS PARTNERSHIPS
             PROJECT
    -------------------------------------------------------- Letter :3.2.6

    The agribusiness partnerships project, implemented by Tri Valley
    Growers, helped two U.S.  companies establish joint ventures in two
    Russian cities.  Although the involvement of U.S.  companies
    increased the probability that the business ventures would be
    sustained, the limited scope of the partnerships makes it unlikely
    that they will have a significant effect on reforming Russia's
    agricultural sector.  USAID has discontinued the entire agribusiness
    partnerships project in Russia.

             OFFICER RESETTLEMENT
             PILOT PROJECT
    -------------------------------------------------------- Letter :3.2.7

    The Russian officer resettlement pilot project was not intended to be
    sustainable after its completion, but instead was motivated by the
    United States' desire to encourage the withdrawal of Russian troops
    from the Baltic countries.\3 The $6-million pilot project objective
    was to construct 450 housing units to resettle demobilized officers
    by July 1994.  As of February 1995, 422 units were either occupied or
    available for use, so in that sense the project was successful.

    --------------------
    \3 The Baltic countries are Latvia, Lithuania, and Estonia.  The
    United States never officially recognized them as part of the former
    Soviet Union.

       COMMON THEMES TO SUCCESSFUL
       PROJECTS
    ------------------------------------------------------------ Letter :4

    Successful projects (1) had strong support and involvement at all
    levels of the Russian government, (2) had a long-term physical
    presence by U.S.  contractors in Russia, and (3) were designed to
    achieve maximum results by supporting Russian initiatives, having a
    broad scope, and including elements that made them sustainable.  A
    critical element to a project's success was the degree to which
    Russian officials were committed to reform in the particular sector.

          STRONG SUPPORT AND
          INVOLVEMENT AT ALL RUSSIAN
          LEVELS
    ---------------------------------------------------------- Letter :4.1

    Russians at both the federal and local levels demonstrated a strong
    commitment to the projects that were contributing to systemic reform.
    The Russian government also provided financial or in-kind support,
    and Russian nationals held leadership roles in the projects.  For
    example, PIER's approach to implementing the coal project included
    working with officials in the Ministry of Fuel and Energy, Fund of
    Social Guarantees, and the federal coal company; academic
    institutions; oblast'\4 and city officials in the two targeted
    regions; local mine management; and representatives of two labor
    unions.  Russian nationals served as codirectors, and PIER staff
    received free apartments and office space.

    To accomplish the housing sector reform project, the Urban Institute
    worked closely with officials in the Ministries of Finance and
    Economy and the State Committee on Architecture and Construction at
    the federal level, the Moscow city government, various maintenance
    firms, banks, and grass-roots condominium associations.  Although
    office space in Moscow is expensive and scarce, the Institute
    received free office space.  In addition, Russian nationals played a
    key role on the Institute's staff.

    In contrast, many less successful projects lacked the buy-in of
    Russians at either the local or federal level and had little Russian
    involvement or contribution.  For example, the State Committee of the
    Russian Federation for the Management of State Property (GKI),\5
    Russia's federal agency overseeing the privatization effort, was
    instrumental in designing the voucher clearing and commercial real
    estate projects.  However, in some cities, local officials were not
    involved in designing the projects and had little interest in them;
    as a result, these projects were not fully successful.

    The officer resettlement project established housing in several
    cities, but not in Novosibirsk, where city officials reneged on a
    previous administration's commitment to provide needed infrastructure
    support.  Because officials at the federal and oblast' levels were
    not involved in the initial agreements, they had no authority to
    require the new city administration to fulfill the contract, nor were
    they willing to provide additional funding for the project.

    The district heating project was not completed in Yekaterinburg
    because local officials did not allow monitoring equipment to be
    installed.  They said the proper Russian authorities had not
    certified the equipment.

    --------------------
    \4 Oblasts, krays, and republics are regional administrative units
    similar to states in the United States.

    \5 GKI is a Russian acronym.

          LONG-TERM PRESENCE BY U.S.
          CONTRACTORS IN RUSSIA
    ---------------------------------------------------------- Letter :4.2

    The successful projects usually had long-term advisers living in
    Russia, which enabled the advisers to build trust, learn about local
    conditions and plan accordingly, monitor progress closely, and
    correct problems as they occurred.  In addition, successful projects
    involved contractors that had appropriate experience to carry out the
    project.  For example, the Urban Institute has had two long-term
    advisers living in Moscow since 1992 who maintained close contact
    with Russians involved in housing reforms.  PIER's project director
    had lived in Moscow for 3 years.  Other members of its American staff
    had lived in Kemerovo and Vorkuta, the key cities of the major coal
    mining oblasts, since 1993 and 1994, respectively.  The two field
    staff have years of experience as coal mine engineers.  Russian
    officials at all levels (1) praised PIER's staff; (2) described
    PIER's assistance as timely, well-targeted, and beneficial; and (3)
    wanted the project to continue and expand.

    Contractors implementing many of the less successful projects did not
    have staff living in the Russian cities being assisted.  For example,
    neither IBTCI nor RCG/Haggler Bailly had permanently assigned
    American staff in the cities being assisted.  IBTCI's consultants
    would fly in, make rapid diagnoses, deal with problems quickly, and
    then leave.  Many U.S.  officials, Russians, and contractors said
    that relying on "fly-through" consultants rather than permanent staff
    was an ineffective approach.

          DESIGNED TO MAXIMIZE RESULTS
    ---------------------------------------------------------- Letter :4.3

    Successful projects--the housing reform, voucher privatization, and
    coal industry restructuring--were designed to be sustainable, have a
    widespread effect, and support existing initiatives.  Each project
    supported ongoing Russian efforts at widespread reform, considered
    local conditions, and contained elements to sustain the effects of
    the project beyond its life span.

    In contrast, some projects were not designed to maximize their
    potential impact.  For example, the project design required
    RCG/Haggler Bailly to provide energy efficiency equipment and audits
    but did not include methods to replicate the project in other cities,
    or extend monitoring efforts to determine how the equipment or
    studies were used.  The USAID Inspector General reported that other
    projects did not include any follow-up steps to ensure that the
    assistance provided was used.\6 In addition, projects focusing on
    health care training and commercial real estate leasing did not
    consider local needs and conditions and thus had limited impact.

    Several projects did not adequately identify outcomes or measurable
    results.  For example, the Tri Valley Growers' contract with USAID
    did not stipulate how many agribusiness partnerships were to be
    established.  The design of the coal project also did not adequately
    identify outcomes or measurable results, but PIER developed an
    effective project nonetheless.  The USAID Inspector General found
    similar problems when reviewing many projects in the region.\7

    --------------------
    \6 Audit of the Bureau for Europe's Technical Assistance Contracts
    (Report No.  8-180-93-05, June 30, 1993); Audit of the Distribution
    of Emergency Medical Supplies to the New Independent States Under
    USAID Cooperative Agreement With the People-to-People Health
    Foundation "Project Hope" (Report No.  8-110-94-006, Mar.  17, 1994);
    and Audit of Activities to Improve Crop Storage Systems in the New
    Independent States (Report No.  8-110-94-014, Aug.  31, 1994).

    \7 Audit of the Bureau for Europe's Technical Assistance Contracts
    (Report No.  8-180-93-05, June 30, 1993); Audit of the ENI Bureau's
    Monitoring, Reporting and Evaluation System (Report No.
    8-000-95-002, Nov.  28, 1994); and Audit of Selected Privatization
    and Restructuring Activities in Russia (Report No.  8-118-95-007,
    Mar.  10, 1995).

          RUSSIAN INVOLVEMENT AND
          COMMITMENT
    ---------------------------------------------------------- Letter :4.4

    It is widely acknowledged that the Russian people themselves will
    determine the ultimate success or failure of political and economic
    reforms.  Without their involvement and commitment to change, outside
    assistance will have a limited effect.  For example, the support and
    involvement of Russian federal agencies, such as GKI in the
    privatization effort and the ministries related to housing, ensure
    that projects in those sectors are likely to have a wide and
    sustained effect.  The coal project's impact depends on Russia's
    commitment to restructure the coal industry.

    In several sectors, a Russian commitment to reform remains elusive.
    Powerful factions in the Russian legislative branch strongly oppose
    land reform, and the Ministry of Health has not demonstrated a
    commitment to health care reform.  This lack of commitment raises
    concerns that projects in the agriculture and health sectors will not
    have widespread benefits.  USAID is now working with the Ministry of
    Environmental Protection and Natural Resources, but the level of
    support from other important federal ministries, including the
    Ministry of Finance, is still questionable.

    Other domestic conditions will also influence a project's success.
    Russia's commitment to breaking up monopolies and its ability to
    attract capital for modernizing outdated equipment, restructuring
    existing state enterprises, and starting new businesses will affect
    the pace and scope of Russia's transformation to a market economy.
    Moreover, projects such as introducing mortgage lending will depend
    on macroeconomic policy and land reforms.  Russia is counting on
    foreign capital to help move the transition process forward, but such
    factors as the unstable economic situation, a poor and uncertain tax
    structure, an undeveloped financial market infrastructure, and an
    increased crime rate make foreign investors hesitant to invest.

       USAID MANAGEMENT PERFORMANCE
    ------------------------------------------------------------ Letter :5

    USAID responded quickly to assist Russia in undertaking its political
    and economic reforms, as called for in the Freedom Support Act.
    However, to respond quickly, USAID made certain exceptions to its
    normal procedures and processes.  Although USAID provided a quick and
    flexible response to a fluid, unpredictable situation,\8 we
    identified several management problems in addition to design problems
    that occurred, in part, because of the quick response.  USAID
    officials agreed that management problems occurred, but they said the
    risks associated with not responding quickly were high.

    --------------------
    \8 Congress granted USAID special authority to provide a quick
    response.  Section 201 of the Freedom Support Act of 1992 amended the
    Foreign Assistance Act by adding chapter 11, section 498B, that
    waived provisions of law in providing assistance to the former Soviet
    Union.

          LACK OF MONITORING AND
          EVALUATION
    ---------------------------------------------------------- Letter :5.1

    The large size of USAID's program, the vast geographic area receiving
    assistance, and staff limitations have prevented adequate monitoring
    in some cases.  We found that USAID officials were unaware of
    positive and negative aspects of the projects implemented by IBTCI,
    RCG/Haggler Bailly, and PIER.  USAID officials had not visited some
    projects, and USAID did not have representatives located outside
    Moscow.  USAID expected its Russian staff to conduct field
    monitoring, but the Russian nationals lacked the necessary training.
    USAID officials said they considered but rejected the idea of
    establishing field offices outside Moscow.

    Without adequate staff, USAID relied mainly on contractors' written
    and oral reports to monitor the projects, but some contractors did
    not report all problems.  The USAID Inspector General also found
    shortcomings in the reporting process:  contractors were not required
    to report on their progress toward specific objectives or
    indicators.\9

    Moreover, USAID did not enforce some of its reporting requirements.
    For example, Deloitte & Touche did not provide the required lists of
    equipment purchased with USAID funds and brought into the country,
    and USAID did not enforce the requirement.

    Although the State Department allowed USAID/Moscow to increase U.S.
    direct-hire personnel and personal services contractors from 27 in
    fiscal year 1993 to 66 in fiscal year 1995, USAID officials said even
    more staff were needed to adequately monitor the program.  However,
    State would not allow the USAID mission to grow further because,
    among other reasons, the USAID assistance program is scheduled to end
    by the end of the decade.

    In some cases, USAID had not determined the relative success or
    failure of projects so that it could apply lessons learned to other
    efforts.  It did not conduct the required periodic
    assessments/evaluations of the coal and agribusiness projects.  The
    omnibus contracts\10 do not require an evaluation of the individual
    tasks, but instead evaluations are to be done at the end of the
    contracts, according to USAID officials.  The omnibus contracts for
    USAID's private sector initiatives alone have obligated approximately
    $200 million and are not scheduled to terminate until 1996, too late
    to apply lessons learned.  In addition, an evaluation that was
    conducted was not accurate.  A contractor evaluated the district
    heating project in June 1993 and gave it high marks.  Our 1995 review
    of the project found major shortcomings, such as equipment still in
    boxes after being delivered in 1993, even though the evaluation
    report said the equipment had been installed and was being used.  The
    USAID Inspector General also found that evaluations had not been
    conducted and that the quality and impact of some project evaluations
    were questionable.\11

    --------------------
    \9 Audit of Selected Privatization and Restructuring Activities in
    Russia (Report No.  8-118-95-007, Mar.  10, 1995) and Audit of the
    Distribution of Emergency Medical Supplies to the New Independent
    States Under USAID Cooperative Agreement With the People-to-People
    Health Foundation "Project Hope" (Report No.  8-110-94-006, Mar.  17,
    1994).

    \10 USAID's Europe and the New Independent States Bureau often
    procured U.S.  technical assistance through multipurpose contracts,
    commonly referred to as "omnibus" contracts.  These contracts
    provided for the performance of activities, many of which needed to
    be further defined.  USAID used the omnibus contracts to retain the
    services of U.S.  companies to mobilize, either in-house or through
    subcontractors, the resources and expertise needed to identify and
    implement project activities.  The description of work in these
    contracts was very general, but required subsequent development of
    "task orders" and "work plans" to further define the activities the
    contractor was to perform.

    \11 Audit of the ENI Bureau's Cooperative Agreement With World
    Learning, Inc.  for Support to Non-Governmental Organizations in the
    New Independent States of the Former Soviet Union
    (Report No.  8-110-95-008, Mar.  10, 1995).

          PROBLEMS ASSOCIATED WITH
          DELEGATED AUTHORITY
    ---------------------------------------------------------- Letter :5.2

    The devolution of management and monitoring responsibility from
    USAID's Washington office to a rapidly growing Moscow office has not
    been smooth, and several problems have developed as a result.  First,
    as USAID's Moscow office assumed more management responsibility,
    contractors had to begin dealing with another layer of management
    review.  This caused delays and confusion among some contractors.
    Second, there were tensions between the Washington and Moscow offices
    because of differences regarding program implementation.  For
    example, the offices disagreed about which reformers and Russian
    government agencies to work with.  Third, the USAID/Moscow office
    lacked some essential documents to enable officials to carry out
    their duties.  We found that key contract and financial documents
    were not available in Moscow, a problem also reported by the USAID
    Inspector General.\12

    The State Department Coordinator opposed giving greater authority to
    USAID/Moscow because he believed USAID/Washington needed to maintain
    a more prominent role.  He said that because assistance to Russia is
    an important foreign policy issue, key decisions should not be
    delegated to the field.  State and USAID/Washington officials said
    they needed quick access to important project data for reporting
    purposes, but quick access to data could not be ensured when projects
    were managed by the USAID/Moscow office.

    --------------------
    \12 Audit of the Bureau of Europe's Technical Assistance Contracts
    (Report No.  8-180-93-05, June 30, 1993) and Audit of the ENI
    Bureau's Monitoring, Reporting and Evaluation System (Report
    No.  8-000-95-002, Nov.  28, 1994).

          LACK OF ADEQUATE INFORMATION
          SYSTEMS
    ---------------------------------------------------------- Letter :5.3

    USAID has not yet developed a good management information system for
    its Russia program.  The USAID Inspector General reported that USAID
    lacked an information system with baseline data, targets, time
    frames, and quantifiable indicators by which to measure program
    progress and results.\13 USAID's Bureau for Europe and the New
    Independent States was exempted from a new agencywide management
    system because the program was intended to be short term and regional
    rather than long term and country-specific.  USAID officials said the
    pressure to provide assistance quickly meant forgoing the traditional
    project design process, which included developing progress
    indicators.

    --------------------
    \13 Audit of the ENI Bureau's Monitoring, Reporting and Evaluation
    System (Report No.  8-000-95-002, Nov.  28, 1994).

          POOR FIELD COORDINATION
    ---------------------------------------------------------- Letter :5.4

    Part of USAID's assistance strategies was to focus on areas where
    reformers were willing to make changes.  USAID believed this would
    help create a synergy that could stimulate the overall impact of the
    projects.

    Some contractors were not aware of each others' activities.  USAID's
    management information system did not list contractors by region, and
    USAID sometimes did not tell new contractors about other contractors'
    activities.  In some cities, contractors contacted each other on
    their own and started coordinating their efforts.  However, this was
    being done on an ad hoc basis.  In Vladivostok and Yekaterinburg,
    U.S.  Consuls General facilitated contractor coordination.  The USAID
    Inspector General found that many projects with similar goals were
    not linked to one another.\14 Poor coordination reduced the
    opportunity to achieve synergy and targeted impact and gave some
    Russians the impression U.S.  assistance was fragmented and
    uncoordinated.\15

    --------------------
    \14 Audit of ENI's Strategy for Managing Its Privatization and
    Restructuring Activities in Russia (Report No.  8-118-95-009, Mar.
    17, 1995).

    \15 The Russian government has no agency to coordinate assistance
    efforts within the country.

       RECOMMENDATIONS
    ------------------------------------------------------------ Letter :6

    We recommend that the USAID Administrator focus assistance efforts on
    projects that (1) will contribute to systemic reforms; (2) are
    designed to be sustainable; (3) are supported by all levels of
    Russian government; and (4) whenever possible, use American
    contractors with an in-country presence.

       AGENCY COMMENTS AND OUR
       EVALUATION
    ------------------------------------------------------------ Letter :7

    In commenting on a draft of this report, USAID said the three
    projects that we had deemed to have not met their primary objectives
    did produce some positive benefits or it was too early to tell the
    impact the projects would have.  USAID also said it was aware of the
    problems that have occurred and has taken steps to correct them or
    terminate activities that could not be fixed.  USAID pointed to a new
    computerized monitoring system that is expected to produce its first
    report in November 1995.

    USAID agreed with our recommendation regarding the focus of its
    assistance projects and said it was taking steps to implement it.  In
    addition, USAID said it was taking corrective action to address the
    management problems we identified.  However, it stressed that its
    assistance has had a positive impact and occurred in a difficult
    operating environment.  USAID indicated that it had made progress in
    setting up its own monitoring, reporting, and evaluation system.  It
    should be pointed out, however, that in November 1994, the USAID
    Inspector General reported that the system was still far from able to
    measure program results.\16

    USAID said that our report would have provided a more balanced and
    accurate view of the systemic impact and sustainability of a
    project's activities if we had considered the activity in the context
    of the whole program.  USAID stated that, in nearly every case, the
    individual projects we focused on were part of a larger project or
    program that would have substantial impact on reforming Russia's
    economy.  USAID is correct that the projects we examined were usually
    one component of a larger sector program; however, USAID is incorrect
    in its assertion that we evaluated projects in isolation and without
    considering the context of the whole program.  The overriding
    objective of USAID's program in Russia is to contribute to reforming
    both the political and economic systems.  This is also the objective
    of the assistance program for each sector and, with few exceptions,
    of each project that supports a sector program.  Our examination
    focused on the individual building blocks that support sector
    programs and ultimately support the reform effort in Russia.  In some
    cases, we found that the individual building blocks will not
    contribute to systemic reform in the sector or in Russia overall.
    Even though this does not mean that an entire program, of which a
    less-than-successful project is a part, will fail in its systemic
    reform objective, it does mean that an unsuccessful project is not
    contributing to a program's success.

    We also disagree with USAID's assertion that significant systemic
    reform has resulted from USAID activities in all sectors.  For
    example, the agribusiness partnerships project, including components
    reviewed by the USAID Inspector General, comprises most of the USAID
    funding going to the sector but is not expected to contribute
    significantly to systemic reforms.  Only a limited degree of systemic
    reform has occurred in other sectors as well, including the health
    care and the environmental sectors.  We believe that a sector
    evaluation, although useful in its own right, would not have allowed
    us to draw conclusions about the role and contribution of individual
    projects.

    USAID provided other comments that we incorporated into the report
    where appropriate.  The full text of USAID's comments is reprinted in
    appendix IV.

    --------------------
    \16 Audit of the ENI Bureau's Monitoring, Reporting and Evaluation
    System (Report No.  8-000-95-002, Nov.  28, 1994).

       SCOPE AND METHODOLOGY
    ------------------------------------------------------------ Letter :8

    We judgmentally selected 10 individual projects from 6 sectors to
    review as case studies.  We selected projects based on their
    geographic distribution, focusing on regions where several projects
    were concentrated.  We also considered the level of obligations and
    expenditures; the type of assistance provided (e.g., training,
    technical assistance, and product delivery); and the type of
    contracting vehicle (e.g., cooperative agreements, grants, and
    contracts).  We generally did not review projects examined by the
    USAID Inspector General, although we analyzed its work to assess
    whether common themes emerged.  (See app.  II for a list of the 10
    projects we studied and USAID Inspector General reports we reviewed.)

    We analyzed USAID and project documents and interviewed USAID and
    other U.S.  government officials, U.S.  contractors, Russian
    counterparts, and beneficiaries.  We visited project sites in Western
    Russia, Siberia, and the Russian Far East in November 1994 and
    February 1995.

    Appendix III provides a detailed analysis of the 10 projects in our
    case study.

    We conducted our work from September 1994 to April 1995 in accordance
    with generally accepted government auditing standards.

    ---------------------------------------------------------- Letter :8.1

    As agreed with your offices, unless you publicly announce this
    report's contents earlier, we plan no further distribution of this
    report until 30 days after its issue date.  At that time, we will
    send copies to the Secretary of State, the Administrator of USAID,
    and other interested congressional committees.  Copies will also be
    made available to others on request.

    Please contact me at (202) 512-4128 if you or your staff have any
    questions concerning this report.  Major contributors to this report
    are listed in appendix V.

    Harold J.  Johnson, Director
    International Affairs Issues

    OBLIGATIONS AND EXPENDITURES FOR
    ASSISTANCE TO RUSSIA
    =========================================================== Appendix I

    In his January 1995 annual report, the State Department Coordinator
    reported about $2.9 billion in obligations and $1.8 billion in
    expenditures for Russia through December 1994.  (See table I.1.)
    Between fiscal years 1990 and 1994, the U.S.  Agency for
    International Development (USAID) allocated assistance to the New
    Independent States (NIS) as a region.  During that time, most
    projects spanned different countries and USAID did not track how much
    money was obligated or expended by country.  Thus, USAID country
    attributions are estimates and should be treated as such.  In fiscal
    year 1995, USAID began keeping country accounts.

                              Table I.1

                   Obligations and Expenditures for
               Assistance to Russia (Fiscal Years 1990
                            through 1994)

                        (Dollars in millions)

    Agency                             Obligations  Expenditures
    --------------------------------  ------------  ------------
    USAID direct                          $1,230.0        $440.4
    USAID transfers and allocations          171.1          98.1
     to other agencies
    ============================================================
    USAID subtotal                         1,401.1         538.5
    Department of Defense                    291.0          72.9
    Department of Agriculture              1,192.0       1,192.0
    ============================================================
    Total                                 $2,884.1      $1,803.4
    ------------------------------------------------------------
    Source:  Department of State.

    PROJECTS AND USAID INSPECTOR
    GENERAL REPORTS GAO REVIEWED
    ========================================================== Appendix II

    We reviewed 10 projects in depth as part of our review.  In addition,
    we reviewed various reports that USAID's Inspector General has issued
    on management issues and projects in Central and East Europe and the
    former Soviet Union.  The 10 projects and the USAID Inspector General
    reports we reviewed are listed in table II.1.

                              Table II.1

                        Projects GAO Reviewed

                        (Dollars in millions)

                                    Start   End
    Project/implementing partner    date    date     Obligations
    ------------------------------  ------  ------  ------------
    Housing policy reform/Urban     Sept.   Aug.            $5.8
    Institute                       92      94

    Voucher privatization/          Feb.    June             4.1
    Deloitte & Touche               93      94

    Coal project/Partners in        June    Aug.           8.0\a
    Economic Reform                 92      95

    Small business development/     Apr.    May 95           5.1
    American Russian Center,        93
    University of Alaska

    Environmental policy &          Apr.    Sept.         26.4\b
    technology/                     94      97
    CH2M Hill International
    Services

    District heating/RCG Haggler    Feb.    Dec.             1.3
    Bailly                          92      93

    Health care training/Partners   Fall    Fall             0.7
    for International Education     93      93
    and Training

    Commercial real estate/         Oct.    Dec.             2.0
    International Business &        93      94
    Technical Consultants, Inc.

    Agribusiness partnerships/Tri   Feb.    Sept.          5.2\a
    Valley Growers                  93      97\c

    Russian officer resettlement/   July    July             6.0
    Planning and Development        93      94\e
    Collaborative International\d

                                                           $64.6
    ------------------------------------------------------------
    Note:  Obligations are as of March 1995.

    \a Amount could be applied to other countries in the NIS as well as
    Russia.

    \b Total estimated amount of three contracts.

    \c USAID stopped accepting proposals for partnerships in Russia in
    September 1994.

    \d Primary implementing partner.

    \e Project had not been completed as of April 1995.

    The following are the USAID Inspector General reports GAO reviewed.

      Audit of the Bureau for Europe's Technical Assistance Contracts
         (Report No.  8-180-93-05, June 30, 1993).

      Audit of the ENI Bureau's Monitoring, Reporting and Evaluation
         System (Report No.  8-000-95-002, Nov.  28, 1994).

      Audit of ENI's Strategy for Managing Its Privatization and
         Restructuring Activities in Russia (Report No.  8-118-95-009,
         Mar.  17, 1995).

      Audit of Selected Privatization and Restructuring Activities in
         Russia (Report No.  8-118-95-007, Mar.  10, 1995).

      Audit of the Reestablishment of Vaccine Production Activity Under
         the New Independent States Health Care Improvement Project No.
         110-0004 (Report No.  8-110-94-004, Feb.  25, 1994).

      Audit of the Medical Partnerships in Russia and Health Information
         Clearing House Activities Under the New Independent States
         Health Care Improvement Project (Report No.  8-110-94-005, Feb.
         28, 1994).

      Audit of the Distribution of Emergency Medical Supplies to the New
         Independent States Under USAID Cooperative Agreement With the
         People-to-People Health Foundation "Project Hope" (Report No.
         8-110-94-006, Mar.  17, 1994).

      Audit of the Vulnerable Groups Assistance Program in Russia Under
         Project No.  8-110-0001 (Report No.  8-110-93-08, Sept.  24,
         1993).

      Audit of Activities to Improve Crop Storage Systems in the New
         Independent States (Report No.  8-110-94-014, Aug.  31, 1994).

      Audit of the ENI Bureau's Cooperative Agreement With World
         Learning, Inc., for Support to Non-Governmental Organizations in
         the New Independent States of the Former Soviet Union (Report
         No.  8-110-95-008, Mar.  10, 1995).

      Audit of the Department of Commerce's Special American Business
         Internship Training Program in the New Independent States
         (Report No.  8-110-93-10, Sept.  24, 1993).

      Audit of the Department of Commerce's Consortia of American
         Businesses in the New Independent States Program (Report No.
         8-110-93-11, Sept.  24, 1993).

      Audit of the Nuclear Regulatory Commission's Technical Assistance
         Activities in Russia (Report No.  8-118-94-012, June 28, 1994).

      Audit of the Department of Energy's Nuclear Safety Technical
         Assistance Activities in Russia and Ukraine (Report No.
         8-110-95-001, Oct.  7, 1994).

    PROJECT SUMMARIES
    ========================================================= Appendix III

    The following provides a detailed analysis of each project that we
    reviewed.  We selected one project in each of the following areas:
    (1) housing policy reform, (2) voucher privatization, (3) coal, (4)
    small business development, (5) environmental policy and technology,
    (6) heating, (7) health care, (8) commercial real estate, (9)
    agribusiness partnerships, and (10) officer resettlement.  Each
    summary contains information on the problems in the sector, USAID's
    project objectives for the selected contract, and the project
    approach used by USAID or the contractor.  We also provide our
    assessment of the contractor's performance, the impact on systemic
    reform, and USAID's management of the contract.  The projects are
    presented on the basis of their capacity to contribute to systemic
    reform.

       HOUSING POLICY REFORM--URBAN
       INSTITUTE
    ------------------------------------------------------- Appendix III:1

    The Urban Institute housing project was successful.  It supported
    reforms already underway, used an experienced contractor with staff
    in country, installed local nationals in high-level positions,
    focused its efforts on both the federal and local levels, and
    contained elements that made it sustainable.  Therefore, this project
    will likely have sustained benefits as legislation is implemented and
    new Russian institutions expand the pilot projects into other areas.

          SECTOR PROBLEM
    ----------------------------------------------------- Appendix III:1.1

    Russia's housing sector \1

    has been beset by housing shortages, production inefficiencies,
    maintenance problems, and deterioration.  This situation occurred
    primarily because the state had a monopoly on housing.  For example,
    it (1) used standardized apartment buildings constructed by
    state-owned companies, (2) controlled apartment construction and
    maintenance, (3) financed all state housing from state assets, (4)
    almost totally subsidized housing and maintenance, (5) guaranteed
    low-cost housing, and (6) distributed housing through waiting lists.
    In addition, because the Soviet government had not raised rents since
    1928, rents covered less than 5 percent of the cost of operating the
    apartments in 1990.  The problem was exacerbated when the Russian
    Federation government stopped paying for maintenance cost of
    apartments and they fell into disrepair.  In addition, the Federation
    devolved the housing assets and responsibilities to municipalities as
    a way of relieving itself of the burdens associated with managing the
    apartments.

    Russia initiated housing reforms in 1991 when it allowed its citizens
    to privatize their apartments at little to no cost.  This action set
    the stage for establishing a private housing sector.

    --------------------
    \1 This information was obtained from (1) USAID's Housing Sector
    Reform Project for the NIS and (2) the Urban Institute's report, The
    Russian Housing Market in Transition.

          PROJECT OBJECTIVES
    ----------------------------------------------------- Appendix III:1.2

    USAID signed its first contract for housing sector reform with the
    Urban Institute in September 1992 for $5.8 million.  This 2-year
    contract required the Institute to provide draft legislative and
    financial advisers to help Russia develop market-oriented housing
    programs and legislation.  Other Institute advisers were expected to
    conduct pilot projects on (1) rent reforms and housing allowances for
    the poor, (2) privatized housing maintenance, (3) condominium
    associations, and (4) mortgage lending.  It was also expected to
    provide targeted training to those implementing reforms and develop
    local institutions to sustain and expand the reforms.  Specific
    objectives and milestones were incorporated into the project design.

          PROJECT APPROACH
    ----------------------------------------------------- Appendix III:1.3

    A USAID team that included Institute representatives met with their
    Russians counterparts in early 1992 to determine their reform
    priorities.  From 12 to 15 meetings at both the national and
    municipal levels in Moscow were needed to clarify Russian reform
    priorities.  To help focus Russian priorities, the team used a "menu"
    of reforms based on experience in housing reforms in Hungary and
    developing countries, and then focused on one or two priorities to
    demonstrate results quickly and build confidence.

    The Russian priorities were formalized through agreements signed in
    March 1992 with USAID, the City of Moscow, the State Committee on
    Architecture and Construction, the Ministry of Finance, and the
    Ministry of Economy.  The team sought joint agreements with the three
    ministries agencies to (1) ensure it would not become captive to any
    one ministry, (2) ensure broad-based agreement on reform priorities,
    and (3) reduce governmental impediments to reform.  The Russian
    counterparts showed their support for the project by providing the
    Institute with free office space, which is highly unusual due to the
    scarcity of office space in Moscow.

    The team's strategy was to work at the national level to help draft
    legislation that would shape and codify reforms.  In addition, it
    planned a series of local demonstration projects to determine the
    effectiveness of the designs in the Russian context.  The team
    augmented these efforts by providing training in Russia and the
    United States.  A key strategy was to take advantage of the Russian
    reforms already underway and try to create "win-win" situations for
    the government and its citizens.

    The Institute's staffing policies were also important to its
    approach.  It provided two long-term resident advisers, including the
    Program Director, who were located in Moscow.  The Director said
    using advisers who were permanently located in Russia rather than
    "fly-through" consultants helped establish trust with their Russian
    counterparts and enabled them to respond immediately.  The Institute
    also employed five Russian housing experts.  Short-term U.S.
    advisers were used on an as-needed basis.  The Director said that
    using local Russians in key positions was critical to establishing
    trust with the Institute's Russian counterparts.  The large Russian
    staff also was less expensive than U.S.  consultants.

          CONTRACTOR PERFORMANCE
    ----------------------------------------------------- Appendix III:1.4

    The Institute achieved its objectives of helping to develop housing
    legislation.  According to the Russian Federation Housing Director,
    the Institute's assistance was critical in drafting the 38 laws,
    decrees, and regulations that have been implemented.  These included
    laws and regulations on property rights, housing finance, rent
    reform, housing allowances for the poor, privatized maintenance,
    condominiums, and mortgage lending.  The Institute is now the
    government's principal housing adviser.

    The Institute also achieved its objectives of establishing pilot
    projects in four areas:  rent reform and housing allowances for the
    poor, privatized maintenance, condominiums, and mortgage lending.

          RENT REFORM AND HOUSING
          ALLOWANCES
    ----------------------------------------------------- Appendix III:1.5

    The Institute helped the City of Moscow develop a program that would
    raise rents over a 5-year period until they covered all the costs of
    operating the apartments.  To reduce resistance to rent increases, it
    tied maintenance improvements to the increases so citizens would see
    an immediate improvement in their housing conditions.  In 1994, the
    Federation initiated a national 5-year program to increase tenant
    payments to cover the full operating costs.  The Institute also
    helped the Federation structure a program in which the municipalities
    began providing housing allowances to the poor.

          PRIVATIZED MAINTENANCE
    ----------------------------------------------------- Appendix III:1.6

    The Institute helped introduce competitive private maintenance for
    municipal housing.  It conducted training sessions, organized the
    competition to select private firms, and conducted a study tour to
    the United States so officials could see private maintenance
    activities.  In March 1993, three private maintenance firms assumed
    management of 2,000 apartments in Moscow, and in October 1993,
    Moscow's mayor extended the program to all areas of the city.  By
    1994, over 60,000 apartments were under privatized maintenance, far
    surpassing the project's goal of 2,000.

          CONDOMINIUM ASSOCIATIONS
    ----------------------------------------------------- Appendix III:1.7

    The Institute's goal for the condominium pilot was to lay the legal
    and procedural groundwork by 1994.  However, it surpassed this goal
    and helped to create 24 functioning condominium associations in
    Ryazan'.  The regulations it helped develop were instrumental in
    registering the first condominium association in Moscow.

          HOME MORTGAGES
    ----------------------------------------------------- Appendix III:1.8

    To address mortgage-lending problems, the Institute developed
    mortgage-lending facilities at several banks; limited lending has
    begun.  For example, the Institute helped Mosbusinessbank, Russia's
    third largest commercial bank, to make home mortgage loans and
    provided assistance in all phases of operations, including legal
    documentation, underwriting, loan servicing, mortgage loan instrument
    development, and risk management.  The Institute then expanded its
    work to eight other banks and provided the necessary materials to
    other banks to expand and sustain mortgage lending.  However,
    hyperinflation has precluded lending to most Russians.

          RESULTS
    ----------------------------------------------------- Appendix III:1.9

    The Institute's critical assistance helped transform Russian
    priorities into workable legislation and pilot projects.  Although
    the Russians are responsible for the pace of reforms, the Institute
    has helped effect systemic changes in Russia's housing sector.  It
    helped pass far-ranging laws that have codified reforms.  The program
    to raise rents and provide subsidies for the poor, which is being
    implemented across the country, is a fundamental change for the
    government and its citizens.

    The project has a strong sustainability component.  Over the next
    several years, it plans to institutionalize the reforms by expanding
    the number of demonstration projects and developing private
    maintenance organizations, condominium associations, and mortgage
    banks.  In addition, it created procedures manuals, explanatory
    guides, and other necessary documentation on implementing rent
    increases, beginning privatized maintenance, creating condominium
    associations, and developing mortgage lending.  The Institute has
    distributed more than 25,000 of these documents, mostly to local
    governments.

    The project has won high praise from USAID and Russian officials.
    The USAID Mission Director in Moscow called the project one of the
    most successful ones he had ever seen.  A USAID official in
    Washington said that, for the money, no USAID project has had more
    macroeconomic impact.  The Russian Federation Housing Director noted
    the Urban Institute's tremendous influence on the government, and
    Russian citizens working in maintenance, condominium associations,
    and mortgage lending also praised the project.

    Despite the program's progress, most Russians have yet to benefit
    from the reforms.  This is because the reforms are relatively recent,
    are tremendously complex, and face opposition by antireformists; they
    are also being implemented in a country with no tradition of
    market-based decisions.  Private land ownership rights are still
    generally uncertain, housing and construction mortgages are generally
    unavailable, additional laws and regulations are needed, and most
    apartment buildings are still maintained by state organizations.  In
    addition, factors beyond the housing sector, such as macroeconomic
    and political instability, slow the transformation to a fully
    developed privatized housing sector.

          USAID MANAGEMENT
    ---------------------------------------------------- Appendix III:1.10

    USAID successfully managed the contract.  It determined the Russians'
    reform priorities, incorporated these into its reform plan, and
    listed these in its contract.  USAID selected a contractor with
    experience in both the sector and region and is effectively
    monitoring the reforms through regular contacts with the contractor
    and Russian counterparts.  Both USAID/Washington and USAID/Moscow
    agreed on the housing strategy.  USAID also had the contractor
    develop measurable goals in its annual work plan.  USAID then
    measured the contractor's progress by comparing its task orders to
    the deliverables.

       VOUCHER PRIVATIZATION--DELOITTE
       & TOUCHE
    ------------------------------------------------------- Appendix III:2

    As part of Russia's privatization effort, Deloitte & Touche
    established a national system of centers to process millions of
    vouchers that Russians received and used in the privatization
    process.

    Overall, the Deloitte & Touche voucher privatization project was
    successful, with a few exceptions.

      The project focused on national reforms, but some areas had lower
         Russian participation than expected.

      Deloitte & Touche kept USAID and the State Committee of the Russian
         Federation for the Management of State Property (GKI) informed
         of project progress but did not meet some of its reporting
         requirements.

      Deloitte & Touche met its amended objective of setting up 30
         centers, but many were underused.

    Several factors contributed to the overall success of the project.
    The Russian GKI helped focus assistance efforts and identified
    problems when USAID had minimal field presence.  Further, the omnibus
    contract system allowed the contractor to institute a rapid roll-out
    as well as adjust the scope of work when warranted.  In addition,
    using existing Russian agencies and using staff and equipment for
    follow-on activities increased the project's effects and
    sustainability.

          SECTOR PROBLEM
    ----------------------------------------------------- Appendix III:2.1

    Because the state controlled Russian enterprises, which were
    generally large and monopolistic, the private sector was virtually
    nonexistent.  The legal and regulatory framework to create the new
    system was not in place; few citizens had entrepreneurial experience
    or exposure to western management, accounting, and marketing
    concepts; and no capital market infrastructure existed.\2

    In August 1992, President Yeltsin announced plans to privatize
    Russia's large and medium-size state-owned enterprises.  Within
    weeks, distribution of privatization vouchers began, with each
    Russian citizen eligible to receive one voucher.  The sale of the
    enterprises was expected to reduce the need for massive state
    subsidies, begin to reduce inefficiencies, and eventually lead to
    higher productivity and innovation as shareholders demand profits.

    Voucher privatization was the initial step in the overall
    privatization process and was used to transfer ownership from the
    state to private individuals.  Unlike the approaches used in some
    Central European countries, Russia chose to privatize enterprises
    before restructuring them.\3 The process is therefore not complete:
    restructuring must still take place before the enterprises can
    function in a market economy.  This may be difficult because
    management and workers received a majority of shares and can resist
    taking the painful steps necessary for restructuring.

    --------------------
    \2 We obtained this information mainly from USAID's project
    memorandum for private sector initiatives, GKI's annual report, and
    the World Bank document, Russia:  Creating Private Enterprises and
    Efficient Markets.

    \3 Enterprise restructuring typically involves organization and
    management changes, choice of product markets, development of
    marketing and distribution capabilities, and reductions in staff.

          PROJECT OBJECTIVES
    ----------------------------------------------------- Appendix III:2.2

    The voucher-clearing centers allowed individual Russians and
    investment fund managers to more easily buy shares via electronic
    transfers in enterprises located in remote areas.  Without the
    centers, people would have had to physically transport vouchers to
    other parts of the country.  There was also a fear that regions would
    not let outsiders, including foreigners, buy shares in highly visible
    enterprises, thereby allowing insiders and local bureaucrats to
    control the process.

    The specific objectives of the project required Deloitte & Touche to
    establish 35 functioning centers in various Russian cities to verify,
    process, and cancel voucher receipts.  The project was carried out
    under two separate contracts at a total cost of $4.1 million.  The
    initial contract (as amended) required the contractor to establish 20
    centers by the end of 1993, and a task order under the omnibus
    contract required 15 more before March 1994.  This would give
    citizens enough time to process their vouchers before the
    privatization program ended in July 1994.  USAID and GKI, the Russian
    agency overseeing the national privatization effort, hoped that many
    of the centers would develop into institutions, such as registrars
    and depositories, in the capital market infrastructure.  The number
    of vouchers the centers were to process was not defined.

          PROJECT APPROACH
    ----------------------------------------------------- Appendix III:2.3

    USAID worked closely with GKI on project design, which called for
    Deloitte & Touche to develop 4 pilot centers and then establish 16
    more after successfully setting up the pilot sites.  To provide
    broader geographic coverage, USAID and GKI decided to extend the
    project and have Deloitte & Touche set up 15 more centers.
    Consultants from another USAID contractor, the Harvard Institute for
    International Development, worked with GKI to design and monitor the
    project.

    Deloitte & Touche established a permanent office in Moscow in June
    1990 and opened a separate office for this project in early 1993.  It
    worked closely with GKI in Moscow and GKI's local offices in various
    Russian cities to identify appropriate cities for the centers and
    suitable partners.  Deloitte & Touche then imported computer
    equipment, established accounting systems, and installed the software
    and telecommunications systems needed to facilitate voucher
    transactions.  Teams of Deloitte & Touche staff then traveled to the
    cities to train center staff, install and test the equipment, and
    test the software and telecommunications systems.  The contractor
    hired Russians to assist with these efforts and usually supplemented
    the work of Russian agencies, typically banks, already working in
    related fields.

          CONTRACTOR PERFORMANCE
    ----------------------------------------------------- Appendix III:2.4

    Under the first contract, Deloitte established all 20 centers before
    its deadline.  Under the second contract, USAID and GKI reduced the
    number of centers from 15 to 10 and extended the deadline by 3-1/2
    months because of implementation delays.  The delays took place
    because of problems with equipment procurement and Russian government
    customs clearance; difficulties locating viable agencies to act as
    centers; and problems at the local level.  For example, some centers
    collapsed when their leadership changed or chose not to participate
    on a national scale for local political reasons.  Both parts of the
    project were completed under budget.

    Of the 30 centers Deloitte & Touche set up, only 23 were used, and
    many of these experienced relatively little activity.  The lack of
    use was attributed to delays in setting up some centers; limited
    public awareness (centers were not responsible for advertising their
    services); limited local interest in voucher auctions in other areas;
    Russian reluctance to use electronic transfers; and a lack of
    compatibility between the project goals and individual center goals.

    Deloitte & Touche was responsive to GKI requests for project changes.
    The task order was revised once it became clear that all 15 centers
    would not be needed.  In some cases, Deloitte & Touche went beyond
    the required tasks at GKI's request.

    Deloitte & Touche generally kept USAID and GKI informed with monthly
    reports on progress and problems.  However, some reports were not
    filed as required, and Deloitte & Touche did not provide an adequate
    inventory of the equipment it procured, which would have ensured the
    accountability and tracking as required by the company's contracts.

          RESULTS
    ----------------------------------------------------- Appendix III:2.5

    The project is considered a success although not cost-effective.  A
    functioning national system was created in a short time, and
    according to GKI, it handled over 70 million vouchers, nearly half
    the vouchers processed in the program.  People were able to buy
    shares in enterprises located in remote areas.  GKI noted that over
    half of the centers have evolved into institutions that are now
    active in capital market activities, such as share registrars and
    depositories.  Our visits to three centers verified that center staff
    and equipment are being used in follow-on activities.  These centers
    intend to become self-financing on a fee-for-service basis when USAID
    assistance ends.

    The scope of voucher privatization in Russia was unprecedented in
    scale and speed.  According to Russian and U.S.  officials, USAID's
    support of GKI and other Russian institutions involved in
    privatization activities was crucial to this phase of the program.
    The Russian Privatization Center estimated that 14,000 large and
    medium-sized enterprises were privatized by July 1994; they employed
    over 60 percent of the industrial workforce.  Nevertheless, the
    overall effect of the privatization program on Russia has yet to be
    determined.  Enterprise restructuring has only begun, monopolies
    still exist, and inadequate tax legislation makes foreigners
    reluctant to provide badly needed capital investment.

          USAID MANAGEMENT
    ----------------------------------------------------- Appendix III:2.6

    USAID used an omnibus contract to plan and implement projects
    quickly.  It allowed USAID to respond quickly to emerging needs
    through task orders that included specific objectives for narrowly
    focused, short-term projects.  USAID officials said this gave them
    the flexibility to change directions quickly, move money into areas
    and projects making rapid progress in reform, and adjust projects to
    meet emerging needs.  Omnibus contracts also allowed USAID to
    obligate a large amount of funding.  Deloitte & Touche has an omnibus
    contract for $41.5 million, with subcontractors performing some of
    the work.  However, the individual task orders lacked an evaluation
    requirement.  USAID officials said an evaluation is planned only at
    the end of the omnibus contract, which ends June 30, 1996.

    We identified several USAID management problems.  For example, USAID
    did not design the project with quantifiable indicators to measure
    progress.  Although Deloitte & Touche set up 30 voucher clearing
    centers, the project design did not specify the amount of activity
    expected at each center or on a national scale.  USAID/Moscow had
    limited information on the project, such as key documents on Deloitte
    & Touche's initial contract or task order, and accurate financial
    data for the project.  USAID officials said key documents had not
    been transferred to Moscow when management moved to Moscow from
    Washington.  Also, the physical distances involved, the geographic
    distribution of project activities, and the lack of staff to visit
    the sites left USAID uninformed about some Deloitte & Touche
    activities.  USAID officials said they relied heavily on GKI and
    Harvard consultants to help monitor the project.  Finally, USAID did
    not require Deloitte & Touche to provide adequate inventory data on
    the $1.1-million worth of equipment purchased with USAID funds.  Not
    having this data hindered USAID from using surplus equipment in other
    projects as planned.

       THE COAL PROJECT--PARTNERS IN
       ECONOMIC REFORM
    ------------------------------------------------------- Appendix III:3

    The coal project is achieving its primary objective of facilitating
    the restructuring of Russia's coal industry and is opening the
    industry to American technology and companies.  The Russian
    beneficiaries expressed appreciation for the assistance and found it
    useful.  Due to the size and cost of the restructuring, however, the
    Russian government must complete the effort.  If the World Bank
    approves a $500-million sector loan, this project will have played a
    key role in restructuring the coal sector.  Although the project is
    meeting its objective, USAID did not provide adequate oversight, did
    not fully understand the beneficiaries' needs or opinions about USAID
    assistance, and did not know the extent to which the project was
    meeting its goals.

          SECTOR PROBLEM
    ----------------------------------------------------- Appendix III:3.1

    Coal is an important component of Russia's economy.  However,
    Russia's coal sector suffers from declines in production and serious
    environmental and safety liabilities, in large part because of the
    centralized structures, subsidized pricing, Soviet-style management,
    and state allocation system.  To solve these problems, the coal
    industry needs to be restructured.\4

    The process of restructuring is both a problem and a solution because
    it creates new challenges.  The major areas that need to be addressed
    in restructuring Russia's coal industry and transitioning from a
    centrally planned to a market economy include reducing the numbers of
    mines and miners as well as the amounts of coal produced and
    government subsidies.  In addition, the coal monopoly must be broken
    up, mines must be privatized, and new relationships and agreements
    must be established between management and labor.  Efforts to
    restructure the coal industry are complicated because the
    state-subsidized coal mines provide many social services and may be
    the only source of energy or employment in the areas where they are
    located.

    Coal industry restructuring will take a heavy toll on miners and
    their families as the industry streamlines its operations, mines are
    closed, and miners lose their jobs.  These same miners, who could
    lose their jobs as a result of the Yeltsin reform program, were
    instrumental in bringing Yeltsin to power in 1991.  The mining
    community in Russia is still considered a politically powerful force.

    President Yeltsin took a major step toward restructuring Russia's
    coal industry in July 1993 when he freed coal prices.  Since that
    time, the industry has made some progress.  For example,
    approximately 72,000 of the 914,000 coal miners and others employed
    by the coal sector in 1992 left the industry between January 1993 and
    June 1994.  In addition, coal production decreased by approximately
    41 percent from 1988 to 1994.  The government also reduced subsidies
    to the coal industry by approximately 20 percent in real terms (i.e.,
    taking inflation into account) between 1993 and 1994.  Finally, the
    Russian coal industry closed 2 of its approximately 273 mines, was in
    the process of closing 14 more mines in 1994, and is preparing to
    close 40 more in the future.

    --------------------
    \4 This information was obtained primarily from coal project
    documents and officials; PlanEcon reports; the World Bank's report,
    Russian Federation Restructuring the Coal Industry:  Putting People
    First (Dec.  1994); and the U.S.  Department of the Interior's
    report, A Cost Comparison of Selected U.S.  and Russian Coal Mines
    (Sept.  1994).

          PROJECT OBJECTIVE
    ----------------------------------------------------- Appendix III:3.2

    As part of USAID's broader effort to assist Russia's energy sector,
    USAID signed a cooperative agreement with the Partners In Economic
    Reform (PIER), a private, nonprofit organization established to
    assist the coal industries in Russia, Ukraine, and Kazakstan.\5

    USAID signed the agreement with PIER for $6.9 million in June 1992
    and has increased funding since then to $8 million.  The project's
    main objective is to facilitate the transformation of the centrally
    planned and controlled coal mining industry to an industry capable of
    operating in a market economy.  The cooperative agreement did not
    specify any measurable goals or deliverables.

    --------------------
    \5 The scope of our evaluation is limited to PIER's activities in
    Russia.

          PROJECT APPROACH
    ----------------------------------------------------- Appendix III:3.3

    In 1989, U.S.  coal representatives visited some of the coal regions
    of the Soviet Union where miners were starting to form independent
    unions, and between 1989 and 1991, groups of independent miners met
    with leaders of the U.S.  coal industry labor and management in the
    United States.  In June 1991, a memorandum of understanding,
    pertaining to continued assistance, was signed by U.S.  and Russian
    coal industry representatives.

    During 1991, circumstances changed drastically.  Boris Yeltsin was
    elected President of Russia in June, the communists mounted a failed
    coup attempt in August, and the Soviet Union dissolved in December.
    These changes opened the door for a broad U.S.  technical assistance
    program in Russia.  As part of this effort, the State Department
    announced the coal project in a January 23, 1992, press release and
    signed a cooperative agreement on June 25, 1992.

    The coal project gained early acceptance because PIER targeted the
    project at a problem (i.e., coal industry restructuring) that the
    Russians had already identified and were struggling to address.  In
    addition, PIER established good working relationships.  For example,
    PIER established a coordinating office in Moscow and cooperation and
    development centers in the Russian cities of Kemerovo and Vorkuta.\6
    An American director heads the coordination office, and an American
    director and a Russian codirector head the cooperation and
    development centers.  In addition, because the American staff lived
    in Russia, they were able to develop and maintain long-term
    relationships with the Russian government, coal industry management,
    and labor unions.  The Russians further demonstrated their support by
    donating rare office space for the coordination offices and
    cooperation and development centers and donating apartments for the
    American directors in Kemerovo and Vorkuta.

    PIER staff worked closely with representatives from the Russian
    government, coal industry management, and labor to (1) reduce the
    number of mines and miners, (2) develop new sources of employment in
    coal-producing regions to absorb displaced laborers, and (3) develop
    a social safety net for those miners left unemployed during and after
    the transition.  PIER has cultivated cooperative efforts between
    government, management, and labor to address problems associated with
    coal industry restructuring.  In addition to these efforts, PIER
    staff has helped build commercial links between the Russian and
    American coal and coal-related industries.

    --------------------
    \6 PIER has implemented the coal project in three countries--Russia,
    Kazakstan, and Ukraine--under one cooperative agreement.  PIER also
    established coordinating offices and cooperation and development
    centers in Ukraine and Kazakstan.

          CONTRACTOR PERFORMANCE
    ----------------------------------------------------- Appendix III:3.4

    PIER made progress in facilitating the restructuring of Russia's coal
    industry and opening the Russian market to U.S.  mining technology.
    Specifically, PIER

      worked closely with the World Bank to evaluate Russia's coal
         industry and develop a restructuring plan;

      conducted detailed studies of employment, unemployment, and social
         programs; government subsidies; labor demand; a social safety
         net, job creation, and mine planning; and enterprise debt in the
         Russian coal industry;

      established a coal-bed methane recovery center;

      mediated discussions between U.S.  and Russian officials on
         equipment certification in an effort to open the Russian market
         to U.S.  high-tech safety equipment;

      established a program to facilitate U.S.  private sector
         investment;\7

      hosted approximately 150 representatives of the Russian government
         and coal industry in the United States where they participated
         in meetings and negotiations with World Bank officials, training
         seminars, and meetings with U.S.  coal industry representatives;

      provided training material and conducted seminars in Russia
         concerning mine safety, labor-management relations, mining and
         mine management in a market economy, and small business
         development in Russia;

      implemented a transition assistance program focused on developing a
         viable social safety net and creating new jobs; and

      provided $200,000 worth of U.S.-manufactured mine health and safety
         equipment to Russian miners.

    The Russian beneficiaries (i.e., government, labor, and management)
    we contacted in Russia stated that the coal project was
    well-targeted, timely, and beneficial.  In addition, they all wanted
    the project to be continued and expanded.

    --------------------
    \7 This program, the PIER-Coalition of American Business in the New
    Independent States in Coal Extraction, Processing, Transportation,
    and Utilization, is intended to promote joint venture and commercial
    activities between U.S.  and NIS private sector counterparts in coal
    and related industries.

          RESULTS
    ----------------------------------------------------- Appendix III:3.5

    PIER has made several contributions to systemic reform.  One of the
    clearest contributions is its work in facilitating a $500-million
    World Bank loan.  By providing U.S.  coal industry experts, PIER
    facilitated the World Bank's work in Russia; contributed extensive
    analysis of the coal industry's problems; built consensus among
    Russian government, management, and labor representatives; and
    brought representative Russian delegations to the United States to
    negotiate with the World Bank.  The World Bank acknowledged PIER in
    its 1994 report for contributing to the Bank's work in Russia.\8

    PIER helped establish relationships between Russian and U.S.  coal
    mining and equipment-manufacturing firms.  According to the
    beneficiaries, these relationships will help Russia attract capital
    investors and gain greater access to U.S.  expertise and technology
    so that it can begin to produce coal efficiently and compete in a
    market economy.  PIER also facilitated the sale of millions of
    dollars of non-USAID-funded U.S.  mining equipment in Russia.  PIER
    formed a consortium of U.S.  industry representatives to help create
    a viable private coal industrial sector.  The consortium is to assist
    coal managers and technicians in operating in a market economy,
    identify ways that private U.S.  firms can participate in
    restructuring the coal industry, establish NIS-U.S.  joint ventures,
    and promote the consortium's services so it can become
    self-sustaining.

    Finally, PIER worked with Russia's Fund for Social Guarantees to
    implement a transition assistance program focused on developing a
    viable social safety net and creating new jobs.  PIER also brought in
    U.S.  experts to provide small business education to miners and
    helped mining communities develop business proposals that can be
    presented to the Russian-American Enterprise Fund, Russian banks, and
    other sources for eventual financing.

    --------------------
    \8 According to a World Bank official, to receive a loan from the
    Bank, the Russian government must meet the Bank's conditions, one of
    which requires developing a restructuring plan.  The Russian
    government provided the Bank with an initial restructuring plan and
    is currently refining the plan to address the Bank's concerns.

          USAID MANAGEMENT
    ----------------------------------------------------- Appendix III:3.6

    USAID started to implement the coal project before it had established
    a USAID mission in Moscow; consequently, the USAID project officer in
    Washington managed the project.  Since the coal project was
    established through a cooperative agreement,\9 without quantifiable
    indicators, PIER designed and implemented the project without direct
    oversight and control by USAID.  PIER provided the required monthly
    program performance reports, annual work plans, and annual progress
    reports to USAID,\10 which then reviewed them.  PIER's staff
    communicated regularly with USAID and felt they had a good reciprocal
    working relationship.

    Despite some success with the project, USAID did not meet its
    monitoring and evaluation requirements.  Although the USAID staff
    should have regularly monitored the project, they visited the Russian
    project sites three times between June 1992 and February 1995.  Two
    of the visits occurred after we began our review.  USAID officials
    said a lack of staff prohibited more frequent visits.  In addition,
    USAID did not conduct the annual assessments or midterm evaluation as
    required and thus lacked an objective basis for evaluating PIER's
    activities and accomplishments.  This, coupled with a lack of
    quantifiable indicators, hindered USAID's ability to independently
    determine the project's impact on coal sector restructuring.

    --------------------
    \9 Cooperative agreements, similar to grants, are awarded to
    nonprofit organizations whose activities are consistent with USAID
    objectives.  USAID is not expected to exercise a substantial degree
    of operational control, but is responsible for approving the
    implementation plans, budgets, and subgrants of the organizations'
    programs.

    \10 PIER also provided USAID with periodic financial reports, and an
    independent accounting firm audited PIER's financial statements in
    1992, 1993, and 1994.

       SMALL BUSINESS
       DEVELOPMENT--UNIVERSITY OF
       ALASKA
    ------------------------------------------------------- Appendix III:4

    The University of Alaska met most of its project objectives while
    encouraging systemic reform, but to date the project has not become
    self- sustaining.  The American staff live in Russia and have built
    trust with Russian officials and institutions, and Russians support
    the project with personnel and in-kind contributions.

          SECTOR PROBLEM
    ----------------------------------------------------- Appendix III:4.1

    New enterprises are a major source of new jobs for most economies.
    However, the development of new enterprises in Russia has been
    hampered by years of central control of the economy, excessive rules
    and procedures for establishing a business, and the lack of
    entrepreneurial skills.

          PROJECT OBJECTIVE
    ----------------------------------------------------- Appendix III:4.2

    To help promote the growth of small, private businesses and alleviate
    unemployment, the United States supported the creation of
    multipurpose business development centers in several Russian cities.
    These centers provided training and advisory services to small
    businesses and worked with local governments to create a hospitable
    environment for private business growth.  USAID's goal is that the
    centers eventually be operated by trained Russians on a
    self-financing, fee-for-service basis.

    The American Russian Center (ARC), established by the University of
    Alaska in Anchorage through an USAID cooperative agreement, was one
    of the first contractors in USAID's program to establish new
    businesses.  The program's two phases, conducted over 2 years, cost
    about $5.1 million.  The agreement called for ARC to provide small
    business training, develop Russian business activities in specific
    geographic areas, and develop business ties between the Russian Far
    East and the United States.

    ARC's initial objectives were to establish a headquarters at the
    University of Alaska and two field centers, as well as to train a
    specific number of people.  A subsequent work plan called for ARC to
    establish two more centers while expanding its program in the two
    original centers.  Specific objectives included increasing (1) the
    number of Russians trained in modern business methods, (2) the number
    of viable Russian small businesses, (3) access to both U.S.  and
    Russian technologies, (4) U.S.-Russian business ties through ARC
    field and business information centers, and (5) U.S.  business
    activity in the Russian Far East.  Creating Russian institutions that
    would be sustainable after USAID assistance ended was also an
    objective.

          PROJECT APPROACH
    ----------------------------------------------------- Appendix III:4.3

    From its headquarters at the University of Alaska, ARC worked closely
    with Russian partners to establish business training centers in four
    Russian cities.  In each city, ARC had a local educational or
    academic institution as a partner.  This partnership was reflected in
    the American and Russian codirectors and staff at each center and
    in-kind contributions such as free office space from the
    institutions.

    ARC's American staff have had a long-term commitment to Russia.
    Full-time staff spoke Russian fluently, lived in the cities where the
    centers were located, and had business experience in the region.
    They were complemented by short-term American teachers who taught a
    1- or 2-month course as well as by itinerant teachers who taught a
    1-week or weekend course in one city and then moved to another city.
    These courses were taught with interpreters.

    ARC's core program was an evening course that taught such skills as
    accounting, marketing, and management that were necessary to write a
    business plan.  This course lasted 1 or 2 months, depending on the
    center, the time of year, and the targeted clientele.  It was
    supplemented by short seminars in the host cities and extension
    seminars in outlying cities and was targeted at specific business
    sectors, such as bankers lending to small businesses.  Russians and
    Americans, both resident and visiting, taught the courses and
    seminars.  The centers also provided business counseling for Russians
    trying to set up their own small businesses.  The training centers
    charged a relatively low fee for its courses and seminars.

    Participants who excelled in the training center programs were
    invited for advanced training in Anchorage.  They were selected, in
    large part, through the business plans they wrote during their core
    course.  In Anchorage, they attended a 5-week course that explored
    topics from the earlier training in more depth, and toured stores,
    manufacturing facilities, and offices in the Anchorage area.  The
    5-week course was followed by 2 weeks of either internships in local
    small businesses or more extensive business tours tailored to the
    participants' interests.

          CONTRACTOR PERFORMANCE
    ----------------------------------------------------- Appendix III:4.4

    ARC successfully fulfilled its first year's work plan targets, and
    then received $3 million for a second year's activities (fiscal year
    1994) from USAID after a March 1994 evaluation of the initial
    $2.1-million project.  USAID also stipulated that, in fiscal year
    1995, ARC must match USAID's funding.  ARC established business
    training centers in four Russian cities:  Yuzhno-Sakhalinsk, Yakutsk,
    Khabarovsk, and Magadan.  It established the Yuzhno-Sakhalinsk and
    Yakutsk centers in the fall of 1993 and Khabarovsk and Magadan
    centers in the fall of 1994.  In May 1995, USAID agreed to provide
    ARC with an additional $1.5 million, even though ARC had not raised
    any matching funds.

    Between the fall of 1993 and January 1995, ARC's Yuzhno-Sakhalinsk
    and Yakutsk Business Training Centers offered four cycles of evening
    courses, lasting 1 or 2 months, that trained 211 Russians--thereby
    exceeding the first year's work plan goal of 200 participants.  The
    two centers also provided individual business counseling to 300
    Russians; the work plan's goal was 200.  In addition, the two centers
    offered 7 extension seminars to 103 Russians in outlying cities.  The
    training centers in Khabarovsk and Magadan had only recently
    completed their first evening courses.

    ARC sponsored 19 technical assistance seminars, meeting the first
    year's work plan goal of 15 to 20 seminars.  Four seminars on banking
    drew
    180 Russian bankers, and 8 seminars on hair salon management drew
    250 women from throughout the Russian Far East.  Forty construction
    managers from Yakutsk participated in training on cold weather
    construction methods.  This seminar led to the government of Yakutsk
    testing American-manufactured plastic piping to replace its existing
    steel piping.

    Between the fall of 1993 and January 1995, 71 Russians completed the
    advanced business training courses at the University of Alaska.  This
    exceeded the first year's work plan goal of 50.  In total, ARC
    trained 1,646 Russians in its USAID-financed programs through January
    1995.\11

    On a more systemic level, ARC developed a database of U.S.  and
    Russian businesses in the Russian Far East and provided assistance or
    information to U.S.  and Russian businesses working throughout the
    region.

    ARC generally coordinated its activities with other U.S.  government
    programs located in cities of the Russian Far East, but there were a
    few exceptions, particularly when contractors worked in separate
    sectors.  For example, in Khabarovsk, where ARC established a center
    in late 1994, the local American codirector did not know the local
    environment project director until we visited.  ARC officials in
    Anchorage were, however, working with CH2M Hill staff to link the
    projects.

    --------------------
    \11 ARC also has a U.S.  Information Agency grant that it uses to
    sponsor exchanges.

          RESULTS
    ----------------------------------------------------- Appendix III:4.5

    The ARC project will contribute to systemic reform on a regional
    basis if it can become financially self-supporting.  USAID recognizes
    that creating small businesses in the region will push the Russian
    government to be more responsive and further develop the area's
    nascent capitalist economy.

    The centers help Russians who come to Anchorage from their relatively
    isolated cities\12 to meet each other and develop business contacts
    with other Russians as well as Americans.  By drawing entrepreneurs
    from cities throughout the Russian Far East, ARC has helped build a
    network of private small businesses that can generate business for
    one another and for the region.

    Once the USAID funding ends, ARC's partnerships with Russian
    institutions will be the key ingredient to sustaining its work.  The
    directors of the Russian institutions plan to continue the program.
    For example, the Rector of the Far Eastern State Transport Academy,
    ARC's partner in Khabarovsk, plans to establish a permanent school
    based on the activities of the project.  Kray and oblast' government
    officials are also highly supportive of ARC.  However, despite the
    Russians' desire to continue the program, most of these institutions
    currently lack the means to support an entire local ARC operation.

    Further, in an April 1994 evaluation report, USAID questioned whether
    ARC would be able to support itself.  Other donors have not yet
    stepped forward to replace USAID.  According to ARC's director, the
    problem lies in the newness of the project and the project's focus on
    Alaska and its businesses.  The ARC director said the project plans
    to include more business internships and tours in the rest of the
    United States.  The director believes that this expanded scope will
    increase ARC funding because large U.S.  institutions and enterprises
    have the funds and business interests in Russia to provide long-term
    support.  If USAID assistance had ended with fiscal year 1994 funds,
    the U.S.  side of ARC would have been curtailed and U.S.  personnel
    in Russia would have been withdrawn.

    --------------------
    \12 Except for Khabarovsk, each of the cities was closed to
    foreigners before the collapse of the Soviet Union.

          USAID MANAGEMENT
    ----------------------------------------------------- Appendix III:4.6

    USAID officials played a significant role in designing the project.
    Because the University of Alaska had no previous USAID contract
    experience, USAID sent an official from Moscow to Anchorage to help
    with the project's design, which has proven to be effective.

    Under a cooperative agreement, USAID has relatively limited
    management and monitoring responsibility.  ARC provided good progress
    reports to USAID.  USAID/Moscow has adequately managed and monitored
    the project.  USAID/Washington has maintained a duplicate document
    set so that it can respond to U.S.  inquiries.

    When it was considering further funding for ARC, USAID sent an
    evaluation team from Moscow to Yakutsk, Yuzhno-Sakhalinsk,
    Khabarovsk, and Anchorage in March 1994.  The team's report became
    the basis for USAID's continued funding of ARC.  Within the amendment
    that provided the fiscal year 1994 funding, USAID included a clause
    stating that USAID would provide $1.5 million in fiscal year 1995 if
    ARC raised $1.5 million in matching funds.  However, in May 1995,
    USAID agreed to provide the $1.5 million even though ARC had not
    raised the matching funds.

       ENVIRONMENTAL POLICY AND
       TECHNOLOGY--
       CH2M HILL
    ------------------------------------------------------- Appendix III:5

    CH2M Hill is an integral part of USAID's $35-million environmental
    policy and technology program.  In September 1993, USAID awarded CH2M
    Hill a contract to serve as the program's core contractor and provide
    the technical support for its environmental activities in Russia,
    Kazakstan, and Ukraine.  In April 1994, an initial delivery order was
    signed to provide support for activities in Novokuznetsk and the
    Russian Far East.  Detailed delivery orders were signed for these
    activities in September 1994.  CH2M Hill also serves as the
    contractor or subcontractor on various components of the program.

    Although the environmental policy and technology project is still
    ongoing, USAID officials said its progress so far has been
    disappointing.  Progress has been slow because CH2M Hill did not fill
    critical staff positions in Russia in a timely manner, and it relied
    on staff located in the United States to manage the projects.  The
    expanded scope of the Far East component further contributed to the
    delay.  Further, USAID field staff lacked authority and information
    to expedite project implementation.  The projects' expected
    contributions to systemic reform and long-term benefits are not
    likely to be significant.

          SECTOR PROBLEM
    ----------------------------------------------------- Appendix III:5.1

    Severe environmental degradation threatens the physical health and
    socioeconomic well-being of people throughout Russia and deters
    economic and political restructuring efforts.  Environmental problems
    range from nuclear safety issues; to pervasive mismanagement of
    natural resources; to some of the worst air, water, and land
    pollution problems in the world.  The breadth and magnitude of the
    economic, health, and ecological costs are difficult to quantify,
    although remediation activities alone are expected to cost
    billions.\13

    Environmental problems are exacerbated by many factors, including
    inattention to environmental consequences, a lack of economic and
    political incentives to use resources efficiently, the inability of
    nongovernmental agencies to participate in environmental decision-
    making, and the inability of governmental institutions to effectively
    regulate state-owned monopolies and curb illegal economic activities.

    --------------------
    \13 This information was obtained primarily from USAID's February
    1993 Project Memorandum and the U.S.  Government's Environmental
    Strategy for Russia, prepared by USAID.

          PROJECT OBJECTIVES
    ----------------------------------------------------- Appendix III:5.2

    Our analysis focused on CH2M Hill's performance as the core
    contractor and two projects where it serves as the primary
    contractor--the Multiple Pollution Source Management project in
    Novokuznetsk and the Sustainable Natural Resources Management and
    Biodiversity Protection project in the Russian Far East.  Both
    projects are to run from September 1994 to September 1997.

    The objectives of the $7.4-million core contract are to coordinate
    all activities under the core contract, monitor and evaluate the
    activities and deliverables, and provide support functions as needed.

    The objectives of the Multiple Pollution Source Management project
    are to reduce pollution-related health risks and promote
    environmentally sustainable economic development; improve public
    health; reduce pollutant emissions from industries and cities; assist
    industries in restructuring in an environmentally sound and
    sustainable manner; and strengthen institutions and train individuals
    to continue improvements initiated during the project.  The
    $13.4-million delivery order included $6.3 million for the
    Novokuznetsk project along with two other projects.

    The Sustainable Natural Resources Management project was expanded
    from a narrowly focused $3-million, 3-year project focused on fire
    prevention and control to a $16.7-million, 5-year project focused on
    sustainable forest management and biodiversity protection.  This
    expansion responded to the Gore-Chernomyrdin Commission's
    recommendations.\14 Specific project objectives are to promote
    sustainable forest management in the Khabarovskiy and Primorskiy
    Territories and protect endangered species and critical habitats in
    the Khrebet Sikhote-Alin' mountain region.  To address these
    objectives, the contract specifies 25 tasks for CH2M Hill and
    multiple subcontractors.  USAID approved a $9.4-million delivery
    order for CH2M Hill to implement and coordinate these activities.

    --------------------
    \14 The Gore-Chernomyrdin Commission was created in 1993 to overcome
    trade barriers in the energy sector, but now includes business
    development, space, environment, science and technology, health, and
    defense diversification.

          PROJECT APPROACH
    ----------------------------------------------------- Appendix III:5.3

    CH2M Hill worked mainly with local and oblast' government officials
    to design and manage the programs.  CH2M Hill consultants spent short
    periods of time in Russia to design the project proposals and then
    returned to the United States to complete the project design.
    Although USAID and CH2M Hill established rapport with local and
    oblast' authorities in the affected cities, the Ministry of
    Environmental Protection and Natural Resources was only involved in
    the initial selection of project activities and their locations.
    Subcontractors, U.S.  nongovernmental organizations, and other
    federal agencies helped implement parts of the project.

    The project approach includes providing technical assistance,
    demonstration projects, training seminars, and limited commodities.
    Several components in both projects continued efforts initiated by
    the U.S.  Environmental Protection Agency, World Bank, and the City
    of Pittsburgh.

    CH2M Hill staff in Washington manage the project, and a regional
    director and site managers in Russia handle the day-to-day activities
    and coordinate with other implementers.  CH2M Hill plans to hire and
    train Russian employees who can eventually manage the activities
    without assistance from its U.S.  office.

          CONTRACTOR PERFORMANCE
    ----------------------------------------------------- Appendix III:5.4

    Project progress to date has been mixed.  CH2M Hill met the
    requirements of its core contract by establishing field offices,
    monitoring project implementation, and providing support functions to
    its field staff.  Even though it has made some progress toward
    addressing the Novokuznetsk project objectives, it has been slow to
    implement the Far East project.  CH2M Hill has missed critical
    milestones for both projects.

    In Novokuznetsk, CH2M Hill established an air pollution database for
    the 180 heating plants in the city and developed a strategic plan to
    address particulate pollution from the heating plants.  It also
    upgraded the city's air pollution program, trained Russian
    counterparts in environmental auditing, and completed environmental
    audits of two large steel mills.  CH2M Hill is currently assessing
    local water monitoring activities and has recommended laboratory
    improvements to more accurately measure the quality of drinking
    water.  CH2M Hill is working with the Novokuznetsk Development Fund
    and local government officials to develop a strategic plan.  However,
    CH2M Hill has not provided an acceptable work plan, which was due on
    November 30, 1994.  The current work is based on the delivery order
    specifications.

    In the Far East project, CH2M Hill has been even slower getting
    started and, according to USAID and Russian officials, had produced
    almost no quantifiable results as of February 1995.  Several factors
    have hindered the project's implementation, including its increased
    complexity; the size of the geographic area; and the large numbers of
    governmental officials, local interest groups, and subcontractors
    involved.  The project covers 2 large regions and will involve at
    least 16 implementing organizations, including 2 U.S.  federal
    agencies, subcontractors, and U.S.  nongovernmental organizations.

    Several problems have delayed the effective implementation of both
    CH2M Hill projects.  One problem was that CH2M Hill experienced
    problems filling critical staffing positions in Washington and Moscow
    and at the field office level.  Although the core contract was
    awarded in September 1993, the regional director did not arrive in
    Moscow until February 1994.  Other positions in Moscow funded in
    September 1994 delivery orders were still being filled as of January
    1995.  The contract to implement field support functions in
    Novokuznetsk and the Far East was awarded in April 1994, but on-site
    managers did not arrive until September and October 1994,
    respectively.  The Far East project manager position was authorized
    in September 1994, but the manager did not move to Russia until
    February 1995.  CH2M Hill officials had difficulty finding qualified
    staff who were willing to relocate to these areas because of the
    acute environmental problems and remote locations.  USAID and CH2M
    Hill officials agreed that the on-site presence is essential for
    making progress.  USAID/Moscow officials said staffing delays and
    delays in producing an acceptable work plan have hurt the credibility
    of the program in the region.

    CH2M Hill also had difficulty developing acceptable work plans that
    define how and when the scope of work will be implemented.  CH2M Hill
    was required to submit the work plans for both the Novokuznetsk and
    the Far East projects within 60 days after signing the contract on
    September 30, 1994.  USAID approved the work plan for the Far East
    project on May 8, 1995, but the work plan for the Novokuznetsk
    project was still being revised as of June 1, 1995.  According to an
    USAID official, the work plans originally submitted were incomplete
    and lacked specific indicators or other factors necessary to evaluate
    the activities.

    Additionally, USAID officials said CH2M Hill had done a poor job of
    providing them with the appropriate reporting documents for these
    activities.  USAID expressed concern over CH2M Hill's failure to
    provide timely delivery of tracking materials, such as monthly
    summaries of financial status by project, monthly presentations of
    progress on select tasks, and weekly briefings on overall project
    progress.  According to USAID officials, CH2M Hill addressed their
    concerns and has recently improved its reporting.

          RESULTS
    ----------------------------------------------------- Appendix III:5.5

    As of February 1995, the CH2M Hill projects had contributed little to
    systemic reforms, and they will not generally be sustainable without
    outside funding support.  This limited contribution is due largely to
    the vast environmental needs in Russia and the massive amounts of
    capital investment needed to modernize or purchase equipment for
    restructuring Russia's environmental sector.  Also, USAID and CH2M
    Hill officials said that Russian monitoring and enforcement
    procedures will be extremely difficult to change and are not
    addressed in these projects.  Finally, the Ministry of Environmental
    Protection and Natural Resources was not involved in designing the
    project, thus reducing the likelihood the project could be duplicated
    on a wider scale.  USAID officials said the project will attempt to
    address systemic reform through efforts to maintain and restock the
    forestry base.

    Some components of the Novokuznetsk project are likely to address
    environmental sector restructuring.  CH2M Hill expects to work with
    Novokuznetsk's industry, citizens, and local government to develop a
    strategic plan and provide recommendations for creating an
    environmentally safe city by 2010.  However, these recommendations
    could require large capital investments.  For example, CH2M Hill
    recently conducted industrial audits for two steel companies.  After
    spending
    6 weeks and using 7 U.S.  advisers and 25 Russian counterparts to
    conduct the audits, company officials said the audits did not provide
    any new information on major pollution sources.  Further, the
    companies do not have the funding to make the recommended
    improvements and will have difficulty obtaining it.  According to one
    steel mill executive, the environmental audit allowed the mill to
    fulfill a condition for a World Bank loan.  The Novokuznetsk project
    places a considerable emphasis on the contractor delivering studies
    and does not establish any indicators to measure progress in reducing
    actual pollution.

    Some components of the Far East project are designed to address the
    region's need to maintain and restock its important forestry base.
    Efforts are planned to (1) strengthen polices and develop an adequate
    environmental regulatory structure, (2) create economic and political
    incentives to use resources efficiently, (3) increase the
    participation of nongovernmental agencies in environmental
    decision-making, (4) promote U.S.-Russian partnerships, (5) promote
    the export of timber products made by Russian workers, and (6)
    conserve biodiversity.

          USAID MANAGEMENT
    ----------------------------------------------------- Appendix III:5.6

    USAID's decision to use a core contract and delivery orders has
    caused delays and excessive paperwork reviews for both CH2M Hill and
    USAID staff.  Under this system, USAID must prepare delivery orders
    and CH2M Hill must submit detailed work plans for each project
    component within
    60 days.  The decision to expand the Far East program has also
    delayed project design and implementation.  The expansion covers a
    larger geographic region and greatly increased the scope of work,
    including the number of activities and subcontractors involved.

    The division of responsibility between USAID/Washington and
    USAID/Moscow has affected the agency's ability to manage the project.
    USAID/Washington maintains overall management authority, but has
    given USAID/Moscow increased monitoring and program responsibility.
    However, USAID/Moscow officials said they still had minimal authority
    to manage the project or make changes.  USAID/Washington must approve
    all program decisions, including minor ones, such as country
    clearances for visitors and all purchases exceeding $500.  In April
    1995, USAID/Moscow submitted an initial request, which remained under
    review as of June 1, 1995, for delegation of authority to the field.

    USAID has had difficulty monitoring the projects.  USAID staff said
    they have not regularly visited the project sites because of the
    difficulty of traveling to the sites and the lack of adequate staff.
    The USAID/Moscow project officer keeps apprised of the project
    activities primarily by talking to project staff over the telephone
    or in informal meetings and by reviewing reports by the contractor or
    visiting teams.

       DISTRICT HEATING--
       RCG/HAGGLER BAILLY
    ------------------------------------------------------- Appendix III:6

    The district heating project is one component of USAID's Energy
    Efficiency and Market Reform Project for the NIS.  The project began
    in January 1992 and is considered the first economic development
    effort undertaken by the United States in the region.  With $5.3
    million in funding, the project was designed to improve district
    heating systems in six countries:  Armenia, Belarus, Kazakstan,
    Kyrgyzstan, Russia, and Ukraine.

    Although the contractor, RCG/Haggler Bailly (RCG/HB),\15 met most of
    its objectives, we found no indication that the project was having a
    significant impact on the sector.  Most of the Russian work was
    concentrated in two Russian cities, Yekaterinburg and Kostroma, and
    the project was not completed in Yekaterinburg.  Because USAID did
    not adequately monitor the project, it was unaware of (1) problems
    that prevented the completion of the project and (2) any long-term
    benefits, if any, to the beneficiaries.  An evaluation conducted by a
    consultant did not identify obvious problems, and USAID did not
    address the recommendations in this evaluation.

    --------------------
    \15 RCG/Haggler Bailly, one of three contractors implementing the
    district heating project in the NIS, is an international management
    consulting firm that specializes in energy and the environment.

          SECTOR PROBLEM
    ----------------------------------------------------- Appendix III:6.1

    Fuel and energy are an important part of Russia's economy.  The
    subsidies provided by the former Soviet government to Russian energy
    consumers, both residential and industrial, created artificially low
    prices and promoted the inefficient use of highly polluting energy.
    Since the dissolution of the Soviet Union, Russia has implemented
    several policies, including increasing or freeing coal, oil, and gas
    prices, to reform its energy sector.  Although still below world
    market levels, the cost of domestic oil and oil products in Russia
    doubled in 1993 and 1994.  Such increases in energy prices have a
    significant influence on inflation and social conditions.  As energy
    prices increase, consumers must find ways to use energy more
    efficiently.  \16

    --------------------
    \16 The information in this section was obtained primarily from
    papers presented at the International Energy Agency's October 1994
    Symposium on Russia's Energy Strategy and the Agency's report,
    Russian Energy Prices, Taxes, and Costs, 1993.

          PROJECT OBJECTIVES
    ----------------------------------------------------- Appendix III:6.2

    In February 1992, RCG/HB was awarded a contract for $550,000 to
    complete the project in Russia.  The project was amended in August
    1992, increasing the total cost to $1.3 million.  The project had
    five objectives:  (1) foster improved management of energy use in
    heating plants by identifying and implementing cost-effective "low
    cost-no cost" energy efficiency improvements; (2) transfer energy
    auditing and management techniques, including financial and economic
    analysis techniques; (3) provide equipment support to implement
    low-cost options, improve monitoring and energy management, and
    identify additional energy efficiency opportunities; (4) support the
    World Bank's efforts to reform Russia's energy pricing policies; and
    (5) promote the emergence of an energy efficiency industry in Russia.

          PROJECT APPROACH
    ----------------------------------------------------- Appendix III:6.3

    RCG/HB and USAID worked with representatives from the Russian
    Ministry of Fuel and Power, the Commission for Humanitarian and
    Technical Assistance of the Russian Federation, nongovernmental
    organizations concerned with energy efficiency and conservation,
    municipal governments, and industrial enterprises.

    The two primary Russian cities selected for the project were
    Yekaterinburg and Kostroma.  In these cities, extensive energy audits
    were conducted of the district heating facilities, and three sites
    (i.e., hospitals, apartment buildings, and heating plants) in each
    city were selected as demonstration sites for U.S.  energy efficiency
    equipment, including flow meters, temperature sensors, and
    thermostatic control valves.  The value of the equipment supplied to
    the demonstration sites was approximately $172,000.  The project
    sites were intended to demonstrate the savings in using no-cost or
    low-cost technologies and also serve to promote American-made
    equipment.

    In addition, RCG/HB conducted energy audit training seminars and
    provided energy audit equipment to technicians in Yekaterinburg,
    Kostroma, Irkutsk, Moscow, Murmansk, and St.  Petersburg.  To
    complete its work, RCG/HB contacted more than 250 U.S.  equipment
    manufacturers to determine their interest in conducting business in
    Russia.  The 12 companies that responded participated (at their own
    expense) in "wrap-up" seminars in four Russian cities when the
    project ended.  The information obtained at these seminars was
    published in a lessons learned document.

          CONTRACTOR PERFORMANCE
    ----------------------------------------------------- Appendix III:6.4

    RCG/HB completed most of the objectives stipulated in its contract.
    The products delivered to complete the objectives included energy
    audits in two cities, energy audit training and distribution of
    energy audit equipment, a study of natural gas pricing in Russia, and
    an energy efficiency industry development effort.  It also produced a
    video about the project that was shown on Russian television.

    RCG/HB was also required to identify, purchase, and install low-cost
    energy efficiency equipment manufactured by U.S.  companies.  RCG/HB
    purchased this equipment; however, due to problems with local
    conditions, some of the equipment was not installed in Yekaterinburg.
    An RCG/HB official said that in June 1993, a Russian subcontractor
    assured RCG/HB that it would install the equipment in Yekaterinburg
    by the end of 1993.  We visited three sites in Yekaterinburg in
    February 1995 and found all the equipment at one site was still in
    shipping containers.  Russian officials said the equipment at the
    other two sites only began operating in September 1994 and January
    1995, respectively.  According to an RCG/HB official, the company had
    not paid the subcontractor and would not pay until the installation
    was completed.  However, USAID had already paid for the equipment,
    valued at $8,000.

    Officials in Yekaterinburg stated that the equipment had not been
    installed in 1993 for two reasons.  First, in two cases, the sites (a
    hospital and an apartment building) were under construction and the
    construction plans had to be altered to accommodate the equipment.
    Second, at the other installation site (a district heating facility),
    the equipment had not been installed, and most likely will not be
    installed, because the proper Russian authorities had not certified
    it.  Officials in Yekaterinburg stated that it would be illegal to
    install and operate the equipment before it was certified.  They
    explained that although the equipment can be used for demonstration
    purposes at consumer locations (e.g., apartment buildings), a public
    utility cannot use the equipment and the information (e.g., energy
    consumption data) it produces as a basis for charging customers.

    Similar equipment was installed in Kostroma, according to USAID and
    RCG/HB officials, even though it had not been certified.  USAID
    officials told us that city officials were willing to install the
    equipment because they realized the potential benefits.

          RESULTS
    ----------------------------------------------------- Appendix III:6.5

    We found no indication that the project had contributed to systemic
    reform in the area of energy efficiency.  Most of the work was
    concentrated in two cities, and the project was not completed in
    either city.  In addition, USAID did not adequately monitor the
    project and could not be certain of any long-term benefits.

          USAID MANAGEMENT
    ----------------------------------------------------- Appendix III:6.6

    USAID used an independent consultant, Management Systems
    International, to evaluate the NIS district heating project,
    including RCG/HB's work in Russia.\17 The evaluation, published in
    July 1993, reported no serious problems and declared the project a
    success.  Specifically, the study indicated that total equipment
    costs for the four cities in Russia amounted to $418,000 and would
    produce an annual savings of $1.4 million.  It also noted that the
    equipment would reduce pollution.  Furthermore, as a result of the
    energy efficiency industry development effort, 12 U.S.  companies had
    sent representatives to the various countries to participate in
    seminars held at the end of the project.

    We found that the consultant's evaluation was deficient.  The
    evaluation did not mention the equipment installation problems in
    Yekaterinburg or the need to have foreign equipment certified by the
    Russian government.  Instead, the evaluation stated that "by April
    1993, all of the equipment was installed and operating." In addition,
    USAID did not specifically direct Management Systems International to
    assess the products RCG/HB was required to produce, such as the
    natural gas pricing study for Russia or the lessons learned from the
    energy efficiency industry development effort.  The evaluation did
    not discuss the quality of either of these products.  USAID officials
    stated that the natural gas pricing study had been completed in a
    collaborative effort with the World Bank, which used it in its work
    pertaining to loans made to Russia's natural gas sector.  However,
    the consultant's report contradicted USAID's statement by noting that
    the World Bank did not make a serious attempt to involve RCG/HB in
    its work in Russia.

    Finally, the evaluation did not discuss the training seminars
    conducted by RCG/HB in Irkutsk, Moscow, Murmansk, and St.
    Petersburg, or the energy audit kit instrumentation supplied to
    technicians in these cities.  The continued use of these deliverables
    is an important factor to consider when evaluating the success and
    sustained benefits of this project.

    USAID officials were not aware of the problems we identified in
    Yekaterinburg nor the shortcomings of the evaluation.  They stated
    that in June 1993, an official from USAID/Washington visited all the
    NIS sites except Yekaterinburg.  They stated that equipment had been
    installed at the sites visited.  A local national employee from the
    USAID mission in Moscow also visited Yekaterinburg in June 1993 but
    did not report any problems at that site.  USAID officials discovered
    the problems we found when they accompanied us to Yekaterinburg.
    USAID said it would take corrective action.

    Also, USAID has no mechanism to monitor various outcomes of the
    project, including (1) the success of U.S.  industry in entering the
    NIS market, (2) policy reforms written into law, and (3) the rate of
    adoption of new technologies.  Although USAID said that the installed
    equipment would produce annual savings at project sites, it did not
    record these savings during the 1993 or 1994 heating seasons.
    Furthermore, USAID had not determined the savings generated by either
    the energy audit kits provided to technicians in six cities or the
    energy audits conducted in Yekaterinburg or Kostroma.

    --------------------
    \17 USAID paid $18,385 for this evaluation.

       HEALTH CARE TRAINING--PARTNERS
       FOR INTERNATIONAL EDUCATION AND
       TRAINING
    ------------------------------------------------------- Appendix III:7

    USAID initiated the NIS Exchange and Training program in the spring
    of 1993 to train NIS leaders about free-market economies and
    democratic governance.  USAID hoped that training the participants in
    the United States would provide them with the technical skills and
    attitudes required to create similar policies, programs, and
    institutions in their own nations.  We reviewed the health care
    training provided to Russians in late 1993.

    Our analysis indicated that the health care training had little
    likelihood of contributing to systemic reform and that USAID now
    considers the training to be irrelevant after Russia changed its
    direction for health care reform.  The training's primary
    objective--to facilitate Russia's transformation to a democratic
    free-market system--was unrealistic for a 2-week training course.
    USAID did not follow up with participants to determine the training's
    impact on systemic reforms.  Although USAID officials said that most
    participants have been involved in follow-on projects, only 25
    percent are slated for follow-on activities planned in 1995 and 1996.

          SECTOR PROBLEMS
    ----------------------------------------------------- Appendix III:7.1

    According to USAID, Russia's health care industry has a number of
    problems.  These problems include the virtual collapse of the
    pharmaceutical and medical supply industry, poor quality of care due
    to training and technical gaps, serious funding shortfalls, and a
    centralized system devoid of incentives for efficiency and cost
    control.  Although Russian policies have produced an educated
    workforce with more doctors per capita than the United States, the
    workforce lacks many of the basic skills and institutions necessary
    to function in a democratic, free-market context.

          PROJECT OBJECTIVES
    ----------------------------------------------------- Appendix III:7.2

    USAID contracted in June 1993 with its worldwide training support
    contractor, Partners for International Education and Training (PIET),
    to conduct training in the United States for 200 NIS leaders and
    professionals at a cost of $2.6 million.  The training objectives
    were to

      facilitate the region's rapid and sustainable transformation from
         authoritarian, centrally controlled regimes to pluralistic,
         democratic countries with free-market economies;

      provide participants with new skills and knowledge to contribute to
         economic and social development;

      promote the value of democratic decision-making;

      provide an understanding of U.S.  programs; and

      lead to long-term relationships with U.S.  institutions.

    USAID also hoped that participants would share their new skills and
    perceptions with their counterparts.

          PROJECT APPROACH
    ----------------------------------------------------- Appendix III:7.3

    Under the PIET contract, USAID missions identified the training
    topics and selected the participants.  USAID/Moscow selected the
    participants based on their positions in oblast' health care systems
    and their planned inclusion in follow-on projects.  According to
    USAID, participants went to the United States before participating in
    follow-on projects so they would be more receptive to reforms.  The
    training project encouraged missions to link training, if
    appropriate, to ongoing or planned developmental assistance by USAID
    and others.

    After course topics and participants were selected, PIET was expected
    to arrange training courses in the United States and provide
    administrative and logistical support for international travel,
    living expenses, medical insurance, tuition, books, and other needs.
    PIET was also expected to (1) ensure that training programs at U.S.
    training institutions were functioning properly, (2) monitor the
    participants' progress, (3) provide USAID status reports, and (4)
    evaluate each training program.

          CONTRACTOR PERFORMANCE
    ----------------------------------------------------- Appendix III:7.4

    PIET subcontracted with Management Sciences for Health to provide the
    training in the United States; 42 Russians were trained in health
    finance and 20 were trained in pharmaceutical management.  USAID
    subsequently contracted with the Academy for Educational Development
    to make training arrangements.

    PIET met its contractual requirements by providing training,
    transportation, and logistics, according to USAID officials.  The
    participants we spoke with in Russia praised PIET's support and
    assistance as well as the quality of the training they received in
    the United States.  Our review of sample course assessments showed
    that other participants generally gave high marks to the training.
    For example, in the evaluation conducted by the USAID mission, most
    participants were satisfied with the course and believed it was
    applicable to their work conditions.

    PIET also met its monitoring and reporting requirements.  PIET
    maintained contact with the training institutions, called a random
    sample of participants once a week, contacted the trainers on an as
    needed basis, and helped participants with general adjustment
    problems.  PIET also provided all the required reports, including
    regular status reports and course assessments.  USAID/Washington
    officials were satisfied with the quality of PIET's support and
    monitoring during the training.

          RESULTS
    ----------------------------------------------------- Appendix III:7.5

    USAID was unable to provide any evidence that the training will help
    Russia's democratic or economic transformation.  Although the
    training may have met some secondary goals, without follow-on
    activities, fulfilling these objectives will not likely result in
    systemic reform.  USAID/Washington officials agreed that the training
    could not meet all of the contract's objectives.  They said that a
    2-week training course could only "facilitate" reaching these
    objectives but not actually attain them.  Further, they provided the
    training quickly as a political imperative to respond to the opening
    in the NIS, and they recognized training alone has limited
    usefulness.

    USAID was unable to substantiate that any of the 62 participants
    contributed to any reforms, partly because the participants lacked
    the authority, expertise, or resources to influence reforms.
    However, the participants who had taken PIET health-related courses
    said the training helped them understand U.S.  programs and they had
    shared their training with others.

    USAID officials in Moscow and Washington said that training alone
    would not influence systemic change and that subsequent training was
    better integrated into follow-on activities.  They said that the main
    purpose of the PIET training was to make participants receptive to
    follow-on reform projects, which USAID thought would occur.  However,
    USAID later dropped plans for follow-on activities in Central Russia
    because the oblasts were not reform-minded and the contractor
    reported that only 25 percent of the Siberian participants would
    participate in follow-on activities.

    The Russians did not see health reform as a priority when the early
    training took place, and Russia has only recently begun to consider
    the direction of reforms, according to USAID/Washington officials.
    Further, this early training is now irrelevant because it was based
    on Russian policy directions that were later discarded as unworkable.

          USAID MANAGEMENT
    ----------------------------------------------------- Appendix III:7.6

    The mission was forced to move much more quickly than it desired
    because it was under extreme congressional pressure to quickly
    establish the training program, according to USAID officials.  As a
    result, the health care training was initiated before it could be
    integrated into follow-on projects more likely to facilitate systemic
    change.  Further, because the Russians were unclear about what
    reforms they wanted, the mission had trouble targeting the training.
    The Russian officials began exploring reform options with the mission
    in December 1994.

    USAID/Moscow officials assessed the training after participants
    returned to Moscow; however, they have had no contact with the
    participants since then.  They did not know which participants, if
    any, would be involved in any of the follow-on activities planned in
    Siberia.

       COMMERCIAL REAL
       ESTATE--INTERNATIONAL BUSINESS
       & TECHNICAL CONSULTANTS, INC.
    ------------------------------------------------------- Appendix III:8

    The International Business & Technical Consultants, Inc.  (IBTCI)
    project did not achieve its goal to increase the availability of
    commercial real estate to small enterprises, although it did provide
    potentially useful technical assistance in three cities.  The project
    did not contribute to systemic reform and was not sustainable.  IBTCI
    did not replicate the pilot project--a project objective--in part
    because the roll-out cities were poorly chosen.  IBTCI was
    responsible for choosing appropriate cities, but its short-term
    consultants lacked sufficient knowledge of Russia and local
    conditions to determine what cities would have cooperative officials
    and could benefit from the project.

          SECTOR PROBLEM
    ----------------------------------------------------- Appendix III:8.1

    Much of Russia's commercial real estate is still owned by the
    government.  Rather than divesting its ownership rights, the central
    government has decentralized those rights to local governments, both
    regional and municipal.  Although this practice is quite common among
    other countries in transition, Russia is different because local
    governments (1) have a virtual monopoly on commercial real estate and
    (2) have not moved toward commercial real estate leasing using market
    mechanisms.  Highly inefficient users occupy valuable commercial
    space, contributing very little to local budgets, while private
    sector development is blocked by the unavailability of property.\18

    --------------------
    \18 Harding, April L., Commercial Real Estate Market Development, The
    World Bank, Discussion Paper Number 109 (May 10, 1995).

          PROJECT OBJECTIVES
    ----------------------------------------------------- Appendix III:8.2

    USAID and GKI recognized this problem and signed a task order with
    IBTCI to develop a solution.  The $2-million task order for the rapid
    diagnosis pilot project and roll-out project was part of IBTCI's
    $13.3-million omnibus contract.  The initial deadline of May 1994 was
    extended to December 1994, but without any increase in the cost or
    level of effort required.  The general purpose of the task order was
    to significantly increase the availability of commercial property in
    Russian cities.

    The specific goals of the task order were to examine the causes of
    limited access to retail space, implement a pilot project in a
    selected city, and then replicate the pilot project in other oblasts.
    IBTCI was to (1) deliver a report on the root causes of and solutions
    to the problem of commercial property access for one city; (2) design
    an implementation plan to address these issues, including procedural,
    legal, administrative, financial, policy, and other measures; (3)
    replicate the pilot project in at least five other oblasts; and (4)
    produce and nationally distribute publicity and instructional
    materials for local state property committees, local authorities, and
    entrepreneurs on how to increase the availability of retail property.

          PROJECT APPROACH
    ----------------------------------------------------- Appendix III:8.3

    IBTCI used a subcontractor, Boston Consulting Group (BCG), to perform
    the rapid diagnosis phase and conduct the pilot project in the City
    of Perm'.\19 The goals of the pilot project were to (1) design and
    test a method for increasing the amount of commercial real estate
    available to small and start-up businesses and (2) identify any
    constraints or impediments that might exist.  The pilot was intended
    to serve as a model for instituting the program in five other Russian
    cities.  IBTCI instituted the roll-out in Irkutsk, Tver', Novgorod,
    Yekaterinburg, and Vladivostok.

    Because it had previously worked in Perm', BCG used staff who already
    had a relationship with municipal officials when it began the
    diagnosis and pilot phases of the project.  In contrast, IBTCI relied
    on consultants who made short visits to the other cities to research,
    plan, and implement the roll-out.

    During the rapid diagnosis phase in Perm', BCG identified three
    feasible ways of improving access to commercial space:  convert
    residential premises to commercial use, develop a secondary real
    estate market, and optimize the leasing process.  BCG, IBTCI, USAID,
    and GKI selected the leasing optimization method because they thought
    some concrete results were possible during the study period, even
    though it was predicted that this alternative would have limited
    support and low potential impact.  Lease optimization means, among
    other things, (1) moving toward market-determined rents, (2) removing
    bureaucratic discretion in space allocation, and (3) creating
    incentives to sublease unused space.

    --------------------
    \19 Perm', with a population of 1.5 million, was selected for the
    pilot because of its intermediate size and presumed representative
    nature of municipalities within the Russian Federation.

          CONTRACTOR PERFORMANCE
    ----------------------------------------------------- Appendix III:8.4

    BCG conducted the rapid diagnosis and pilot phase of the project from
    November 1993 to March 1994 in Perm'.  BCG devised a two-track
    auction system for making municipality-controlled real estate
    available to private businesses.  The first track was an auction for
    the right to lease specific commercial real estate properties (i.e.,
    a one-time premium).  The second track was an auction for the rental
    rate at which a property would be leased.  The purpose of this system
    was to introduce market mechanisms into the allocation and pricing of
    commercial real estate.  Under this system, bidding for the right to
    lease and the rent to be paid replaced government bureaucrats with
    market mechanisms.

    The results of the first auction, which occurred on March 1, 1994,
    were not promising.  In the first track auction, three properties
    were available.  The right to lease them was sold for each property.
    In the second track auction, 15 properties, all basements, were
    available.  Bids were made on only 3 of the 15, and each received
    only one bid.  The rental rate for the three properties did not
    exceed the rent that started the bidding.

    The results of a second round of auctions, which occurred in May
    1994, were also disappointing.  In the first track auction, 10
    properties were available, but the right to lease was sold for only 4
    properties, although several parties bid on them.  In the second
    track auction, three properties were available, but only one received
    a bid, and that was the starting bid.  These results were not
    perceived to have significantly increased the amount of commercial
    real estate available in Perm'.

    IBTCI started work on the roll-out in mid-February, before the Perm'
    pilot was completed or its results evaluated.  IBTCI soon found that
    none of the five cities chosen for the roll-out had conditions that
    approximated, let alone duplicated, those in Perm'.  The roll-out
    cities seem to have been chosen more for their geographic and
    population distributions than for any existing economic, political,
    and regulatory conditions that might make the Perm' model
    replicable.\20 Because of these differences, IBTCI had to deviate
    from the Perm' model and basically develop five new pilot projects;
    nonetheless, it still experienced problems.

    Irkutsk officials were not cooperative with IBTCI and declared the
    information needed to assess the commercial real estate situation a
    state secret.  Local officials were not ready to participate in the
    project.

    In Tver', an auction system had been functioning between June 1992
    and December 1993.  The original investment tender process used in
    the auction was challenged in court and hopelessly compromised.
    IBTCI introduced a new tender process in Tver', but the new system's
    effectiveness has not yet been demonstrated.

    The Novgorod officials opposed conducting right-to-lease auctions
    because they feared losing future revenue and the city had
    experienced poor results from a similar auction in November 1993.
    IBTCI focused on establishing a market for municipal, oblast', and
    private commercial space by creating a real estate listing center,
    developing a secondary market, and encouraging officials to allow
    increased and legalized subletting.  The listing center's
    effectiveness has not yet been demonstrated.

    In Yekaterinburg, an effective auction system has been in place since
    1992.  City officials were not interested in IBTCI's original task of
    increasing the use of commercial leases.  Instead, they wanted
    assistance in how to use retail and commercial space efficiently and
    increase the city's revenues from property leases.  Although IBTCI
    submitted some analyses and recommendations addressing their
    concerns, city officials told us that IBTCI came to town on different
    occasions, spent little time there, did not speak to the appropriate
    local officials, and presented an academic report that was of little
    use to them.

    Vladivostok city officials were interested in privatizing commercial
    real estate, but were unable to devise a method that would use
    mortgages to provide substantial revenue for the city.  IBTCI devised
    a mortgage instrument that allowed the city to continue receiving
    income by holding the mortgages and allowed small business owners to
    bid for a property and provide as little as 5 percent of the final
    cost as a down payment.  The city auctioned one property in August
    1994 for under $7,000, but local officials doubted they would use an
    auction again because the city did not not have any more excess
    property.

    By the time IBTCI had completed its work in the five cities, none of
    the cities had participated in any activities that remotely resembled
    the Perm' model.  As a result, the objective of replicating the pilot
    project in other cities was not achieved.

    There were various reasons for IBTCI's inability to replicate the
    Perm' model.  First, tensions between IBTCI and BCG caused some
    problems.  BCG performed both the rapid diagnosis and the pilot
    phases, but IBTCI determined that BCG's approach was not adequate.
    Russian officials monitoring the project were aware of tensions
    between IBTCI and BCG early in the project, but IBTCI was obligated
    to fulfill the contract and replicate the model.  The tensions
    between IBTCI and BCG resulted in little continuity of personnel from
    the pilot to the roll-out phases.

    Second, a provision of Russia's 1994 State Privatization Program Act
    and its implementing regulations caused problems in the Perm' pilot.
    The provision gave lessees who obtained their leases competitively
    (i.e., at an auction) the right to buy the property at the end of the
    leases.  The implementing regulation set an extremely low selling
    price for such privatized properties.  IBTCI said that the act's
    provision and the implementing regulation stopped the Perm' model
    because city officials did not want to lose revenue from leases and
    did not want to be forced to sell leased property for extremely low
    prices.  Even though reports identified the problem as early as
    January 1994, USAID, GKI, and the Russian Privatization Center took
    no effective action to address the issue.

    Third, although officials at the federal level agreed earlier that
    the project should be done and that the Perm' model was viable, local
    officials in the roll-out cities did not agree with the Perm' model
    or its usefulness in their cities.

    Fourth, although the consultants used by IBTCI for this project had
    some experience in Russia and some spoke Russian, Russian authorities
    questioned the level of some IBTCI consultants' professional
    experience.  In addition, the consultants did not have enough
    knowledge of the Russian localities and local politics to choose
    roll-out cities well.  IBTCI staff did not reside in the cities
    during the roll-out.  Instead, they would fly in, do a few days work,
    then leave.  Thus, they were unable to identify what cities would be
    the best candidates for replicating the Perm' model.  Even BCG had
    problems carrying out a successful pilot, despite its knowledge of
    and relationships in Perm'.

    --------------------
    \20 Novgorod and Tver' were included because they are relatively
    small cities, under 250,000 in population.  Irkutsk and Vladivostok
    represented medium-sized cities, with populations between 250,000 and
    750,000.  Yekaterinburg was to represent large cities, population
    over 750,000, and be a match for Perm'.

          RESULTS
    ----------------------------------------------------- Appendix III:8.5

    The project was not sustainable and did not contribute to systemic
    reform.  Although IBTCI's final report provided solutions to specific
    problems, the project did not implement the pilot or develop a method
    that could be replicated in other cities.  USAID officials and the
    Russians who were in charge of disseminating the report did not know
    whether or where the IBTCI "solutions" had been applied in any but
    the six cities.  City officials we interviewed in Yekaterinburg and
    Vladivostok were not using the concepts of the project.

    GKI and the Russian Privatization Center had originally proposed the
    project, which suppported a federal initiative.  However, an existing
    GKI act and its implementing regulation potentially forced local
    governments to sell leased property at low prices to anyone who
    bought the lease at an auction.  This regulation contributed to the
    poor results of the project.

          USAID MANAGEMENT
    ----------------------------------------------------- Appendix III:8.6

    USAID managed this project--from Washington with limited help from
    USAID/Moscow-- jointly with GKI and the Russian Privatization Center;
    it relied on consultants from the Harvard Institute for International
    Development and the Center to help monitor the project.  Nonetheless,
    USAID did not monitor the project adequately.  Even though IBTCI
    filed the required reports, these reports failed to describe how much
    the roll-out deviated from the Perm' model.  Center officials said
    they first became aware that the pilot was not being implemented in
    other cities in late May 1994, long after the roll-out could be
    redirected to other cities.

    When problems were known in some cases, USAID did not take any
    corrective action.  For example, a Harvard consultant who visited
    some sites raised questions about the cities selected for the
    roll-out, but USAID took no corrective action.  Similarly, the
    problem with the State Privatization Program Act and its implementing
    regulations was mentioned in reports in January 1994, but no action
    was taken to resolve it.  Finally, USAID officials said they were
    aware of the tensions between BCG and IBTCI, but simply told IBTCI to
    work the problem out themselves.  USAID/Moscow officials said they
    did not have enough staff to intervene when problems arose, visit the
    project sites, and talk with beneficiaries about how the project was
    progressing.

       AGRIBUSINESS PARTNERSHIPS
       PROJECT--TRI VALLEY GROWERS
    ------------------------------------------------------- Appendix III:9

    The lack of quantifiable objectives or time frames in the Tri Valley
    Growers' (TVG)\21 project design makes it difficult to measure the
    project's success.  TVG helped to facilitate the work of two
    agribusiness partnerships in Russia; nevertheless, USAID concluded
    that TVG did not perform adequately.  It is too early to determine
    the long-term economic viability of the partnerships; however, the
    involvement of U.S.  companies increases the likelihood the
    partnerships will be maintained.  The partnerships will probably not
    have any measurable effect on Russia's agricultural sector because of
    their limited size and number.

    --------------------
    \21 Tri Valley Growers is an agribusiness association based in
    California.  Its members are involved in value-added fruit and
    vegetable preserving, processing, packaging, distribution, and
    marketing.

          SECTOR PROBLEM
    ----------------------------------------------------- Appendix III:9.1

    Agriculture plays an important role in the Russian economy.  Although
    estimates vary, Russia has approximately 27,000 large state and
    collective farms, which cultivate approximately 90 percent of
    Russia's arable land.  Approximately 270,000 private farms cultivate
    5 percent of the arable land.  The remaining 5 percent is made up of
    private garden plots.  The total farming population comprises about
    26 percent of the country's population.  Subsidies and income
    transfers to the agricultural sector represent 25 percent of Russia's
    public expenditures.  Some of these subsides could be expected to be
    eliminated if the agricultural sector were privatized.\22

    Russian agriculture is a low-productivity sector.  For example, milk
    cows and potato and grain crops yield about half of western levels,
    and labor productivity is probably as low as one-tenth.  In addition
    to low productivity, Russia has been plagued by losses of up to 50
    percent in its storage and handling systems.  Finally, Russia's food
    processing system suffers from poor management and a lack of quality
    produce, additives, ingredients, and packaging materials.

    Although the Russian government has begun reforming the agricultural
    sector, the actual transformation of farms and agribusiness
    enterprises into market-oriented, productive entities is moving
    slowly.  In 1992, it reorganized state and collective farms and
    agricultural input and output distribution enterprises into joint
    stock companies.  However, most farms have not altered their
    operations to increase productivity and competitiveness.

    --------------------
    \22 This information was obtained primarily from the Rural
    Development Institute's report, Agrarian Reform In Russia (May 1993);
    a report produced by Chemonics International for USAID, Agriculture
    Post-Privatization In Russia (Jan.  1994); and USAID project
    documents.

          PROJECT OBJECTIVES
    ----------------------------------------------------- Appendix III:9.2

    In August 1992, USAID developed the agribusiness partnerships project
    as the cornerstone of its Food System Restructuring Project.  The
    agribusiness partnerships project was designed to create efficient
    systems for providing inputs to agriculture and for processing and
    distributing agricultural products.  USAID intended to catalyze NIS
    private sector activity by facilitating the involvement of private
    U.S.  agribusinesses.  Between January and May 1993, USAID signed
    cooperative agreements with three agribusiness cooperatives:
    Citizens Network for Foreign Affairs (CNFA), TVG, and Agricultural
    Cooperative Development International.  We reviewed USAID's
    cooperative agreement with TVG,\23 which had obligated $5.2 million
    for the region.\24

    To achieve the project's objective, TVG was to facilitate
    partnerships between American and NIS private agribusiness-related
    enterprises.  However, the agreement did not specify the number of
    partnerships or the related time frames.

    --------------------
    \23 At the time of our review, USAID's Inspector General was
    reviewing the cooperative agreement with CNFA.

    \24 The combined budget for the three cooperative agreements under
    the agribusiness partnerships project was $59.3 million.  As of
    February 1995, $52.5 million had been obligated.

          PROJECT APPROACH
    ----------------------------------------------------- Appendix III:9.3

    TVG established an office in Moscow staffed by one American director
    and three Russian nationals.  This office was supported by several
    TVG headquarters staff in California.  The American director did not
    have an agribusiness background but was responsible for managing the
    office, identifying potential Russian and American agribusiness
    partners, reviewing partnership proposals, and submitting the
    proposals to USAID for final subgrant approval.

    According to TVG officials, TVG identified potential Russian
    agribusiness partners through a network of contacts at the Ministry
    of Agriculture, Association of Individual Farms and Agricultural
    Cooperatives of Russia, World Bank, European Bank for Reconstruction
    and Development, Peace Corps, investment funds, and regional and
    local administrations.  To identify American partners, TVG canvassed
    its members in the United States, advertised in agricultural
    publications, contacted agribusinesses via telephone, and looked for
    firms already operating in Russia.

    Once identified, TVG worked with the American and Russian partners to
    develop proposals for USAID's approval.  After receiving USAID
    approval, TVG awarded subgrants to U.S.  agribusinesses working in
    Russia primarily to provide technical assistance and agricultural
    training to help create efficient food systems.  The American
    agribusiness partner was required to provide at least 2.5 times the
    level of funding provided by USAID, to ensure its commitment to the
    partnership and the long-term economic viability and sustainability
    of the joint venture.

    The items purchased with the USAID subgrants are referred to as
    "additionality" components, or those components that might otherwise
    not be included in the joint venture without USAID funding.
    Additionality components include additional training and facility
    expansion.

          CONTRACTOR PERFORMANCE
    ----------------------------------------------------- Appendix III:9.4

    TVG established six partnerships in five NIS countries, with two in
    Russia.  As of March 1995, one additional partnership in Russia was
    awaiting USAID approval.  The first partnership established in Russia
    was with Petoseed Company, Inc., and is located in Krasnodar.\25
    Petoseed produces vegetable seeds that will be sold in the NIS and
    internationally.  During the 1994 growing season, Petoseed produced
    11,000 pounds of seed in Russia.  The second TVG Russian partnership
    involves CTC Foods Company, which is building a potato processing
    facility in Pushchino.  If finished, the facility will produce dried
    potato flakes that will be sold primarily to hospitals and schools.

    The two American agribusiness partners exceeded the required level of
    partnership funding in both partnerships.  Contributions by USAID,
    U.S.  agribusiness partners, and Russian beneficiaries to the TVG
    partnerships in Russia are shown in table III.1.

                                       Table III.1

                         Agribusiness Partnerships Established by
                               Tri Valley Growers in Russia

    Su
    bg
    ra                                             Russian
    nt        Total                 U.S. firm         firm        Grant    U.S. firm
    ee  partnership  USAID grant        share      share\a     expended     expended
    --  -----------  -----------  -----------  -----------  -----------  -----------
    Pe   $1,085,975     $309,250     $776,725            0      $44,054     $181,338
     t
     o
     s
     e
     e
     d
    CT    4,300,000      800,000    2,200,000   $1,300,000      428,249    1,624,789
     C
     F
     o
     o
     d
     s
    ================================================================================
    To   $5,385,975   $1,109,250   $2,976,725   $1,300,000     $472,303   $1,806,127
     t
     a
     l
    --------------------------------------------------------------------------------
    \a The Russian partners' contributions are generally in-kind
    contributions.

    Source:  TVG statistics as of May 25, 1995.

    TVG had difficulty identifying partnerships.  TVG staff had
    difficulty beginning work in Russia because of poor telecommunication
    and office facilities, the chaotic Russian business environment, the
    limited number of American firms willing to invest in Russia, limited
    funding, and a small staff.  According to a TVG official, Petoseed
    and CTC Foods contacted TVG to participate in the project.  However,
    both companies were already working in Russia before USAID had
    established the agribusiness project and located Russian partners on
    their own.  He said they would have invested in Russia without USAID
    involvement.  An official at the Association of Individual Farms and
    Agricultural Cooperatives of Russia told us that the Association
    tried to work with TVG to identify Russian partners but received only
    "empty promises."

    Although USAID never specified the number of partnerships that it
    wanted to establish within a given time frame, it concluded that TVG
    had not performed adequately.  Between May and December 1993, USAID
    expressed concern about the number of partnership proposals TVG was
    submitting and the quality of the proposals.  A February 1995 USAID
    review of the agribusiness project stated that TVG required more
    support by USAID staff and was less committed to the project than
    CNFA.  TVG closed its office in Moscow in August 1994 and has stopped
    the Russia part of its program because USAID terminated the
    agribusiness partnerships project in Russia.  Nevertheless, USAID's
    review noted that the partnerships to which TVG had made subgrants
    were doing well.  However, a TVG official told us in May 1995 that
    because of financial problems, CTC Foods may not be able to continue
    its work in Russia.  Consequently, the processing facility in
    Pushchino may never be constructed.  According to a USAID official,
    TVG's Moldova office now monitors the Russian subgrants.

    --------------------
    \25 Petoseed also established seed production facilities in Ukraine
    and Moldova through TVG.

          RESULTS
    ----------------------------------------------------- Appendix III:9.5

    The agribusiness partnerships developed by TVG in Russia have not
    been operating long enough to adequately judge their impact.
    However, due to their limited scope, it is unlikely that the
    partnerships will have a significant effect on reforming Russia's
    agricultural sector.

          USAID MANAGEMENT
    ----------------------------------------------------- Appendix III:9.6

    USAID/Washington designed the agribusiness partnerships project in
    1992, before the USAID/Moscow mission was opened.  USAID/Washington
    and USAID/Moscow split the oversight responsibilities:  Washington
    was primarily responsible for TVG's compliance with the cooperative
    agreement and Moscow was responsible for subgrant proposal
    evaluation.  Final grant approval was a joint Moscow/Washington
    effort.

    Although USAID/Moscow raised continued concerns about the
    agribusiness partnerships project's ability to influence systemic
    reform, the project proceeded.  USAID/Moscow officials called for a
    review of the project as early as November 1993, and they developed a
    statement of work for an evaluation team.  However, USAID/Washington
    told USAID/Moscow to "forget the assessment and get on with the job."
    Consequently, no assessment was conducted.  According to USAID
    officials, an evaluation is planned for June 1995.

    According to USAID officials, the agency wanted to implement the
    project quickly and demonstrate results.  TVG's Moscow director
    stated she was pressured to submit proposals quickly because USAID
    was being pressured by Congress.  However, both CNFA and TVG
    officials complained that USAID's subgrant approval process was
    arduous and lengthy.  They said it took several months for USAID to
    accept or reject a proposal and added that USAID/Washington caused
    most of the delay.  USAID/Washington officials said the delays were
    caused by the time required to research legal issues, conduct
    environmental audits, and work through the Washington bureaucracy.

    The cooperative agreement with TVG called for quarterly program
    performance reports and annual progress reports.  An independent
    accounting firm was to audit TVG's financial statements.  Although
    USAID officials said that TVG met all of its reporting requirements,
    our review indicated that TVG had not submitted annual reports.
    According to USAID officials responsible for the project in Russia,
    USAID staff visited only half the project sites established by TVG,
    CNFA, and Agricultural Cooperative Development International between
    May 1993 and November 1994.

    USAID was required to annually assess the performance and program
    direction of the cooperative agreement and contract for an
    independent external evaluation.  As of March 1995, it had done
    neither.  However, CNFA completed an evaluation of the agribusiness
    partnerships project in August 1994 at USAID/Moscow's request.  It
    reported that the project had not started agribusiness partnerships
    quickly, had not made a significant contribution to sectoral reform,
    and had little to show for the "additionality" purchased with USAID
    funds.  CNFA's internal evaluation did not address TVG's performance.
    USAID completed an internal review in February 1995, but the review
    did not cover the additionality components.  The review stated that
    it was unrealistic to expect the overall project to have a
    significant, measurable impact on the food system in the NIS.

    USAID has discontinued the agribusiness partnerships project in
    Russia and, as of September 1994, stopped accepting proposals for
    Russian agribusiness partnerships.  In addition, USAID has decided
    not to obligate any additional funds for the project.  Agency
    officials stated that the project itself cannot adequately address
    the obstacles of reforming the agricultural sector and indicated that
    other projects, such as the Russian-American Enterprise Fund, were
    better vehicles for financing joint ventures.

       OFFICER RESETTLEMENT HOUSING
       PILOT
    ------------------------------------------------------ Appendix III:10

    The Russian officer resettlement pilot project has been successful in
    providing the required housing units, although not within the
    original time frames.  The project's secondary objectives--to provide
    job skills training for demobilized officers and to help facilitate
    housing sector reform--were only partially met.  By implementing a
    pilot program, USAID was able to test the viability of a housing
    construction project and apply lessons learned to the $160-million
    follow-on project.

    Planning and Development Collaborative International (PADCO), the
    contractor tasked to provide construction management services, was
    successful in part because it (1) had experience in working on
    housing sector reform in Russia, (2) established a physical presence
    in Moscow and in the field, (3) obtained at least some buy-in and
    involvement from the local Russian government, and (4) employed
    Russian staff to oversee construction activities.

          SECTOR PROBLEM
    ---------------------------------------------------- Appendix III:10.1

    The Russian Ministry of Defense has traditionally provided qualifying
    retired and demobilized military officers with a dwelling unit or
    plot of land and some job skills training.  After the Soviet Union
    dissolved, between 200,000 and 350,000 officers needed housing;
    approximately 42,000 were located in the Baltic Republics of Estonia,
    Latvia, and Lithuania.  However, since the dissolution, the Russian
    government has lacked the housing stock to resettle all the
    demobilized military officers.  Further, Russia's severe economic
    problems, housing shortages, and lack of "social guarantees" for
    these retired officers has delayed troop withdrawals.

          PROJECT OBJECTIVE
    ---------------------------------------------------- Appendix III:10.2

    President Clinton announced the Russian officer resettlement program
    at the Vancouver Summit in April 1993.  Later, in July 1993, he
    stated at the G-7 Heads of State meeting in Tokyo that the program
    should encourage rapid withdrawal of demobilized Russian officers
    from the Baltic Republics.  The Russian Officer Resettlement
    Initiative is being conducted in two phases--a $6-million pilot and a
    $160-million follow-on project.  The pilot's primary objective was to
    construct 450 housing units by July 1994 for the resettlement of
    demobilized Russian military officers.  The follow-on project was to
    provide up to 5,000 units (2,500 constructed and 2,500 voucher
    certificates) by November 30, 1996, for officers demobilized in the
    Baltics or other countries outside Russia.  The pilot project's
    secondary objectives were to provide job skills training, experiment
    with new housing technologies, assist private firms in housing
    development and construction, and expand the scope of housing
    choices.

          PROJECT APPROACH
    ---------------------------------------------------- Appendix III:10.3

    To implement the pilot project, USAID contracted with PADCO for
    construction management services.  It also awarded fixed-price
    contracts to five Russian builders and one private voluntary agency
    to construct housing units in five cities.  Finally, it provided a
    grant to the International Catholic Migration Commission for
    training.

    According to project officials, PADCO assisted the project design
    team that included officials from USAID/Washington and USAID/Moscow.
    This team visited potential project sites, evaluated projects, and
    negotiated construction contracts.  PADCO was responsible for
    managing the construction activities and monitoring contractor
    performance.  U.S.  officials said the design team created the pilot
    with only minimal input from the Ministry of Defense or the Ministry
    of Construction.  USAID officials added that the design team
    conducted its own field assessment to select participating cities and
    worked almost exclusively with the local authorities in these cities.
    The local authorities were to provide infrastructure services such as
    heating, water, and road access for the housing units.

    USAID relied on the Russian Ministry of Defense to select the
    officers to receive the housing.  The initial design for the pilot
    program did not stipulate where the officers should come from, but as
    a result of the Tokyo G-7 meetings in July 1993, USAID established
    criteria that gave priority to demobilized officers living in the
    Baltics.

    The criteria also included housing for officers from other areas
    outside Russia because two cities were reluctant to provide
    infrastructure for officers exclusively from the Baltics.  Under this
    criteria, officers demobilized in other areas would be included.
    USAID's compromise with these cities allowed some demobilized
    officers from their own jurisdictions to receive housing.  In Nizhniy
    Novgorod, half the officers could come from its jurisdiction, while
    in Volgograd, 40 percent of the officers could come from its
    jurisdiction.  USAID and PADCO officials said beneficiary composition
    would also be an issue in the follow-on project.

    PADCO's project staff established a long-term presence in Moscow and
    traveled regularly to the various building sites.  It also hired and
    trained Russian construction specialists to supervise the
    construction in each city.  USAID officials said PADCO's experience
    in Russian housing issues helped facilitate this project.

          CONTRACTOR PERFORMANCE
    ---------------------------------------------------- Appendix III:10.4

    PADCO and Russian contractors generally met the program's primary
    objective of providing housing units, although not within established
    time frames.  As of July 1994, only 94 (21 percent) of the 452 units
    were completed, although as of February 15, 1995, the project had
    provided 422 units (93 percent) through a combination of construction
    and voucher certificate activities.  (See table III.2 and figs.
    III.1 and III.2.)

                                       Table III.2

                          Status of Russian Officer Resettlement
                          Pilot Project as of February 15, 1995

                                                                 Pending
              Originally                  Added/  Completed/  completion  Transferre
    Location     planned  Terminated    allotted      issued   /approval  d to owner
    --------  ----------  ----------  ----------  ----------  ----------  ----------
    Construc
     tion
    Lipetsk           40          40
    Nizhniy          128                                 128                      45
     Novgorod
     1
    Nizhniy           50          50                                               0
     Novgorod
     2
    Novosibi         180         180                                               0
     rsk
    Tula 1            14                                  14                      12
    Tula 2            16                                  16                       9
    Tula 3                                    30                      30           0
    Volgogra          32                                  32                      29
     d 1
    Volgogra                                  80          80                      75
     d 2
    Volgogra                                  72          72                       0
     d 3
    ================================================================================
    Subtotal         460         270         182         342          30         170
    Voucher
     certifi
     cates
    Novgorod                                  40          40           2          38
    Pskov                                     40          40           2          38
    ================================================================================
    Subtotal                                  80          80           4          76
    ================================================================================
    Total            460         270         262         422          34         246
    --------------------------------------------------------------------------------
       Figure III.1:  Housing Units
       Built in Volgograd for Russian
       Officers

       (See figure in printed
       edition.)

       Figure III.2:  Duplexes Built
       in Tula for Russian Officers

       (See figure in printed
       edition.)

    Of the 10 project sites in 5 cities, USAID terminated 3:  one because
    newly elected local officials refused to meet the previous
    administration's commitments to provide infrastructure support to the
    housing units and 2 because contractors defaulted on their building
    commitments.

    In Novosibirsk, USAID and PADCO officials said federal and oblast'
    officials were not involved in the initial agreements.  Therefore,
    they had no authority to require the local administration to abide by
    the contract, and they would not allocate additional funds for the
    infrastructure.

    In Lipetsk, the contractor was a private voluntary agency that
    subcontracted with a local Russian construction firm to execute the
    work.  When the subcontractor defaulted, the agency was unable to
    find a replacement to complete the work.  At the Nizhniy Novgorod
    50-unit project site, project officials said the Russian contractor
    ran into financial problems and stopped work, claiming that the
    $8,500 per unit allowed in the contract was not enough to cover
    costs.  Although the city offered several incentives, including a
    $300,000 letter of credit and land for additional construction, USAID
    and PADCO officials said the contractor was unwilling to spend his
    own funds and the contract was terminated by USAID.

    USAID officials said the contractor at the 128-unit Nizhniy Novgorod
    site was concerned that $8,500 per unit was not enough to cover the
    cost of construction.  The contractor had completed almost 70 percent
    of construction when increased construction costs, caused by rapidly
    rising inflation (9 percent a month) and the devaluation of the
    ruble, forced him to stop work.  According to USAID officials,
    because the contractor had done a good job, used his own funds from
    other projects, and was well-connected with city and oblast'
    officials, he negotiated an agreement so that the oblast' and USAID
    would cover the increased costs of the 128 units.  To ensure the
    project's completion, USAID and the oblast' administration each
    provided an additional $700,000, thus increasing the per unit price
    to $19,500.

    Because contracts were terminated months after they were awarded,
    USAID developed a method to meet the housing requirements quickly.
    It awarded a contract to the Urban Institute to implement a voucher
    certificate program, which allowed officers to purchase existing
    local housing in a participating area or housing under construction.
    Because of increased construction costs, the inclusion of land, and
    infrastructure costs, the vouchers were increased from the $8,500 per
    unit in the construction program to a maximum of $25,000 per unit.
    According to USAID officials, using voucher certificates allowed the
    pilot program to provide housing units much quicker than through
    direct construction.  As of January 30, 1995, 80 vouchers had been
    disseminated to the officers, and 76 (95 percent) of the them had
    been used to purchase units, which were turned over to the officers.

    The International Catholic Migration Commission's efforts to address
    one of the project's secondary goals of job skills training has shown
    limited results, according to USAID housing officials.  As of
    December 1994, it had arranged training for 46 beneficiaries who
    attended business courses in Pskov, Novgorod, and Volgograd.  The
    USAID official said construction delays and the subsequent delays in
    officers relocating to their respective cities affected start-up
    activities.  Further, the official said the Commission did not
    adequately identify the officers' training needs and failed to
    recognize that many of them were not interested in training.

    Project officials said only minimal progress was made in addressing
    other secondary goals, such as demonstrating new housing
    technologies, expanding customer choices, and implementing more
    stringent quality control standards.  For example, in Tula,
    contractors constructed 14-, 16-, and 30-unit duplexes, which took as
    long or longer to build than traditional high rise structures.  (See
    fig.  III.2.) PADCO field representatives worked with local builders
    to ensure that quality control measures were introduced and achieved.

          RESULTS
    ---------------------------------------------------- Appendix III:10.5

    The officer resettlement pilot project accomplished its objective of
    providing housing to demobilized officers.  The project was not
    designed to address systemic reform or to be sustainable, and it did
    not do so.  PADCO officials said the attempts to sustain the effects
    of the project's secondary objectives were short-lived, although the
    lessons imparted by PADCO--new housing technologies, housing choices,
    and quality control measures-- may have some positive effect on the
    building industry and contractors.  USAID and Urban Institute
    officials said the lessons learned from implementing the voucher
    certificate activity by the banks, realtors, and local governments
    may be used to facilitate the local governments' transition to a
    private housing market.

    As a result of the pilot project, USAID incorporated the lessons
    learned as it designed the $160-million follow-on initiative to
    provide 5,000 units to officers from the Baltics.  The primary
    changes included (1) obtaining total support, involvement, and buy-in
    from all three levels of Russian government; (2) using the voucher
    certificate program to expedite the relocation of 2,500 officers; (3)
    stipulating that a maximum of 10 percent of the demobilized officers
    could come from local jurisdictions; (4) using a U.S.  construction
    management firm as the prime construction contractor and subcontract
    to the individual builders; (5) using only experienced,
    well-connected Russian builders; (6) selecting partially completed
    buildings and sites with existing infrastructure; (7) using a
    traditional Russian housing design; and (8) providing a more
    realistic per unit cost ($25,000 versus $8,500).  According to USAID
    officials, these changes are expected to allow the follow-on project
    to proceed more quickly and efficiently than the pilot.

          USAID MANAGEMENT
    ---------------------------------------------------- Appendix III:10.6

    USAID/Moscow had management responsibility for the project and
    generally did a good job of managing, monitoring, and overseeing it
    and coordinating with USAID/Washington.  PADCO officials said
    USAID/Moscow actively assisted the contractors in reaching acceptable
    compromises with government officials and contractors.  The
    USAID/Moscow project team reviewed project status reports, visited
    project sites, and held regular meetings with contractors.  Finally,
    AID terminated work when problems could not be overcome.

    (See figure in printed edition.)Appendix IV
    COMMENTS FROM THE U.S.  AGENCY FOR
    INTERNATIONAL DEVELOPMENT
    ========================================================= Appendix III

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    (See figure in printed edition.)

    The following are GAO's comments on USAID's letter dated June 1,
    1995.

    GAO'S COMMENTS

    1.  We have incorporated these comments into the report where
    appropriate.

    2.  Although we noted project shortcomings, we also recognized the
    contribution Deloitte & Touche made toward the privatization process
    and considered the project a success.  Moreover, we recognized
    USAID's positive contribution to the overall privatization effort.

    3.  We conducted a detailed review of Tri Valley Growers'
    performance, one of the three contractors responsible for
    implementing the agribusiness partnerships project, to determine
    whether this expenditure of funds had any sustainable impact.  We
    concluded that it did not.  Although we did not draw any conclusions
    about the agribusiness partnerships project as a whole, our analysis
    casts doubt on whether the project can have a systemic impact if the
    individual partnerships are not having an impact.  (See comments 29
    and 30 for additional discussion.)

    4.  It is too early to know whether USAID's prediction concerning the
    outcome of ongoing activities in the energy sector will result in
    significant sector reform.  Many of these projects are just starting
    and must overcome many obstacles.  For example, in our September 1994
    report on nuclear safety, we reported that there are no guarantees
    that the international assistance effort will result in safer
    reactors or expedite the closure of the riskiest reactors.\1 In fact,
    in the absence of a commitment to close down the reactors, the
    assistance may encourage their continued operation.  We noted that
    donor countries face formidable challenges in promoting the closure
    of the Soviet-designed reactors because the countries operating them
    depend on the nuclear power to meet their needs for domestic energy
    and export income.

    5.  We agree that the new evaluation system is promising in that it
    should provide an improved basis to evaluate USAID's programs in the
    NIS.  However, since the first report is not due until November 1995,
    it is too early to know whether the system will fulfill its promise.
    The value of the system will depend on the indicators selected, the
    reliability of the data, and the subjective judgments of USAID
    officials preparing the reports.  For the system to have credibility,
    USAID will have to be able to identify shortcomings as well as
    successes.

    6.  We have modified the report to reflect this information.

    7.  We were able to reconcile obligations and expenditures in the
    USAID financial report with other USAID documentation.  Accordingly,
    we have deleted the examples from the report.

    8.  Our draft report included information on the work of the Consuls
    General.  We have modified our report to update the information on
    increased site visits.

    9.  Although market forces played a role in the limited use of some
    of the centers, the lack of local support as well as other factors
    also caused the low activity levels at some centers.  More
    importantly, it is questionable whether USAID should spend funds on
    activities without a market unless it has a strategy to create demand
    for the product it is financing.

    10.  We visited only one site (in Siberia during February) because of
    the limited amount of time we had in country.  Vorkuta can be reached
    by plane in the winter.  USAID can visit the sites at other times
    during the year.  We believe that three site visits--two occurred
    after we began our audit--in 31 months is inadequate for monitoring
    purposes.  Day-to-day contacts with PIER staff are important;
    however, they do not substitute for site monitoring or provide USAID
    with an objective basis for evaluating the project's success.

    11.  The accomplishments noted in our report are those that have had
    the greatest impact.  PIER did not provide us with any statistics
    that indicated increased mine safety or productivity.  Moveover, a
    September 1994 study produced by the U.S.  Department of the Interior
    indicates that mine safety in Russia is actually getting worse.  The
    beneficiaries we spoke with indicated that they were implementing new
    mining methods introduced by PIER; however, they did not mention any
    measurable increases in productivity.  In addition, although
    productivity and efficiency are important, overall production for the
    coal sector is still too high.  Finally, PIER's Moscow director
    stated that this project has had the greatest impact in the areas of
    restructuring, private sector involvement, and social safety net
    development.

    12.  Our report does not state that an interim evaluation is imminent
    and may lead to activities being redirected.  Our report states that
    "USAID management admitted that no annual assessments or midterm
    evaluations were conducted," even when required by the cooperative
    agreement.

    13.  USAID's new procedures did not affect the program during the
    time frames we reviewed.  Also, the work plan example should be taken
    in the context that several iterations of the plan have been
    submitted and revised since November 1994.

    14.  Our draft report did not recommend that more authority be
    delegated.

    15.  Our report was modified to show that the Ministry of
    Environmental Protection and Natural Resources was involved in the
    initial selection of project activities.  However, the Ministry did
    not participate in designing the projects as USAID suggests.  USAID
    acknowledges the almost immediate shift of its relationship from the
    central to the local government once the projects were selected.  We
    remain concerned over the lack of federal involvement, especially
    regarding the provision of resources and the limited potential for
    replicability.  We believe that without outside funding or support
    from the federal level, sustainability and replication will be
    difficult.

    16.  As indicated in the report, we found that as of February 1995,
    the Far East project had contributed little to systemic reform and is
    unsustainable without outside funding.  The report discusses the
    project's attempt to address systemic reform through efforts to
    maintain and restock the forestry base.

    17.  As indicated in the report, the deliverables identified in the
    delivery orders generally cited reports and studies as the results.
    We are unable to verify USAID's statements regarding the project's
    results.

    18.  Although we recognize that this project was only one of many in
    the energy sector, we found that the project is unlikely to
    contribute to systemic reform because of its design and the lack of
    monitoring and follow-up by USAID.

    19.  USAID suggested that we select this project in part because
    USAID represented it as a success, based on an independent
    evaluation.  We visited only Yekaterinburg for two primary reasons:
    USAID suggested that it was a good site to visit and it was one of
    only two cities where equipment was installed and extensive energy
    audits and training were provided.  However, as we reported, the
    equipment had not been installed.  USAID's assertion that our
    conclusion is based almost entirely on the site visit is wrong.  Our
    conclusion is also based on discussions with representatives from
    RCG/Haggler Bailly, Joseph Technology, Honeywell, and USAID officials
    responsible for the project and our review of numerous documents on
    the entire project.

    20.  We did not make statements or draw conclusions about other
    projects in the program or about the overall program.  We noted that
    there was no indication that the project we reviewed had contributed
    to systemic reform.  The energy efficiency audits and demonstration
    sites can only have an impact on systemic reform if USAID ensures
    that (1) equipment is installed; (2) equipment and training is used;
    (3) the recommendations in the energy audits are implemented; (4) the
    results of the project are monitored, recorded, and publicized; (5)
    appropriate personnel have access to the demonstration sites; and (6)
    problems such as lack of equipment certification are corrected.  At
    the time we conducted our fieldwork, USAID was not ensuring any of
    these elements because the project did not include any mechanisms for
    long-term monitoring or replicating the project.  USAID, in its
    comments, acknowledged that the project alone is unlikely to have an
    impact on systemic reform.

    21.  We agree that the dollar value of the uninstalled equipment
    constitutes a relatively small percentage of all the equipment
    purchased.  The fact that USAID was unaware of the problems in
    Yekaterinburg and had not monitored whether any cost savings had been
    achieved and did not know whether any systemic improvements had
    resulted from the equipment, energy audits, and training provided is
    the issue.

    22.  The consultant's evaluation indicated that all of the equipment
    was installed in April 1993.  The documents we reviewed indicate that
    the equipment was provided in April and May 1993.  If the equipment
    did not arrive until June 1993 as USAID suggests, USAID should have
    known the evaluation had problems when it indicated that all the
    equipment was installed 1 or 2 months before the equipment ever
    arrived.

    23.  We contacted individuals who were identified as participants by
    USAID/Moscow.

    24.  Our conclusion that the training was irrelevant was not based on
    our discussions with the participants of the PIET training courses.
    It was based on statements by various USAID officials, including the
    USAID/Moscow Mission Director and the Chief of USAID/Washington's
    Europe and the Newly Independent States/Health and Population Office.
    For example, USAID officials told us that a 2-week training course
    without follow-on activities could not be expected to result in any
    systemic reform.  In addition, as we also noted in the report,
    because USAID had not conducted any long-term monitoring of the
    participants, it had no evidence that any of the participants
    instituted systemic changes based on the training.  The opinions and
    views of the participants we interviewed were used to provide insight
    as to why no systemic changes had occurred.

    25.  Contrary to USAID's comments, we referred to the course
    evaluations in our draft report.  We stated that ".  .  .  most
    participants were satisfied with the course and believed it was
    applicable to their work conditions." However, our assessment was not
    concerned with whether the participants were satisfied with the
    course but with whether the goals of the PIET contract and the
    Freedom Support Act were fulfilled.

    26.  We question USAID's assertion about the actual positive impact
    of the training and follow-on assistance.  First, at the time of our
    review, follow-on assistance was not planned to begin for another 6
    months and no assessment had been completed to confirm or deny
    USAID's assertion.  Second, despite repeated USAID statements that
    the Siberian participants were involved in follow-on projects, USAID
    project officials did not know how many were actually involved.  When
    the contractor compiled this data for us, the USAID project official
    in charge of the program was surprised that 75 percent of the
    participants in Siberia were not involved with the planned follow-on
    activities.  Finally, USAID health officials we spoke with were
    unanimous in their assessment that the follow-on project's progress
    to date has been a disappointment.

    27.  Contrary to USAID's assertion, we met with the Russian
    Privatization Center (RPC) official responsible for the project.
    This official questioned the competence of some of the consultants.

    We also believe that the characterization of this project as a
    qualified success is an overstatement.  Project task orders were
    never modified, thus the focus of the project remained to increase
    the availability of commercial real estate.  However, after project
    completion, large amounts of commercial real estate continued to be
    leased in the selected cities under conditions that encouraged
    inefficient use, and the municipalities failed to maximize revenues.

    Furthermore, we noted that the Center, in one of the memorandums
    quoted by USAID, questioned the reliability of IBTCI representations
    and indicated that USAID and the Center believed that the Perm' model
    was being rolled out to the other cities.  The memorandum stated the
    following:

         At our most recent meeting, on June 2, Jay Kalotra [the project
         director] presented a preliminary draft of the wrap-up memo for
         the project.  At that time, I reminded Jay several times that
         deviations from the Perm' model would have to be rigorously
         defended to both USAID and the senior management of the RPC.
         Jay's response was that IBTCI deviated from the Perm' model in
         large part because the model was ill-suited to the chosen
         roll-out cities.  To a considerable extent, this may be true.
         However, this obviously does not exonerate IBTCI, since their
         task was to find cities where a pure roll-out could be
         performed.  [underscoring supplied]

         In its most recent memos, IBTCI suggests that they told us at
         the outset they would adopt a broader approach than BCG took in
         Perm'.  While it is true that IBTCI states some very ambitious
         goals, it is disingenuous to suggest that we agreed to replace
         the Perm' model with something else.  .  .  .  [underscoring
         supplied]

         .  .  .  However, as noted above, even three weeks ago IBTCI was
         maintaining the pretense of Perm'-style results.  And it was
         only when we actually visited the project sites that we could
         see the extent to which deviations had occurred.  [underscoring
         supplied]

    IBTCI was clearly responsible for selecting the cities.  GKI relied
    on the contractor, but GKI's approval did not relieve the contractor
    of this responsibility.  Because the contractor did not attempt to
    use the Perm' model during the roll-out phase, it is impossible to
    determine if the Perm' model could be applicable to the other cities.

    28.  Our report draft specifically stated that the IBTCI team spoke
    Russian and had meaningful business experience.  The issue was
    IBTCI's awareness of local conditions in order to replicate the Perm'
    model.  Although the project manager on the roll-out was also the
    comanager for the pilot, none of the city team leaders had worked on
    the Perm' pilot.

    29.  Our analysis casts doubt on whether the entire project can have
    a systemic impact if the individual partnerships are not having an
    impact.

    30.  We did not assess the activities of CNFA because the USAID
    Inspector General was evaluating CNFA's work.  Discussions with the
    USAID Inspector General's office indicate that some of CNFA
    partnerships are experiencing problems and consequently are not
    contributing to systemic reform.  In addition, CNFA's August 1994
    evaluation of the agribusiness partnerships project indicates that
    the overall project had not achieved the desired results.  Also,
    USAID's February 1995 review of the agribusiness partnerships states
    the following:

         It is not realistic to expect the agribusiness partnerships
         program to have a significant, measurable impact on overall food
         systems in the NIS.  The limited number of partnerships being
         supported suggests such national level impacts are unlikely
         during the life of the activity, if ever.

    USAID/Moscow staff stated that the agribusiness partnerships project
    could not by itself influence systemic reform.

    31.  Although TVG has established three partnerships in Russia, only
    one (Petoseed) is functioning.  According to TVG staff, CTC Foods has
    run into financial problems; consequently, its potato processing
    facility may never be constructed.  Finally, the third partnership
    (Big Sky Foods Trading, Inc.) has only recently been approved, and it
    is too early to determine whether this project will be successful.

    32.  USAID staff in Moscow and Washington characterized the project
    as discontinued because no more partnerships can be introduced and no
    more funds will be obligated.

    33.  USAID/Moscow staff said TVG had not performed adequately and had
    not identified appropriate partnerships.  The documents we reviewed
    also indicated that TVG was not performing well.

    34.  Our report includes examples of the causes for delays in the
    approval process, including the need to deal with legal issues.

    35.  We modified the report to reflect this foreign policy goal.
    However, the primary objective for the pilot project announced in
    April 1993 did not focus on relocating officers from the Baltics.
    The announcement made in July 1993 focused the program on the removal
    of officers from the Baltics.

    36.  USAID is incorrect in stating that the oblast' was involved in
    signing the memorandum for Novosibirsk.  It was only signed by USAID
    and the municipality of Novosibirsk.

    --------------------
    \1 Nuclear Safety:  International Assistance Efforts to Make
    Soviet-Designed Reactors Safer (GAO/RCED-94-234, Sept.  29, 1994).

    MAJOR CONTRIBUTORS TO THIS REPORT
    =========================================================== Appendix V

    NATIONAL SECURITY AND
    INTERNATIONAL AFFAIRS DIVISION,
    WASHINGTON, D.C.

    Louis H.  Zanardi
    Eugene D.  Beye
    Edward J.  George, Jr.

    EUROPEAN OFFICE

    Peter J.  Bylsma
    David M.  Bruno
    Jodi McDade Prosser