    Millennium Challenge Corporation: Vanuatu Compact Overstates
    Projected Program Impact (11-JUL-07, GAO-07-909).

    In January 2004, Congress established the Millennium Challenge
    Corporation (MCC) for foreign assistance. Congress has
    appropriated almost $6 billion to MCC. As of March 2007, MCC had
    signed almost $3 billion in compacts with 11 countries, including
    a 5-year, $65.7 million compact with Vanuatu. MCC states that the
    Vanuatu compact will have a transformational effect on the
    country's economy, increasing per capita income and GDP and
    benefiting 65,000 poor, rural people. GAO examined (1) MCC's
    methods of projecting economic benefits, (2) MCC's portrayal and
    analysis of the projected benefits, and (3) risks that may affect
    the compact's impact. GAO reviewed MCC's analyses and met with
    officials and business owners in Vanuatu as well as with other
    donors.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-07-909
        ACCNO:   A72527
      TITLE:     Millennium Challenge Corporation: Vanuatu Compact
    Overstates Projected Program Impact
         DATE:   07/11/2007
      SUBJECT:   Budgeting
    	     Cost analysis
    	     Cost control
    	     Economic growth
    	     Economically depressed areas
    	     Financial statements
    	     Impacted areas
    	     Income statistics
    	     Program evaluation
    	     Federal corporations
    	     Federal aid to foreign countries
    	     Risk management
    	     Cost estimates
    	     Vanuatu

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-07-909

         * [1]Results in Brief
         * [2]Background

              * [3]Description of Vanuatu
              * [4]Vanuatu's Economy
              * [5]MCC's Compact with Vanuatu

         * [6]MCC Projected Compact's Impact Using Estimates of Benefits,

              * [7]MCC Calculated Benefits, Costs, and Beneficiaries

                   * [8]Benefits
                   * [9]Costs
                   * [10]Beneficiaries

              * [11]MCC Used Projected Benefits and Costs to Determine ERR and E

         * [12]MCC's Data Do Not Support Its Portrayal of Compact Benefits

              * [13]Portrayal of Per Capita Income Benefit Suggests Larger Effec
              * [14]Portrayal of GDP Benefit Differs from Underlying Analysis
              * [15]Portrayal of Poverty Reduction Does Not Quantify Benefits to
              * [16]Estimates of Compact Benefits Are Generally Reasonable, but

         * [17]Several Risks May Lead to Reduced Project Benefits

              * [18]Construction Costs May Exceed Contingencies
              * [19]Compact Benefits Are Likely to Accrue More Slowly Than Proje
              * [20]Project Maintenance after Compact Expiration Cannot Be Ensur
              * [21]Induced Benefits Require Tourism Providers and Agricultural
              * [22]Efficiency Gains May Not Cause Measurable Change in Per Capi

         * [23]Conclusions
         * [24]Recommendations
         * [25]Agency Comments and Our Evaluation

              * [26]Compact Impact on Incomes and Growth
              * [27]Compact Beneficiaries
              * [28]Program Risks
              * [29]Per Capita Income Impact
              * [30]Calculation of Beneficiaries

         * [31]GAO Comments
         * [32]GAO Contact
         * [33]Staff Acknowledgments

              * [34]Order by Mail or Phone

    Report to the Chairman, Committee on Foreign Affairs, House of
    Representatives

    United States Government Accountability Office

    GAO

    July 2007

    MILLENNIUM CHALLENGE CORPORATION

    Vanuatu Compact Overstates Projected Program Impact

    GAO-07-909

    Contents

    Letter 1

    Results in Brief 3
    Background 6
    MCC Projected Compact's Impact Using Estimates of Benefits, Costs, and
    Catchment Area 12
    MCC's Data Do Not Support Its Portrayal of Compact Benefits 17
    Several Risks May Lead to Reduced Project Benefits 24
    Conclusions 29
    Recommendations 30
    Agency Comments and Our Evaluation 30
    Appendix I Objectives, Scope, and Methodology 35
    Appendix II Prior Development Assistance to Vanuatu 38
    Appendix III MCC Estimates of Per Capita Income Impact and Poor, Rural
    Beneficiaries 40
    Appendix IV MCC Cost Estimates for the Vanuatu Compact 47
    Appendix V Illustrative Alternative Calculations of Vanuatu Compact Impact
    49
    Appendix VI Comments from the Millennium Challenge Corporation 52
    GAO Comments 57
    Appendix VII GAO Contact and Staff Acknowledgments 63

    Tables

    Table 1: Average Annual Contributions of Top Five Donors to Vanuatu in
    2004 and 2005 38
    Table 2: Number of Poor, Rural Beneficiaries of Vanuatu Compact, as
    Calculated by MCC 43
    Table 3: Effect of Calculation and Data Errors on Beneficiary Population
    in Efate and Santo 44
    Table 4: Effect of MCC Assumptions on Beneficiary Population in Efate and
    Santo 44
    Table 5: Summary of Compact Impacts as Presented by MCC, with GAO
    Recalculations 49
    Table 6: Summary of Compact ERR, GDP, and Per Capita Income impacts under
    Alternative Scenarios of Accounting for Risks to Benefits 50
    Table 7: Summary of Selected Compact Project Impacts under Alternative
    Scenarios Accounting for Risks to Benefits 51

    Figures

    Figure 1: Location of Vanuatu 7
    Figure 2: Development and Implementation of Vanuatu Compact 9
    Figure 3: MCC Vanuatu Projects by Size and Location 11
    Figure 4: MCC Statement of Impacts in March 2006 Congressional
    Notification 12
    Figure 5: MCC's Logic Model for the Vanuatu Compact 14
    Figure 6: MCC Benefits Calculations as Source of Statements about Impact
    of Vanuatu Compact 17
    Figure 7: Vanuatu Compact's Projected Impact on Real Per Capita Income
    According to MCC Statement and MCC Data Relative to 2005 Per Capita Income
    19
    Figure 8: MCC Methodology for Projecting Vanuatu Compact's Impact on Real
    Per Capita Income 20
    Figure 9: Vanuatu GDP Growth with and without MCC Compact 21
    Figure 10: MCC Analysis of Distribution of Vanuatu Compact Benefits 22
    Figure 11: MCC Compact's Projected Impact on Vanuatu Per Capita Income
    with and without Adjusted Population Estimates 41
    Figure 12: MCC Compact's Projected Impact on Vanuatu Per Capita Income in
    Real Terms Relative to Per Capita Income without MCC 42
    Figure 13: MCC Efate Ring Road Catchment Area 45

    This is a work of the U.S. government and is not subject to copyright
    protection in the United States. It may be reproduced and distributed in
    its entirety without further permission from GAO. However, because this
    work may contain copyrighted images or other material, permission from the
    copyright holder may be necessary if you wish to reproduce this material
    separately.

    Abbreviations

    ADB Asian Development Bank
    CEO Chief Executive Officer
    ERR economic rate of return
    GDP gross domestic product
    IMF International Monetary Fund
    MCA Millennium Challenge Account
    MCC Millennium Challenge Corporation
    PWD Public Works Department

    United States Government Accountability Office
    Washington, DC 20548

    July 11, 2007

    The Honorable Tom Lantos
    Chairman
    Committee on Foreign Affairs
    House of Representatives

    Dear Mr. Lantos:

    In January 2004, Congress established the Millennium Challenge Corporation
    (MCC) to administer the Millennium Challenge Account (MCA) for foreign
    assistance. MCC's mission is to reduce poverty by supporting sustainable,
    transformative economic growth in developing countries that create and
    maintain sound policy environments. MCC carries out this mission by
    funding projects or activities in developing countries that have
    demonstrated a commitment to ruling justly and democratically, encouraging
    economic freedom, and investing in people. Congress appropriated almost $6
    billion for fiscal years 2004 to 2007 to MCC, and the President has
    requested an additional $3 billion in MCC funding for fiscal year 2008.

    As of March 2007, MCC had signed compacts with 11 countries, totaling
    approximately $3 billion, including a 5-year, $65.7 million compact with
    the Pacific island nation of Vanuatu.^1 Although MCC's compact with
    Vanuatu is its smallest compact monetarily, it provides by far the largest
    amount relative to the country's population and gross domestic product
    (GDP).^2 In a process known as due diligence, MCC analyzed Vanuatu's
    proposal for compact assistance to determine the compact's expected
    economic rate of return (ERR)^3 and impact on poverty reduction and
    economic growth. After completing its due diligence analysis, MCC stated
    in its compact and in its notification to Congress that it expects the
    Vanuatu compact to have a transformational impact on Vanuatu's economic
    development--an effect MCC defines as "a dramatic and long-lasting impact
    on poverty reduction through sustainable economic growth."^4 MCC states
    that its compacts will provide or contribute to a transformational impact
    in 5 of its 11 compacts.^5

    ^1An MCC compact is an agreement between the U.S. government, acting
    through MCC, and the government of a country eligible for MCA assistance.

    ^2MCC's $65.7 million compact with Vanuatu provides $317 per capita; in
    contrast, MCC's $547 million compact with Ghana--its largest
    compact--provides $25 per capita. The amounts provided per capita by the
    11 compacts signed to date range from $6 for Madagascar to $317 for
    Vanuatu.

    ^3Project cash flows are determined by comparing program spending against
    future expected increases in value added or income. The internal rate of
    return is calculated for these cash flows to summarize the economic
    impact. MCC refers to this internal rate of return as the economic rate of
    return.

    At your request, we reviewed MCC's economic analyses of the Vanuatu
    compact. Specifically, we examined

               o MCC's methods of projecting the compact's economic benefits and
               methods of calculating the benefits,

               o MCC's portrayal and analysis of the projected benefits, and

               o risks that could affect the compact's impact on poverty
               reduction and economic growth.

               We reviewed MCC's record of due diligence and supplemented this
               review with interviews with MCC officials to identify MCC's logic,
               data, methods, and assumptions for determining the compact's
               projected costs and benefits, and to identify the projected
               effects on GDP, per capita income, and poverty. We evaluated MCC's
               statements about the compact's impacts in its notification to
               Congress, in the Vanuatu compact, and in its "investment memo"^6
               by comparing these statements with the underlying analyses and
               data used to support them. We could not validate most of

    ^4Millennium Challenge Account, Best Practices in Compact Development
    (Washington, D.C.: 2006).

    ^5For example, in Nicaragua, MCC expects that the compact will transform
    project areas into an engine of economic growth; in El Salvador, MCC
    states that the compact provides an historic opportunity to transform the
    country's economic development; and in Armenia, MCC is undertaking road
    and irrigation projects to transform the economic performance of Armenia's
    agricultural sector.

    ^6The "investment memo" is an MCC internal document prepared by MCC's
    compact assessment team and submitted to MCC's investment
    committee--consisting of MCC's Chief Executive Officer (CEO), vice
    presidents, and other senior officials. The committee reviews the memo and
    decides whether to recommend proceeding to compact negotiations.

               MCC's underlying data and assumptions because the data were not
               available or could not be checked within the time frames of our
               engagement. We examined MCC's methodologies and checked the
               analyses for calculation errors. Further, we identified risks to
               MCC's compact results based on our review of MCC's internal
               documentation, donor reporting, and academic development
               literature. We interviewed Vanuatu and MCC officials and contacted
               the contractor that assisted in MCC's analyses. We also
               interviewed interested parties such as tourism and agriculture
               business owners in Vanuatu. We focused our analysis and field work
               on MCC's three transportation infrastructure projects on Vanuatu's
               two most populous islands, Santo and Efate, which represent 56
               percent of compact cost. We modeled several areas of project risks
               to illustrate their maximum impact on the economic analyses of
               ERR, GDP, and per capita income. In modeling these risks, we used
               the data from MCC's economic analyses; we did not validate these
               data. We conducted our review from August 2006 through May 2007 in
               accordance with generally accepted government auditing standards.

    			  Results in Brief

               MCC projected the impact of the Vanuatu compact by estimating the
               program's benefits, costs, and beneficiaries and calculating the
               compact's effect on per capita income, GDP, and poverty reduction.
               According to MCC, transportation infrastructure improvements will
               provide direct benefits from construction spending in the local
               economy, reduced transportation costs, and improved services. The
               improved infrastructure will also provide induced benefits from
               growth in Vanuatu's tourism and agriculture sectors. MCC estimated
               the value of these benefits over a 20-year period, beginning in
               full in 2008 or 2009 and growing each year. MCC developed its
               project cost estimates based on existing cost estimates prepared
               for the government of Vanuatu and for another donor. To determine
               the number of poor, rural beneficiaries, MCC defined a catchment
               area--the geographic area in which benefits may be expected to
               accrue--using maps of Vanuatu and data from the most recent
               Vanuatu census. Using its projected benefit and cost data, MCC
               calculated (1) the compact's ERR, comparing projected benefits to
               projected costs; (2) per capita income, determining the total
               benefits and dividing the total value by Vanuatu's baseline
               population; and (3) the compact's effect on Vanuatu's GDP by
               computing the total benefits added to the economy.

               MCC's portrayal of the Vanuatu compact's impact does not reflect
               the data and analysis underlying its projections of the compact's
               benefits. MCC states that as a result of the compact, per capita
               income will increase by approximately $200, or 15 percent, by 2010
               and $488, or 37 percent, by 2015. This statement suggests that per
               capita income in 2010 and 2015 will be, respectively, 15 percent
               and 37 percent higher than without the compact. However, MCC's
               underlying data show that these figures represent the sum of gains
               in per capita income for individual years relative to 2005 rather
               than actual, or net, annual gains as of 2010 or 2015. For example,
               MCC sums the per capita income gains relative to 2005 for 2006 to
               2010, averaging 3 percent, as 15 percent in 2010, without stating
               that these gains are cumulative. Our analysis of MCC's data shows
               that actual gains in per capita income, relative to income in
               2005, would be $51, or 3.9 percent, in 2010 and $61, or 4.6
               percent, in 2015. Likewise, MCC states that Vanuatu's GDP will
               increase by "an additional 3 percent a year." However, MCC's
               underlying data and calculations show that while the level of
               Vanuatu's GDP will grow by 6 percent in 2007, the economy's growth
               rate in subsequent years continues at approximately 3 percent,
               MCC's assumed rate without the compact. Regarding MCC's portrayal
               of the compact's impact on poverty reduction, MCC's publicly
               available documents state that the compact is expected to benefit
               approximately 65,000 poor, rural inhabitants "living nearby and
               using the roads to access markets and social services." However,
               MCC's underlying documentation shows that these 65,000
               beneficiaries will not receive the majority of the benefits but
               will instead share the 43 percent of the compact's monetary
               benefits that MCC expects to go to the local population; the
               remaining 57 percent will go to beneficiaries such as tourism
               services providers, transport companies, and local businesses.
               Finally, although MCC's estimates of compact benefits and
               beneficiaries are generally reasonable, some of the calculations
               and assumptions it used are problematic. Correcting MCC's
               calculations of per capita income benefits slightly reduces the
               estimated benefit, and correcting MCC's calculations and fully
               discounting some of its assumptions regarding the count of
               beneficiaries would reduce estimated beneficiaries on Santo and
               Efate by about one-third.

               We identified five key risks that could affect the Vanuatu
               compact's projected impact on poverty reduction and economic
               growth.

               o The contingencies included in MCC's calculations of construction
               costs may not be sufficient to cover potential cost overruns. We
               received MCC cost estimate documentation for 5 of MCC's 11
               construction projects showing that MCC's estimates for these 5
               include design contingencies of 20 percent; however, a previous
               study found average cost overruns of more than 20 percent in
               transportation infrastructure projects in other countries.
               Further, the risk of excessive cost overruns is significant in a
               small country such as Vanuatu. Any construction cost overrun could
               cause MCC to reduce the compact's scope and therefore its
               benefits.

               o Although the compact's benefits are projected to begin shortly
               after completion of the projects, some of these benefits will
               likely accrue more slowly. For example, according to agricultural
               and timber producers, their businesses will likely respond
               gradually to any increased market opportunities.

               o Whereas the benefit projections assume continued maintenance of
               completed projects, MCC's ability to ensure such maintenance will
               end in 2011. Moreover, previous donors to Vanuatu have found
               Vanuatu's record of maintaining donor projects to be poor. Reduced
               maintenance would lead to reduced benefits from the project.

               o The projected induced benefits from expanded tourism and
               agriculture depend on businesses and rural inhabitants responding
               to opportunities created by improved infrastructure.

               o Efficiency gains that MCC counts as direct benefits, such as
               time saved in transit, may not be put to economic use and result
               in increased per capita income, as MCC projects.

               Our analysis of these areas of risk illustrates the extent to
               which MCC's benefit projections are dependent on assumptions of
               immediate realization of benefits, successful long-term
               maintenance, realization of induced benefits, and benefits from
               efficiency gains. Accounting for these risks can reduce overall
               compact ERR from 24.2 percent, as projected by MCC, to between 5.5
               percent and 16.5 percent, and some projects may have a negative
               ERR.^7

               To help better express and determine the impact of its compacts,
               this report recommends that the Chief Executive Officer (CEO) of
               MCC (1) revise the public reporting of the projected impact of the
               Vanuatu compact, (2) assess whether similar statements in other
               compacts accurately reflect underlying data, and (3) improve MCC's
               economic analysis by phasing costs and benefits and more fully
               accounting for risks to project benefits.

    ^7We modeled risk scenarios by first assuming the phasing of costs and
    benefits. We then combined this phasing with three additional models that
    assumed a lack of maintenance, a lack of induced benefits, and a lack of
    monetized efficiency gains. See appendix V for more information about our
    methodology.

               We provided a draft of this report to MCC. In commenting on the
               draft, MCC did not directly acknowledge our recommendations. MCC
               acknowledged that its use of projected cumulative compact impact
               on income and growth was misleading, but asserted that it had no
               intention to mislead and that its portrayal of projected compact
               benefits was factually correct. MCC questioned our finding that
               its underlying data and analysis do not support its portrayal of
               compact benefits and our characterization of the program's risks.
               In response to MCC's comments, we clarified the terms and
               presentation of our analysis of MCC's statements and data and
               provided additional information on the basis for our findings. We
               also provided technical corrections where appropriate.

    			  Background

    			  Description of Vanuatu

               Vanuatu consists of 83 islands spread over hundreds of miles of
               ocean in the South Pacific, 1,300 miles northeast of Sydney,
               Australia (see fig. 1). About 39 percent of the population is
               concentrated on the islands of Santo and Efate. Vanuatu's capital,
               Port Vila, is on Efate, and Vanuatu's only other urban center,
               Luganville, is on Santo. The country has three official
               languages--English, French, and Bislama--but more than 100 other
               dialects are also spoken. Traditional custom chiefs have a
               significant role in the state particularly in rural areas. Civil
               unrest has caused occasional disruptions; for example, riots
               erupted in Port Vila over the misappropriation of assets from the
               Vanuatu National Provident Fund in 1998, and an ethnic conflict on
               Efate led Vanuatu's parliamentary government to declare a state of
               emergency in March 2007.

               Figure 1: Location of Vanuatu

    			  Vanuatu's Economy

               In the past decade, Vanuatu's real GDP growth averaged 2 percent,
               although more rapid population growth led to a decline in per
               capita GDP over the same period. Average growth of real GDP per
               capita was negative from 1993 to 2005. In its economic analyses,
               MCC used a baseline 2005 Vanuatu per capita income of $1,326. An
               estimated 40 percent of Vanuatu's population of about 207,000 has
               an income below the international poverty line of $1 per day.
               Agriculture and tourism are the principal productive sectors of
               Vanuatu's economy, contributing approximately 15 percent and 19
               percent to GDP, respectively. Although agriculture represents a
               relatively small share of Vanuatu's overall economy, approximately
               80 percent of Vanuatu's residents live in rural areas and depend
               on subsistence agriculture for food and shelter, selling surplus
               commodities to generate cash for school fees, transportation,
               consumer goods, and services. Copra (dried coconut) is the main
               cash crop; kava and cacao are also grown. The tourism sector is
               dominated by expatriates of foreign countries living in Vanuatu,
               who also predominate in other formal sectors of the economy such
               as plantation agriculture and retail trade. According to the Asian
               Development Bank (ADB), tourism is one of the most promising
               sectors in Vanuatu in terms of its potential for earning foreign
               exchange and generating employment. According to a survey
               conducted by the Vanuatu National Statistics office, the largest
               markets for Vanuatu tourism are Australia and New Zealand. Despite
               prior donor efforts, private sector development in Vanuatu's
               tourism and agricultural sectors faces challenges, including
               political uncertainty, and high costs of doing business. (See app.
               II for a summary of donor efforts and Vanuatu's development
               challenges.)

    			  MCC's Compact with Vanuatu

               On May 6, 2004, MCC determined that Vanuatu was eligible to submit
               a compact proposal for Millennium Challenge Account (MCA)
               funding.^8 Vanuatu's proposal identified transportation
               infrastructure as a key constraint to private-sector development.
               Based on the analysis performed during approximately 5 months of
               due diligence,^9 MCC signed a 5-year, $65.7 million compact with
               the government of Vanuatu on March 2, 2006, with entry into
               force^10 on April 28, 2006.^11 In keeping with its statutory
               requirements, MCC submitted a congressional notification of the
               compact signing on March 7, 2006.

    ^8The Millennium Challenge Act of 2003 requires MCC to determine whether
    countries are eligible for MCA assistance each fiscal year. Countries with
    per capita income at or below a set threshold may be selected as eligible
    for assistance if they meet MCC indicator criteria and are not statutorily
    barred from receiving U.S. assistance. MCC uses 16 indicators divided into
    three categories: Ruling Justly, Encouraging Economic Freedom, and
    Investing in People. To be eligible for MCA assistance, countries must
    score above the median relative to their peers on at least half of the
    indicators in each category and above the median on the indicator for
    combating corruption. GAO, Millennium Challenge Corporation: Compact
    Implementation Structures Are Being Established; Framework for Measuring
    Results Needs Improvement, [35]GAO-06-805 (Washington, D.C.: July 28,
    2006).

               Figure 2 illustrates the chronology of the development and
               implementation of the Vanuatu proposal and compact.

    Figure 2: Development and Implementation of Vanuatu Compact

    ^9Prior to the due diligence process, a transaction team conducts a
    preliminary assessment of each proposal and reports its findings in an
    internal "opportunity memo" to the MCC investment committee. If the
    opportunity memo is approved, the team launches a detailed due diligence
    review, using MCC's published proposal guidelines and criteria to analyze
    project costs and benefits and assess the proposal's consultative process,
    rationale, environmental and social impact, and sustainability. The
    transaction team makes recommendations based on its assessment of the
    proposal in an investment memo to the MCC investment committee.

    ^10MCC and the country's accountable entity must complete supplemental
    agreements, including a disbursement agreement and a procurement
    agreement, before the compact enters into force and funds are disbursed.

    ^11By U.S. law, all compacts must end within 5 years of entry into force.

    The $65.7 million Vanuatu compact includes $54.5 million for the
    rehabilitation or construction of 11 transportation infrastructure assets
    on 8 of Vanuatu's 83 islands, including roads, wharves, an airstrip, and
    warehouses (see fig. 3). The compact also includes $6.2 million for an
    institutional strengthening program to increase the capacity of the
    Vanuatu Public Works Department (PWD) to maintain transportation
    infrastructure.^12 The remaining $5 million is for program management and
    monitoring and evaluation. More than half of the compact, $37 million, is
    budgeted for three road projects on Santo and Efate islands. On both
    islands, the compact will upgrade existing roads, while on Santo the
    compact also includes five new bridges on an existing road. To oversee and
    manage the compact programs, the Vanuatu government has established
    MCA-Vanuatu, an independent entity housed within the Vanuatu Ministry of
    Finance and Economic Management.

    ^12The institutional strengthening program includes $5.74 million for
    equipment purchases; of this amount, $1.4 million is provided directly to
    PWD and the remainder will purchase equipment for the use of the MCC
    construction contractor, to be turned over to the PWD in specified
    condition 4 years later.

    Figure 3: MCC Vanuatu Projects by Size and Location

    The compact and the congressional notification state that the compact will
    have a transformational impact on Vanuatu's economic development,
    increasing average per capita income by approximately $200--15 percent--by
    2010 and increasing total GDP by "an additional 3 percent a year." The
    investment memo further quantifies the per capita income increase as
    $488--37 percent--by 2015. The compact and the congressional notification
    further state that the compact will provide benefits to approximately
    65,000 poor, rural inhabitants (see fig. 4).

    Figure 4: MCC Statement of Impacts in March 2006 Congressional
    Notification

    MCC Projected Compact's Impact Using Estimates of Benefits, Costs, and Catchment
    Area

    To project the Vanuatu compact program's benefits, costs, and
    beneficiaries and calculate the compact's impact on per capita income,
    GDP, and poverty reduction, MCC made site visits to Vanuatu and reviewed
    available documents to gather needed data. MCC determined that investments
    in transportation infrastructure will lead to increases in incomes and
    used the available data and made assumptions to determine the compact's
    benefits and costs. MCC also estimated the number of beneficiaries within
    a defined catchment area--the geographic area in which benefits may be
    expected to accrue. MCC then used the benefits and costs to calculate
    summary statistics about the compact's ERR and effects on Vanuatu's GDP
    and per capita income.

    MCC Calculated Benefits, Costs, and Beneficiaries

    To prepare its economic analyses of the compact program's benefits, costs,
    and number of beneficiaries, MCC sent a team of employees and contractors
    to Vanuatu. The contractor MCC retained had previously served as a
    consultant to the ADB in Vanuatu and had prepared a multiyear study of
    projects proposed to the ADB as part of its Vanuatu Outer Islands
    Infrastructure Development Project in 2000 to 2003. Seven of MCC's 11
    transportation infrastructure projects were initially developed as part of
    the ADB study. While in Vanuatu, the MCC team met with officials
    representing other donors, Vanuatu's Department of Economic and Sectoral
    Planning, the National Statistics Office, and the Vanuatu PWD, among
    others. The MCC team also met with a number of interested and informed
    parties representing the transportation, tourism, and agriculture sectors
    and reviewed a number of documents to gather the information needed for
    its economic analyses.^13

      Benefits

    MCC's compact aims to benefit poor, rural agricultural producers and
    providers of tourism-related goods and services by reducing transportation
    costs and improving the reliability of access to transportation services.
    MCC's logic model for the compact posits that improvements in
    infrastructure will lead to (1) direct benefits from increased
    transportation reliability, construction spending, and reduced
    transportation costs and (2) induced benefits from greater tourism and
    agricultural trade. The direct and induced benefits in turn lead to
    increases in per capita income and GDP and reduction in poverty (see fig.
    5).

    ^13MCC used Vanuatu statistical surveys such as the Vanuatu 1999 Household
    Income and Expenditure Survey, 1993 Agricultural Survey, 2004 Tourist
    Survey, and a monthly Hotel Occupancy Survey as baseline data sources.
    Some data were updated to 2005 values from earlier data based on growth
    factors. MCC assessed the data it used as the best available, but noted
    that some data have shortcomings. Other data--such as the amount of
    spending by expatriates in Vanuatu, and the percentage of tourist spending
    that remains in the country--were not available, and MCC had to estimate
    their value. Since signing the compact with Vanuatu, MCC has funded
    additional surveys to improve the quality of baseline data and track key
    compact performance indicators.

    Figure 5: MCC's Logic Model for the Vanuatu Compact

    MCC calculated several different direct and induced benefits for the
    compact's projects over a 20-year period. Direct benefits include the
    local value added of construction spending in the economy, reduced
    spoilage of agricultural goods, time saved in transit on the improved
    roads, and reduced user costs for operators of ships, aircraft, and
    vehicles.^14 Induced benefits are those projected to result from Vanuatu's
    response to new economic opportunities in tourism and agriculture. For
    example, MCC assumes that the projects will cause Vanuatu tourism to grow
    at 7 percent per year instead of the recent rate of 5 percent; tourists
    will increase their daily spending; new hotel rooms will be constructed
    and hotel occupancy will increase; and crop, livestock, and fisheries
    production will increase. Benefits from the completed projects are counted
    as beginning in full in 2008 or 2009 and are assumed to increase a minimum
    of 3 percent every year thereafter. MCC expects the benefits of the
    compact to flow from different sources, depending on the project and its
    location. In Efate, the Ring Road is expected to provide direct benefits
    from decreased road user costs and induced benefits through tourism and
    foreign resident spending. In Santo, MCC anticipates benefits from all of
    these efforts, as well as the induced benefit of increased agricultural
    production. On other islands, where tourism is not as developed, the
    benefits primarily derive from user cost savings and increased
    agriculture.^15

    ^14MCC determined reduced road user costs by means of (1) traffic counts
    prepared by Vanuatu's PWD and MCA-Vanuatu for the MCC and (2) an
    estimation of road user costs per kilometer based on the Highway Design
    and Maintenance Standards Model (HDM)-III developed by the World Bank.
    HDM-III is a software package that provides the economic decision criteria
    for evaluating road construction or maintenance strategies. MCC determined
    the costs to road users by multiplying the average annual daily traffic on
    the road by the length of the road and by the average road user cost per
    kilometer. Road user cost includes vehicle operating costs and the value
    of passengers' time. The difference in user cost before and after
    construction provides the benefit.

      Costs

    To calculate compact construction and maintenance costs, MCC used existing
    data from previous contractor studies and data from the Vanuatu PWD. In
    MCC's economic model, construction costs are assumed to be incurred in the
    first year after compact signing. Although listed on the cost side of the
    model, MCC also counted the effect of the construction spending on the
    economy as a benefit in the year it takes place.^16 According to MCC's
    contractor, it used two different primary sources in developing the MCC
    cost estimates:

               o The contractor used its own 2003 analysis for the ADB projects
               and updated its analysis to account for inflation and changes in
               project scope.

               o The contractor used a cost analysis prepared by a different
               contractor for the government of Vanuatu, dated approximately
               2004, as a basis for MCC's cost estimates for the Santo and Efate
               projects. MCC's contractor then prepared a new estimate because
               MCC's projects had a different scope of work. We requested from
               MCC a copy of the original report to the government of Vanuatu.
               According to MCC officials, MCC did not have it and the government
               of Vanuatu was not willing to provide the report to MCC for our
               review. These Santo and Efate projects represent 56 percent of
               compact cost.

    ^15Although not included in the economic analyses, other benefits may
    accrue to Vanuatu from the MCC compact. Increased economic activity in
    tourism may have spillover benefits for other sectors of the economy, and
    the welfare of Vanuatu's citizens may improve due to increased access to
    health care and educational opportunities.

    ^16MCC assumed that 16 percent of the total cost of construction would be
    captured in the local economy and counted this as a benefit.

               MCC assumed a cost for continued annual maintenance based on past
               estimated maintenance costs provided by the PWD. MCC applied
               assumptions and estimated maintenance costs based on incomplete
               information, because the PWD did not track maintenance costs in
               its budget. For some projects, MCC assumed that maintenance costs
               would decrease once the construction project was complete. MCC
               also estimated and included the cost of a periodic major
               reinvestment for some projects. For example, on the Efate Ring
               Road, MCC assumed a cost for rehabilitation in 2017 and 2026.

               Beneficiaries

               MCC assessed the number of poor, rural beneficiaries by
               determining the catchment area--the geographic area in which
               benefits may be expected to accrue. For example, MCA-Vanuatu
               officials told us that they defined the catchment area in Efate as
               consisting of villages within 3 miles of the rehabilitated Efate
               Ring Road. MCC used Vanuatu maps to identify villages in the
               catchment area and used the 1999 Vanuatu National Population and
               Housing Census to determine the number of persons living in those
               villages. In all, MCC calculated that approximately 65,000 poor,
               rural people on the eight islands would benefit from MCC projects.

    			  MCC Used Projected Benefits and Costs to Determine ERR and Effects
    			  on GDP and Per Capita Income

               MCC used its projection of costs and benefits over a 20-year
               period as the basis for calculating three summaries of the
               compact's impact: its ERR, effect on per capita income, and effect
               on GDP (see fig. 6). The compact's ERR reflects the ratio of the
               benefits of the compact in relation to its costs, expressed as a
               percentage. For the Vanuatu compact, MCC reported an overall
               compact ERR of 24.7 percent over 20 years.^17

    ^17In its final April 2006 economic analysis, MCC adjusted this
    calculation downward slightly to 24.2 percent.

               Figure 6: MCC Benefits Calculations as Source of Statements about
               Impact of Vanuatu Compact

               MCC also prepared a sensitivity analysis to assess how a range of
               possible outcomes would affect compact results. MCC's tests
               included

               o a 1-year delay of the start date for accrued benefits;

               o a 20 percent increase of all costs;

               o a 20 percent decrease of all benefits; and

               o a "stress test," with a 20 percent increase of all costs and a
               20 percent decrease of all benefits.

               MCC tested a best-case scenario based on a 10 percent increase in
               benefits and a 10 percent decrease in costs. For the overall
               compact, MCC calculated a best-case ERR of 30.2 percent and
               worst-case of 13.9 percent. MCC also used the cost and benefit
               data for the compact projects to determine the projects' expected
               impacts on Vanuatu's per capita income and GDP.

    			  MCC's Data Do Not Support Its Portrayal of Compact Benefits

               The anticipated benefits of the Vanuatu compact that MCC stated in
               March 2006 in the compact and in its Congressional Notification do
               not accurately reflect MCC's supporting data. MCC projects an
               increase in per capita income of $200 by 2010, but this increase
               is cumulative; MCC's underlying analysis shows a projected
               increase of $51 in 2010. Similarly, while MCC's statement about
               the compact's transformational effect on GDP could be interpreted
               as increasing the growth rate, the projected effect on GDP
               represents an increase to the level, not the growth rate, of GDP.
               The supporting data show the growth rate remaining roughly the
               same as it would be without the compact. In addition, the 65,000
               rural poor cited by MCC may share less than half of the compact's
               benefits with others in Vanuatu who are not poor or not rural. In
               addition, although MCC's estimates of project benefits and
               beneficiaries are generally reasonable, MCC made some calculation
               errors and questionable assumptions in developing these estimates.
               Correcting MCC's calculations of the per capita income benefit
               slightly reduces the benefit to $49, or 3.7 percent, in 2010;
               correcting calculations and fully discounting some of MCC's
               assumptions regarding beneficiaries would reduce the estimate of
               beneficiaries on Santo and Efate by about one-third.

    			  Portrayal of Per Capita Income Benefit Suggests Larger Effect
    			  Than Data Support

               MCC's portrayal of the compact's projected impact suggests a
               greater effect on Vanuatu's per capita income than its analysis
               supports. In its publicly available documents such as the Vanuatu
               compact, and the congressional notification, issued in March 2006,
               MCC states that the transportation infrastructure project "is
               expected to have a transformational impact . . . increasing
               average income per capita (in real terms) by approximately $200,
               or 15 percent of current income per capita, by 2010." In addition,
               MCC's investment memo states that the compact will cause per
               capita income to increase by $488, or 37 percent, by 2015.

               MCC's statements suggest that as a result of the program, average
               incomes in Vanuatu will be 15 percent higher in 2010 and 37
               percent higher in 2015 than they would be without the compact.
               However, MCC's underlying data show that these percentages
               represent the sum of increases from per capita income in 2005 that
               MCC projects for each year. For example, according to MCC's data,
               Vanuatu's per capita income in a given year between 2006 and 2010
               will range from about 2 percent to almost 4 percent higher than in
               2005; however, MCC sums these percentages as 15 percent, without
               stating that this percentage is a cumulative increase from 2005.
               Our analysis of MCC's data shows that actual gains in per capita
               income, relative to income in 2005, would be $51, or 3.9 percent,
               in 2010 and $61, or 4.6 percent, in 2015 (see fig. 7).

    Figure 7: Vanuatu Compact's Projected Impact on Real Per Capita Income
    According to MCC Statement and MCC Data Relative to 2005 Per Capita Income

    Note: MCC's statement: "Increasing average income per capita (in real
    terms) by approximately $200 or 15 percent of current income per capita by
    2010" and by $488--37 percent--by 2015.

    Figure 8 further illustrates MCC's methodology in projecting the compact's
    impact on per capita income levels for 2010 and 2015.

    Figure 8: MCC Methodology for Projecting Vanuatu Compact's Impact on Real
    Per Capita Income

    Portrayal of GDP Benefit Differs from Underlying Analysis

    Like its portrayal of the per capita income benefit, MCC's portrayal of
    the program's GDP impact differs from that supported by the underlying
    data. In the compact and the 2006 Congressional Notification, MCC states
    that the compact will have a transformational effect on Vanuatu's economy,
    causing GDP to "increase by an additional 3 percent a year." Given the GDP
    growth rate of about 3 percent that MCC expects in Vanuatu without the
    compact, MCC's statement of a transformational effect suggests that the
    GDP growth rate will rise to about 6 percent. However, MCC's underlying
    data show that although Vanuatu's GDP growth rate will rise to about 6
    percent in 2007, in subsequent years the GDP growth rate will revert to
    roughly the rate MCC assumes would occur without the compact,
    approximately 3 percent (see fig. 9). Although MCC's data show that the
    compact will result in a higher level (i.e., dollar value) of GDP, the
    data do not show a transformational increase to the GDP growth rate.

    Figure 9: Vanuatu GDP Growth with and without MCC Compact

    Notes:

    According to MCC, "GDP is expected to increase by an additional 3 percent
    a year as a result of the MCA program."

    According to MCC data, the compact will have a small impact on GDP growth
    rate in later years. In 2010 to 2015, the GDP growth rate resulting from
    the compact will be 3.1 percent, compared with 3 percent without the
    compact.

    Portrayal of Poverty Reduction Does Not Quantify Benefits to Rural Poor

    MCC's portrayal of the compact's projected impact on poverty does not
    identify the proportion of the financial benefits that will accrue to the
    rural poor. In the compact and the Congressional Notification, MCC states
    that the program is expected to benefit "approximately 65,000 poor, rural
    inhabitants living nearby and using the roads to access markets and social
    services." However, in its underlying documentation, MCC specifies that 43
    percent of the monetary benefits are expected to go to the local
    population. The remaining 57 percent of the benefits are expected to
    accrue to other beneficiaries, including expatriate tourism services
    providers, transport providers, government, and local businesses (see fig.
    10). Given that many of MCC's benefits flow to the expatriate-dominated
    tourism sector, MCC makes an appropriate assumption that the local
    population will not receive all the benefits. However, the compact,
    congressional notifications, and other publicly available MCC documents do
    not provide this information.

    Figure 10: MCC Analysis of Distribution of Vanuatu Compact Benefits

    Note: MCC defines "local population" as comprising local producers, local
    consumers, and inhabitants of rural communities.

    Although MCC expects that 43 percent of monetary benefits will go to the
    local population, MCC does not establish the proportion of
    local-population benefits that will go to the rural poor. Because MCC
    defines the local population as "local producers, local consumers and
    inhabitants of remote communities," the 65,000 poor, rural beneficiaries
    that MCC projects may share local-population benefits with those who are
    urban and are not poor.

    Estimates of Compact Benefits Are Generally Reasonable, but Some Calculations
    and Assumptions Are Problematic

    MCC's documentation states that it used conservative assumptions in
    developing its benefit estimates. Although our fieldwork and meetings in
    Vanuatu generally affirmed MCC's assumptions about benefits, many
    participants in our Vanuatu discussion groups noted that the positive
    impacts of the MCC compact are contingent on other factors, such as the
    development of tourist facilities and activities. However, our review of
    MCC's analyses identified some calculation errors in its determination of
    the compact's impact on per capita income and estimation of the number of
    compact beneficiaries, as well as questionable assumptions in regard to
    the beneficiary population. (See app. III for a detailed discussion of
    these calculation errors and the effect of MCC's assumptions.)

               o Calculation errors. To determine per capita income effects, MCC
               incorrectly adjusted the value of a benefit stream for inflation
               that was already presented in real (i.e., inflation-adjusted)
               terms. In addition, MCC failed to account for population growth in
               projecting per capita income effects from the compact. Although
               these errors largely cancel each other, correcting them reduces
               MCC's projection of the per capita income benefit slightly, from
               $51 to $49 in 2010 and from $61 to $57 in 2015. Correcting
               calculation errors significantly increases the number of rural
               beneficiaries; MCC's estimate of compact beneficiaries in Santo
               and Efate did not take into account population growth since the
               1999 census.

               o Assumptions. Our analysis of MCC's data for Santo and Efate
               indicates that MCC's count of beneficiaries may be overestimated.
               MCC's count of poor, rural beneficiaries includes all rural
               inhabitants in the catchment area, indicating that MCC assumes
               that all rural inhabitants are poor. However, according to ADB
               reporting, rural poverty in Vanuatu is widespread but not
               universal. The poverty level, defined as having an income of one
               U.S. dollar per day, is 51 percent in rural areas.^18 Further, in
               defining the catchment areas in Efate and Santo, MCC assumed that
               residents of villages near existing paved portions of the Ring
               Road not improved by MCC, as well as those on off-shore islets,
               would benefit fully from the compact.

               Correcting calculation errors and fully discounting MCC's
               assumption including these residents in the catchment area would
               reduce the beneficiary count on Efate and Santo by 32
               percent--from 26,553, as stated by MCC, to 18,070--indicating that
               MCC may overestimate the compact's beneficiaries.

    ^18In its Vanuatu Monitoring and Evaluation Plan, MCC is using the poverty
    measure proposed by Vanuatu, the fraction of individuals with monthly cash
    income less than 20,000 vatu (approximately $185 U.S. dollars or $6 per
    day).

               Several Risks May Lead to Reduced Project Benefits

               We identified five key risks that may affect the Vanuatu compact's
               projected impact on poverty reduction and economic growth. First,
               the contingencies included in MCC's calculations of construction
               costs may not be sufficient to cover average transportation
               project overruns. Second, although the compact's benefits are
               projected to begin shortly after completion of the projects, some
               of these benefits are likely to accrue more slowly. Third, while
               the benefit projections assume continued maintenance of completed
               projects, MCC's ability to ensure such maintenance will end in
               2011, and Vanuatu's record of road maintenance is poor. Fourth,
               the projected induced benefits from expanded tourism and
               agriculture depend on businesses and rural inhabitants responding
               to opportunities created by improved infrastructure. Fifth,
               efficiency gains, such as time saved in transit, may not result in
               increased per capita income, as MCC projects. Our analysis of
               these areas of risk illustrates the extent to which MCC's
               projections of benefits are dependent on assumptions of immediate
               realization of benefits, successful long-term maintenance,
               realization of induced benefits, and benefits from efficiency
               gains.^19

    			  Construction Costs May Exceed Contingencies

               Although MCC considered the risk of construction cost increases,
               the contingencies used in its calculations may not be sufficient
               to cover actual construction costs. We received documentation from
               MCC's contractor of its MCC cost estimates for only 5 of MCC's 11
               construction projects. For these five, it used a design
               contingency of 20 percent.^20 In its due diligence book, MCC
               states that its cost estimates include physical contingencies,
               with an average value of 15 percent, and price contingencies, with
               an average value of 12 percent. However, cost overruns of more
               than 20 percent occur in many transportation projects.^21 For
               example, a study of more than 250 transportation projects in
               Europe, North America, and elsewhere found that costs for all
               projects were 28 percent higher, on average, than forecasted at
               the time of decision to build, while road projects averaged
               escalations of 20.4 percent.^22 Further, as MCC's analysis notes,
               the risk of excessive cost overruns is significant in a small
               country such as Vanuatu, because (1) few comparable projects have
               been undertaken in Vanuatu and, consequently, little previous cost
               information is available and (2) no local contractors are capable
               of undertaking a project of this size, and foreign contractors may
               apply large margins to cover unknown factors.^23 Any construction
               cost overrun must be made up within the Vanuatu compact budget by
               reducing the scope, and therefore the benefits, of the compact
               projects.^24 Reduced project benefits would in turn reduce the
               compact's ERR and effects on per capita income and GDP.

    ^19We modeled risk scenarios with alternative methods of looking at
    compact benefits that assume phasing of costs and benefits and phasing
    coupled with lack of maintenance, lack of induced benefits, and lack of
    monetized efficiency gains. See appendix V for more information about our
    methodology.

    ^20For one of these five, the $400,000 South West Bay airstrip project,
    the estimate also included an additional 10 percent construction
    contingency.

               In addition, because MCC was unable to provide the data that it
               used to produce the estimates for the Santo and Efate projects,
               the project costs may be at further risk from unchecked
               assumptions and data (see app. IV).

    ^21GAO recently reported this occurring as part of U.S. Agency for
    International Development (USAID)-funded road construction in a
    tsunami-affected area of Indonesia. In Indonesia, construction cost per
    mile increased by 75 percent, USAID reduced the length of road to be built
    by more than one third and the agency may extend the planned completion
    date by 5 months. GAO, Foreign Assistance: USAID Signature Tsunami
    Reconstruction Efforts in Indonesia and Sri Lanka Exceed Initial Cost and
    Schedule Estimates and Face Further Risks, [36]GAO-07-357 (Washington,
    D.C.: February 28, 2007).

    ^22Bent Flyvbjerg, Mette Skamris Holm, and Soren Buhl, "Underestimating
    Costs in Public Works Projects: Error or Lie?," Journal of the American
    Planning Association, Vol. 68, No. 3 (2002), cited in GAO, Highway and
    Transit Investments: Options for Improving Information on Projects'
    Benefits and Costs and Increasing Accountability for Results,
    [37]GAO-05-172 (Washington, D.C.: January 24, 2005).

    ^23MCC cites the "design-construct" contract proposed for the MCA program,
    which will include design and construction of all the projects as one
    package, as key to mitigating this risk. However, MCC's analysis also
    recognized that nonconstruction-related issues (such as access to parts of
    the project site) have the potential to delay the contractor and increase
    costs and that such issues can be significant for major road upgrade
    projects where the competing interests of the contractor, adjacent
    villages, and the general public must be balanced. MCC's analysis states
    that, to help manage the risk of project-related disputes and delays, MCC
    plans to have experienced consultants work with local PWD staff who have
    an understanding of the social and cultural issues.

    ^24According to the compact, the government of Vanuatu must pay any
    environmental mitigation and remediation costs in excess of the budget.

               Compact Benefits Are Likely to Accrue More Slowly Than Projected

               Although MCC's analysis assumes compact benefits from 2008 or
               2009--shortly after the end of project construction--we found that
               benefits are likely to accrue more slowly. Our document review and
               discussions with tourism services providers and agricultural and
               timber producers suggest that these businesses will likely react
               gradually to any increased market opportunities resulting from
               MCC's projects, in part because of constraints to expanding
               economic activity.^25 For example:

               o According to tourism officials and business owners, several
               types of activities need to progress to enable industry growth.
               Factors needed to foster tourism growth in Vanuatu include
               marketing Vanuatu as a tourist destination; promoting different
               types of products to different markets; improving domestic and
               international air and sea access; developing or upgrading the
               electric power, water, and road infrastructures; and recruiting
               and training workers. Tourism service providers in Santo and Efate
               identified improving air capacity as an important need.

               o According to an official from the fisheries department in Santo,
               besides rough roads, the lack of ice-making equipment and lack of
               feed are important barriers to developing Santo's small-scale
               aquaculture.

               o Timber production expands slowly; for example, a timber company
               owner in Santo stated that newly planted trees could not be
               harvested for about 15 years.

               o In both Efate and Santo, feeder roads are in worse condition
               than the project roads and are critical to agriculture
               shipments.^26

               These constraints suggest that future benefits related to tourism
               and agriculture will be phased in once transportation
               infrastructure is improved. Moreover, MCC assumes all construction
               costs will be incurred in the first year, instead of phasing these
               costs over the multiyear construction schedule. Our analysis shows
               that if costs are phased over 3 years and benefits are phased over
               5 years, overall compact ERR declines from 24.2 percent, as
               projected by MCC, to 16.5 percent (see app. V).

    ^25Benefits from construction activities may also be reduced by a delayed
    procurement. MCA-Vanuatu officials initially told us they anticipated
    issuing an invitation for bid to contractors by the end of February 2007.
    As of May 2007, the invitation had not yet been issued. MCC currently
    expects construction to begin in 2008, further reducing the likelihood of
    benefits starting in 2007 as MCC anticipated in its analyses.

    ^26It is unclear whether responsibility for feeder roads lies with the
    government at the central or provincial level; neither takes
    responsibility. The Santo PWD expects to focus on maintaining feeder roads
    once the East Coast Road is done.

               Project Maintenance after Compact Expiration Cannot Be Ensured

               Uncertainty about the maintenance of completed transportation
               infrastructure projects after 2011 may affect the compact's
               projected benefits. According to World Bank and ADB officials,
               continuing donor involvement is needed to ensure the maintenance
               and sustainability of completed projects. In addition, during our
               visits to Efate and Santo, tourism and agriculture business
               representatives cited continued road maintenance as a critical
               concern. However, although MCC has budgeted $6.2 million for
               institutional strengthening of the Vanuatu PWD, MCC has no means
               of ensuring the maintenance of completed projects after the
               compact expires in 2011; the Millennium Challenge Act limits
               compacts to 5 years.

               Although the conditions precedent^27 to the Vanuatu compact
               require the government's commitment to ongoing project
               maintenance, World Bank and ADB officials told us, based on their
               experience with Pacific countries, that covenants such as these
               are difficult to enforce after the project is completed and often
               are not effective in ensuring sustainability. According to Vanuatu
               government officials, funds from vehicle registration and related
               fees are available for maintenance of the completed projects;
               however, the government has not dedicated these funds for this
               purpose. In addition, according to donor reporting, the government
               has failed to sustain maintenance of previous donor projects,
               primarily because of a lack of funds and a shortage of resources,
               skills, and capabilities. Some of the department's equipment on
               Santo Island is more than 27 years old, and newer equipment
               remains idle for long periods because spare parts are scarce;
               overall, the department lacks adequate equipment for Santo's
               1,000-kilometer road network. Although the compact provides funds
               for equipment and for technical assistance to increase capacity,
               MCC cannot ensure the maintenance of infrastructure after the
               compact ends.

    ^27Conditions precedent are specific steps that must be completed by the
    compact country prior to MCC's providing a disbursement. According to MCC
    officials, the government has thus far met or exceeded the requirements of
    these conditions.

               Poor maintenance performance will reduce the benefits projected in
               the MCC compact.^28 Our analysis shows that with phasing of costs
               and benefits, no large periodic maintenance expenditures, and
               inadequate maintenance performance, overall compact ERR decreases
               from 24.2 percent, as projected by MCC, to 16.5 percent with
               phasing and to 13.8 percent without maintenance.^29

    			  Induced Benefits Require Tourism Providers and Agricultural
    			  Producers to Respond to Opportunities

               The compact's induced benefits depend on the response of Vanuatu
               tourism providers and agricultural producers. However, constraints
               affecting these economic sectors--such as the air-passenger
               capacity limitations that affect Vanuatu's tourism--may prevent
               the sector from expanding as MCC projects, affecting the
               reliability of MCC's estimate of program benefits.

               Some development assistance organizations do not include induced
               benefits in calculations of ERR. In particular, World Bank
               officials told us they have not considered induced benefits in
               economic analyses of infrastructure for 10 years, because they
               deem such benefits too conjectural and subject to manipulation.^30
               For example, if a projected road is assumed to transform an area's
               agricultural production, it is easy to show large benefits and
               high ERR.^31

    ^28In technical comments on a draft of this report, the World Bank noted
    that initial construction quality and subsequent traffic can affect a
    road's projected benefits and, if not included in the investment analysis,
    can have important implications for the actual ERR that may be equal to,
    or larger than, the impact of poor maintenance. The World Bank also noted
    that weak governance and political corruption can also have a large impact
    on a road's sustainability.

    ^29This simulation results in a modest impact on ERR because it reflects
    both a reduction in future benefits and a reduction in future costs for
    road maintenance.

    ^30In technical comments on a draft of this report, the World Bank noted
    that it measures induced benefits directly, comparing base traffic levels
    with expected growth in demand for transport as a result of investments.
    This method results in relatively straightforward and transparent
    calculations. The World Bank observed that models using other, more
    indirect approaches to estimating traffic, such as examining the impact of
    increased agricultural production, require careful consideration and that
    it has generally used such methods less often over the last 10 years
    because the results are sometimes unclear and sensitive to assumptions.
    Typically, it uses such methods only when a road project is expected to
    have significant restructuring effects--such as a completely new road or
    bridge in an urban environment--that could drastically change land use
    patterns.

               Limited response to the compact by tourism providers and
               agricultural producers would have a significant impact on compact
               benefits. For example, our analysis shows that the phasing of
               costs and benefits reduces the overall compact ERR from 24.2
               percent to 16.5 percent, and the omission of induced benefits
               (i.e., no producer response) further reduces the compact ERR to
               5.5 percent. Two projects would have a negative ERR.

    			  Efficiency Gains May Not Cause Measurable Change in Per Capita Income

               MCC counts efficiency gains--such as time saved because of better
               roads--as compact benefits. However, although efficiency gains
               could improve social welfare, they may not lead to changes in per
               capita income or GDP or be directly measurable as net additions to
               the economy.

               Excluding efficiency gains from ERR calculations reduces the
               compact's overall ERR. For example, our analysis shows that the
               phasing of costs and benefits and the omission of efficiency gains
               from the Vanuatu ERR calculation causes the overall compact ERR to
               decline from 24.2 percent to 16.5 percent with phasing and to 11.8
               percent without calculating efficiency gains. This analysis also
               results in a negative ERR for two projects (see app. V).^32

    			  Conclusions

               MCC obtained the input of knowledgeable stakeholders in projecting
               the benefits, costs, and number of beneficiaries of its compact
               with Vanuatu, which addresses one of Vanuatu's primary constraints
               to economic growth. However, MCC's public statements about these
               benefits--particularly its projection of the compact's effect on
               per capita income--suggest greater impacts than MCC's underlying
               data and analysis support. MCC's statements can be understood only
               by reviewing supporting source documents and spreadsheets, which
               are not publicly available. These gaps could lead to unrealistic
               expectations about the compact's effect within Vanuatu; for
               example, by suggesting that per capita incomes will increase so
               quickly, MCC suggests that its compact will achieve sustainable
               growth in a way that other donors to Vanuatu have not been able to
               achieve. Further, these gaps between MCC's statements and
               underlying analysis raise questions about other compacts'
               projections of transformative impact on country economies or
               economic sectors. Without an accurate representation of the
               compacts' projected benefits, the extent to which the compacts
               further MCC's goals of poverty reduction, economic growth, and
               transformative development cannot be accurately evaluated.
               Further, MCC's economic analyses for the Vanuatu compact did not
               fully consider the phasing of costs and benefits, such as the time
               required to improve the infrastructure and for the economy to
               respond to the opportunities from the improved infrastructure.
               Additionally, MCC's analysis did not fully account for risks that
               could substantially reduce compact benefits.

    ^31However, the lack of consideration of induced benefits is not without
    its flaws. World Bank officials told us that in the fast-growing economy
    of Vietnam, the World Bank funded a road on the basis of current use of
    traffic without projecting the gains to the economy. The road is now very
    congested, and the project was underinvested. According to the World Bank,
    consideration of projected traffic increases--that is, induced
    traffic--are taken into account only when the project is not justified on
    the basis of existing traffic alone. However, rapid traffic growth is
    considered exceptional.

    ^32The ERR in this case does not take into account opportunity costs and
    is therefore an internal rate of return.

               Recommendations

               We recommend that the CEO of MCC take the following actions:

               o revise the public reporting of the Vanuatu compact's projected
               impact to clearly represent the underlying data and analysis;

               o assess whether similar statements in other compacts accurately
               reflect the underlying data and analysis; and

               o improve economic analysis by phasing the costs and benefits in
               compact ERR calculations and by more fully accounting for risks
               such as those related to continuing maintenance, induced benefits,
               and monetized efficiency gains as part of sensitivity analysis.

    			  Agency Comments and Our Evaluation

               MCC provided written comments regarding a draft of this report. We
               have reprinted MCC's comments, with our responses, in appendix VI
               and incorporated technical comments from MCC where appropriate.
               Although MCC did not directly acknowledge our recommendations, it
               questioned our finding of a gap between its portrayal of the
               compact's benefits and its underlying analysis. In addition, MCC
               responded to our discussion of beneficiary numbers and program
               risks. Following is a summary of MCC's comments and our evaluation
               of these comments.

    			  Compact Impact on Incomes and Growth

               MCC acknowledged that its use of projected cumulative compact
               impact on income and growth was misleading, but it asserted that
               (1) it had no intention to mislead and (2) its portrayal of
               projected compact benefits was factually correct and consistent
               with its underlying data. Our report does not state or imply that
               MCC intended to mislead; we have not determined why MCC made its
               statements portraying projected benefits on income and growth.
               Further, MCC's portrayal of the projected benefits could be
               considered factually correct and consistent with the underlying
               data only if the reader knows that the portrayal represents
               cumulative income and growth over 5 years. However, MCC's public
               documents and communications to Congress never describe the
               benefits it projects as being cumulative over 5 years. At issue in
               this finding are both the facts represented by MCC's data and
               calculations and MCC's representation of these data and
               calculations in its public statements. As our report notes, MCC's
               statements can be understood accurately only by reviewing
               supporting source documents and spreadsheets, which are not
               publicly available.

               In addition, MCC's portrayal of the compact's effect in cumulative
               terms is not consistent with its description of per capita income
               in Vanuatu in annex I of the compact, which states that per capita
               income in Vanuatu declined by 15.4 percent between 1994 and 2003.
               This calculation is consistent with the World Bank's data, which
               show Vanuatu's per capita income in 2003 as approximately 15.4
               percent less than its per capita income in 1994--a comparison of
               2003 and 1994 income levels, not a reflection of cumulative
               decline. If the methodology MCC uses to describe past per capita
               income in Vanuatu is used to describe the compact's projected
               future impact, then the projected increase in Vanuatu's per capita
               income in 2010 should be portrayed as 3.9 percent, not the
               cumulative 15 percent that MCC presents in its congressional
               notifications and public documents.

               MCC also commented that its data on the compact's effect on
               Vanuatu's GDP are consistent with its assessment that GDP will be
               perpetually 3 percent higher with the MCC investment than without
               it. We agree that MCC's data reflect this, but MCC's portrayal of
               this benefit in its public statements implies an effect on the
               growth rate, rather than the level, of Vanuatu's GDP. MCC couples
               its portrayal of the GDP benefit with a description of the
               compact's impact as "transformational"--that is, as MCC defines
               the term, having "a dramatic and long-lasting impact on poverty
               reduction through sustainable economic growth." However, although
               MCC's data show projected benefits to Vanuatu's economy, including
               an increase in the level of GDP, the projected change to the
               overall Vanuatu economy, as reflected by the GDP growth rate, is
               smaller than 3 percent. In 2010-2015, for example, the GDP growth
               rate increase resulting from the compact will be 0.1 percent.

    			  Compact Beneficiaries

               MCC commented that our report suggests that (1) nonpoor households
               could feasibly be excluded from the count of beneficiaries, (2)
               persons living on off-shore islets and in the hinterland should
               not be counted as beneficiaries, and (3) MCC's definition of the
               local beneficiary population includes urban dwellers. Regarding
               nonpoor households, we do not suggest that they could be excluded
               or that this would be desirable. Our report indicates that MCC's
               analysis shows there will be poor, rural beneficiaries as well as
               other beneficiaries and that MCC has not quantified what portion
               of the compact's benefits poor, rural beneficiaries will receive.
               Regarding off-shore and hinterland beneficiaries, we do question
               whether inhabitants of off-shore islets and of villages more than
               10 kilometers from MCC projects on Santo and Efate would benefit
               fully from MCC's projects. However, these persons are less than
               half of the total number we question. In Santo and Efate, 51
               percent of the catchment area population are not inhabitants of
               the hinterland or off-shore islets, but rather residents of Efate,
               near the tourism center and capital of Port Vila and along a
               portion of the Ring Road that is already paved. It is not clear
               how construction of a road elsewhere on the island will benefit
               these residents when they already have access to Port Vila via the
               existing paved road. Regarding the definition of "local
               population"--which MCC's underlying documentation says will
               receive 43 percent of compact benefits--MCC's written comments
               asserted that its definition of this population does not include
               urban dwellers. Although we asked MCC to support this assertion,
               MCC did not provide any additional documents. In separate
               comments, MCC added that although its documentation does not
               contain a very detailed reference specifically saying exactly how
               many business people, transport providers and tourism operators
               are foreigners and urban, MCC believes it reasonable to conclude
               that urban dwellers are included in these categories. However, we
               note that not every urban dweller is a business person, transport
               provider, or tourism operator and that some urban beneficiaries
               would therefore fall into the "local population" that receives an
               estimated 43 percent of benefits. Further, MCC's definition of the
               term in its internal documents includes persons who are not poor
               and not rural.

    			  Program Risks

               MCC commented that it undertakes sensitivity analysis for all of
               its ERR calculations, although it may focus on risks other than
               those we identified. In addition, MCC states that it differs
               significantly from us regarding the precise nature and severity of
               program risks in general. As our report notes, we offer several
               alternative scenarios to illustrate the maximum impact of certain
               areas of risk on projected compact benefits. The scenarios look at
               the effect of individual risk areas, rather than the possibility
               of projects' failing to achieve certain benefits in multiple areas
               simultaneously. Regarding induced benefits and efficiency gains,
               we note that MCC's documentation of its economic analysis
               questions the valuing of efficiency gains as induced benefits to
               the Vanuatu economy. Regarding maintenance, MCC states that its
               compact provides a significant amount of money to assist the
               Vanuatu PWD in providing timely and adequate maintenance. However,
               as our report notes, Vanuatu's record of maintenance is poor, and
               achieving a change in maintenance performance through donor
               assistance would show a significant break from past experience.
               Finally, regarding contingencies, MCC questions our use of the
               Flyvbjerg, Holm, and Buhl study as a basis for evaluating its use
               of contingencies and states that it provided us with price and
               contingency information to support its use of contingencies. The
               Flyvbjerg, Holm, and Buhl study and our report use the same basis
               of comparison for evaluating the adequacy of contingencies, and we
               have updated the language of our report to make this clear. In
               addition, as our report notes, we did not receive full
               documentation of the cost estimates and contingencies that MCC
               used in developing its economic analyses. We received no
               supporting documentation for the cost and contingency estimates
               for the Santo and Efate projects, which account for 56 percent of
               the total compact budget. A key document that MCC's contractor
               said it used as a basis for developing the Santo and Efate costs
               was unavailable to us, because the government of Vanuatu would not
               provide it to MCC. Cost overruns remain a risk to project
               benefits, and we have presented them as such.

               As agreed with your offices, unless you publicly announce the
               contents of this report earlier, we plan no further distribution
               until 30 days from the report date. At that time, we will send
               copies of this report to interested congressional committees as
               well as to the CEO of MCC and the Secretary of the Treasury. We
               will make copies available to others on request. In addition, this
               report will be available at no charge on the GAO Web site at
               http://www.gao.gov.

               If you or your staff have any questions about this report, please
               contact David Gootnick at (202) 512-3149 or gootnickd@gao.gov.
               Contact points for our Offices of Congressional Relations and
               Public Affairs may be found on the last page of this report. GAO
               staff who made major contributions to this report are listed in
               appendix VII.

               Sincerely yours,

               David Gootnick
    			  Director
    			  International Affairs and Trade

    			  Appendix I: Objectives, Scope, and Methodology

               At the request of the Chairman of the House Committee on Foreign
               Affairs, we examined the Millennium Challenge Corporation's (MCC)
               economic analyses of the Vanuatu compact. Specifically, we
               examined

               o MCC's methods of projecting the compact's economic benefits and
               methods of calculating the benefits,

               o MCC's portrayal and analysis of the projected benefits, and

               o risks that could affect the compact's impact on poverty
               reduction and economic growth.

               To accomplish our objectives, we reviewed MCC's record of due
               diligence, including its opportunity memo, investment memo, due
               diligence book, economic analysis spreadsheets, and contractor
               reports. These documents are restricted from public dissemination
               based on MCC policy, but MCC made them available to us for
               analysis. We also reviewed the supporting documents used in MCC's
               due diligence process, such as previous donor reports and studies.
               We supplemented this review with interviews with MCC officials in
               Washington, D.C., and MCA-Vanuatu officials in Vanuatu. We then
               used this information to (1) summarize the process that MCC used
               to develop its economic model and (2) identify the logic, data,
               methods, and assumptions used to determine the compact's projected
               costs and benefits, number of beneficiaries, and effects on per
               capita income, gross domestic product (GDP), and poverty.

               We evaluated MCC's statements of compact impact in its public
               documents such as the congressional notification and the compact
               itself--as well as MCC's internal investment memo--by comparing
               these statements with the underlying analyses and data used to
               support them. We analyzed the November 2005 investment memo
               spreadsheet containing the source data and calculations used for
               MCC's statements about per capita income and GDP and reviewed its
               contents and formulas to determine the methodology and source data
               for MCC's statements and to assess the accuracy of the
               calculations. We then used MCC's own data and formulas from the
               November 2005 investment memo to determine MCC's impact in
               individual years and compare the results with the statements made
               in the Vanuatu compact and congressional notification. We could
               not validate most of MCC's underlying data and assumptions,
               because some data, such as construction and maintenance cost data,
               were not available or could not be checked within the time frames
               of our engagement. Further, we used the calculations and formulas
               from MCC's November 2005 investment memo, corrected population
               figures, and the data from MCC's April 2006 final economic
               analysis to recalculate MCC's statements about per capita income
               impacts. We reviewed MCC's economic analysis spreadsheet to find
               the amount of benefits that MCC's analysis assumed would flow to
               poor, rural beneficiaries and reviewed spreadsheets provided by
               MCA-Vanuatu to determine the assumptions, methodology and data
               used to calculate the beneficiary population in Santo and Efate.
               We compared the spreadsheet data with Vanuatu village maps and
               population figures from the 1999 National Population and Housing
               Census provided by MCA-Vanuatu to evaluate MCC's calculations and
               assumptions about the villages included as beneficiaries.

               We identified risks to MCC's compact results based on our review
               of MCC's internal documentation and donor and academic literature.
               We also met with Vanuatu, MCC, and contractor officials and
               interested parties such as tourism and agriculture business owners
               in Vanuatu. We focused our field work on the three projects
               representing more than half of the compact budget and on Vanuatu's
               two most populous islands, Santo and Efate. We examined MCC's due
               diligence documentation and contacted MCC's contractor to
               determine the methodology used to estimate construction costs. We
               then compared MCC's procedures to findings and GAO's best practice
               criteria for cost estimation. We compared MCC's assumptions about
               maintenance continuing for 20 years to previous donor experiences
               gleaned from donor reporting or interviews with the Asian
               Development Bank, Australian Agency for International Development,
               European Union, International Monetary Fund, New Zealand Agency
               for International Development, and World Bank. We determined the
               need for the phasing of benefits based on discussions with
               knowledgeable agriculture and tourism representatives in Vanuatu
               and a review of MCC's source documentation. We further compared
               MCC's treatment of induced benefits to the approach taken by the
               World Bank and MCC's treatment of direct benefits to economic
               principles about the monetization of efficiency gains. We modeled
               risk scenarios that provide alternative methods of looking at
               compact benefits using MCC's April 2006 economic model
               spreadsheet. In modeling these risks, we used the data from MCC's
               economic analyses; we did not validate these data. In these
               scenarios, we determined the effect of the scenario on the
               economic rate of return by assuming the phasing in of costs over 3
               years and of benefits over 5 years, first determining the effect
               of phasing alone, then examining the phasing of costs and benefits
               coupled with the following:

               o Lack of maintenance--eliminating all future large maintenance
               expenditures and taking away a fraction of the growing expected
               benefits each year, so that at the end of the 20-year period,
               2027, benefits basically return to their 2012 level.

               o Lack of induced benefits--deleting those benefits from the cost
               model.

               o Lack of monetized efficiency gains--deleting those benefits from
               the cost model.

    			  Appendix II: Prior Development Assistance to Vanuatu

               Of Pacific island countries, Vanuatu receives among the largest
               amounts of aid; however, Vanuatu's economic performance and social
               development since 1990 have lagged behind other countries in the
               region. Vanuatu depends on donor assistance for its development
               budget; from its gaining independence, in 1980, to 2002, aid
               receipts averaged around 17 percent of GDP. In 1997, Vanuatu
               initiated structural adjustment efforts with ADB assistance
               designed to stimulate the private sector and help realize the
               country's growth potential in agriculture and tourism. However,
               these reforms have had a limited impact on growth in
               private-sector investment and GDP. In 2003, the government of
               Vanuatu prepared the Priorities and Action Agenda to focus
               available resources, including donor assistance, on Vanuatu's
               development priorities. The current strategies of Vanuatu's major
               donors--which include Australia, France, New Zealand, the European
               Union, Japan, and MCC--are aimed at strengthening Vanuatu's
               productive sectors and supporting private sector-led development
               and are based on the Vanuatu government's stated goals. MCC
               anticipates that its average annual assistance level of $13.1
               million over the next 5 years will position the United States as
               one of Vanuatu's top two official donors (see table 1).

               Table 1: Average Annual Contributions of Top Five Donors to
               Vanuatu in 2004 and 2005

    (Dollars in millions)
    Donor                 Average annual contribution
    Australia                                   $19.1
    France                                       $5.3
    New Zealand                                  $4.8
    European Union                               $4.6
    Japan                                        $3.5

               Source: Organization for Economic Cooperation and Development.

               According to the ADB, Vanuatu's development budget has
               concentrated on building new capital infrastructure; however,
               according to donor reporting, these assets have not been
               adequately maintained. From 1996 to 2004, Vanuatu received about
               $35 million in assistance for transportation infrastructure
               projects including the construction and rehabilitation of roads,
               bridges, and wharfs, and the extension and upgrading of airport
               terminals and runways.

               According to studies by the World Bank and ADB, Vanuatu faces
               significant development challenges. Some of Vanuatu's challenges
               are shared by other small states and island economies,^1 while
               others are either the result of, or exacerbated by, political
               instability and, according to ADB, inappropriate policies or
               regulations. Like other small island economies, Vanuatu's
               agriculture and tourism sectors are sensitive to extreme weather
               conditions and natural disasters. For example, frequent cyclones
               cause production shocks and restrict the range of viable crops and
               tree species. Vanuatu's narrow production base also leaves the
               economy open to economic shocks from changes in commodity prices,
               especially for copra. Despite these constraints, Vanuatu has a
               wealth of natural resources and growth opportunities in
               agriculture, fisheries, and tourism. However, growth has been
               hindered by substantial barriers to private-sector development,
               including political uncertainty,^2 high costs of doing business,
               poor and costly infrastructure, lack of a secured transactions
               framework,^3 and issues with land tenure.

    ^1According to the World Bank, some of the characteristics shared by small
    states such as Vanuatu include remoteness and insularity, susceptibility
    to natural disasters, limited institutional capacity, limited
    diversification, reliance on external trade, and limited access to
    external capital. Commonwealth Secretariat/World Bank, Small States:
    Meeting Challenges in the Global Economy (Washington, D.C.: 2000).

    ^2Beginning in the early 1990s, Vanuatu has witnessed a series of
    short-lived coalition governments, comprising numerous parties and
    involving frequent changes of ministers. There have been eleven
    governments in power since 1990, with most lasting only about 1 year. This
    constant turnover has delayed reforms as coordination of policies and
    obtaining consensus for needed reforms has proved difficult. International
    Monetary Fund, Vanuatu: Selected Issues, IMF Country Report No. 07/93
    (Washington, D.C.: 2007).

    ^3The legal framework for secured transactions is incomplete. According to
    the ADB, the legal system does not allow for property to be used as
    collateral for loans, particularly outside of Port Vila and Luganville.
    Thus borrowers can neither purchase property on credit nor obtain loans
    against their own assets. This accounts for high borrowing costs, higher
    interest rates, smaller loans offered, and loans of shorter maturity,
    which limit the private sector's access to credit and entrepreneurship.
    IMF, Vanuatu: Selected Issues.

               Appendix III: MCC Estimates of Per Capita Income Impact
               and Poor, Rural Beneficiaries

    			  Per Capita Income Impact

               In its economic analysis for Vanuatu, MCC did not correctly
               account for population growth in projecting per capita income and
               incorrectly deflated the value of the benefit stream that was
               already in real terms. When the benefit stream is no longer
               deflated, its value increases. However, when population growth is
               accounted for, the per capita income benefit is reduced. MCC
               assumes that the population is growing by approximately 2.6
               percent each year, but MCC's analysis of cumulative income per
               capita benefits for 2006 to 2015 is based on 2005 population
               estimates, rather than on an estimate of population that increases
               each year. Figure 11 contrasts the estimated increase in income
               per capita over the 2005 baseline in the given year with and
               without our corrections to deflating of benefits and to population
               growth. The errors largely cancel out, reducing the per capita
               income benefit projected by MCC from $51 (3.9 percent) to $49 (3.7
               percent) in 2010 and from $61 (4.6 percent) to $57 (4.3 percent)
               in 2015.

               Figure 11: MCC Compact's Projected Impact on Vanuatu Per Capita
               Income with and without Adjusted Population Estimates

               Note: In addition to adjusting for population growth and
               eliminating the deflating of benefits already in real terms, our
               revised projections of per capita income benefits use April 2006
               data from MCC, including a benefit start date of 2007 for
               construction spending, a benefit start date of 2008 for some
               projects, and updated numeric values for costs and benefits.

               The impact of the MCC compact on per capita income is a relatively
               small addition to Vanuatu's expected per capita income without
               MCC. Figure 12 shows, in dollar terms, MCC's projected impact on
               Vanuatu's per capita income compared with the expected levels of
               per capita income, using MCC's assumptions of 3 percent annual
               growth in GDP and 2.6 percent in population.

               Figure 12: MCC Compact's Projected Impact on Vanuatu Per Capita
               Income in Real Terms Relative to Per Capita Income without MCC

               Note: Assumes 0.4 percent growth in per capita income, based on
               MCC's assumed GDP growth of 3 percent without the compact and
               MCC's assumed Vanuatu population growth rate of 2.6 percent.

    			  Calculation of Beneficiaries

               MCC calculated approximately 65,000 beneficiaries on the eight
               islands receiving MCC projects (see table 2).^1 However, MCC's
               determination of beneficiary numbers in Santo and Efate contained
               calculation errors and relied on problematic assumptions that
               villages near portions of the road not improved by MCC, as well as
               those on off-shore islets, would fully benefit from the compact.
               Correcting the calculation and data errors and fully discounting
               MCC's assumptions would reduce the beneficiary count on Efate and
               Santo by 32 percent, from MCC's stated 26,553 to 18,070.

    ^1Before signing the compact, MCC determined the number of potential
    beneficiaries but did not quantify targets for the compact's effect on
    poverty. MCC has funded an additional survey that will provide updated
    baselines and the basis for establishing these targets.

               Table 2: Number of Poor, Rural Beneficiaries of Vanuatu Compact,
               as Calculated by MCC

    Island              Project                        Number of beneficiaries
    Efate               Round Island Road                               13,819
    Santo               Port Olry Road                                   7,404
    Santo               South Coast Road Bridges                         5,330
    Subtotal, Santo and                                                 26,553
    Efate
    Other projects
    Tanna               Whitesands Road                                  6,190
    Malekula            Lits Lits Road                                   5,676
    Malekula            Southwest Bay Airstrip                           4,570
    Pentecost           Loltong Wharf and North-South                    7,858
                           Road
    Epi                 Lamen Bay Wharf                                  1,560
    Malo                Malo Roads Upgrading                             3,480
    Ambae               Ambae Roads Reconstruction                       9,340
    Total                                                               65,227

               Source: GAO analysis of MCC data.

               Our examination of the calculation of MCC's beneficiary estimate
               for the Efate Ring Road, Santo East Coast Road, and Santo South
               Coast Road Bridges shows that MCC's estimate of rural
               beneficiaries in Santo and Efate would be increased by
               approximately 5,100 persons, or 19 percent, if adjustments were
               made to correct data errors^2 (see table 3).

    ^2We identified the following errors: (1) MCC's beneficiary estimates use
    data from the 1999 census and have not been updated to 2005. This results
    in an underestimate of the beneficiary population. (2) In the data
    provided to us, the total of the populations of villages included in the
    MCC analysis does not match the beneficiary count in the monitoring and
    evaluation plan. (3) For the Efate Ring Road project, MCC's data
    double-counted some villages. (4) For all three projects, the 1999
    population figures used for some villages do not match the populations
    shown in the 1999 Vanuatu census. For the Santo East Coast Road, the
    population figures do not match for 53 of 56 villages in the catchment
    area.

    Table 3: Effect of Calculation and Data Errors on Beneficiary Population
    in Efate and Santo

                                                               Santo South
                                         Efate Ring Santo East  Coast Road
                                               Road Coast Road     Bridges  Total
    MCC Statement: beneficiary count
    in monitoring and evaluation plan     13,819      7,404       5,330 26,553
    GAO recalculation of MCC data^a       15,439      7,518       8,732 31,689
    Difference                             1,620        114       3,402  5,136

    Source: GAO analysis of MCC data.

    aGAO's recalculation uses 1999 Vanuatu census data, removes double-counted
    villages, and updates the population to 2005 using MCC's assumed
    population growth factor in rural areas of 2 percent per year.

    MCC also made problematic assumptions about the geographic reach of
    project benefits that increased the number of beneficiaries by including
    more rural areas as benefiting from the project. Fully discounting these
    assumptions would reduce the beneficiary count from our adjusted total of
    31,689 to 18,070 (see table 4).

    Table 4: Effect of MCC Assumptions on Beneficiary Population in Efate and
    Santo

                                                             Santo South
                                       Efate Ring Santo East  Coast Road
                                             Road Coast Road     Bridges    Total
    GAO calculation using MCC data      15,439      7,518       8,732   31,689
    Discounting population of areas    (6,889)          0     (3,222) (10,111)
    not adjacent to MCC road
    project
    Discounting population of          (2,779)      (196)       (533)  (3,508)
    off-shore islets
    Total^a                              5,770      7,322       4,978   18,070

    Source: GAO analysis of MCC data.

    aBecause of rounding, numbers in columns may not sum to totals.

    In estimating beneficiaries of both the Efate Ring Road and the Santo
    South Coast Road Bridges projects, MCC included the populations of
    villages not located near MCC's road projects and populations that face
    geographic barriers to using the projects. These population counts were
    not prorated or adjusted to account for the barriers that some villages
    face in realizing benefits from MCC's projects.

               o In Efate, MCC assumed that the populations of villages located
               along an already improved portion of the road adjacent to Port
               Vila in the Mele, Eratap, and Erakor areas--which will not be
               addressed by the MCC compact--would benefit from construction of
               the road elsewhere on the island (see fig. 13). MCA-Vanuatu stated
               that the populations of these villages were included because they
               own land where the road will be constructed and most of the
               villagers' gardens are located in these areas. (The source for
               this assumption is not documented.) The inhabitants of these
               villages represent 45 percent of the total number (15,439) of
               poor, rural Efate beneficiaries. MCC also did not account for
               "most" villagers and instead included the entire village
               population in these areas in its totals.

               Figure 13: MCC Efate Ring Road Catchment Area

               o For the Santo South Coast Road Bridges, MCC included villages in
               West Santo more than 10 kilometers from the bridge projects and
               villages across a river that vehicles can cross only by fording;
               after heavy rains, the river becomes impassable. MCC's compact
               program does not include a bridge across this river. Using a 2005
               population estimate, we determined that MCC included 3,222 people
               in the count of beneficiaries of South Coast Road Bridges based on
               this assumption--37 percent of the total number (8,732) of South
               Coast Road Bridges beneficiaries.

               For all three projects, MCC assumed that inhabitants of off-shore
               islets would fully benefit, although they are not connected to the
               improved roads and must travel by boat to reach them. MCC
               officials told us that they expect that residents of off-shore
               islets will benefit from increased tourism spending after
               construction of roads and would benefit from decreased
               transportation costs after coming to the mainland. Using 2005
               population estimates, we determined that 2,779 persons (18 percent
               of the Efate Ring Road beneficiary population) live on off-shore
               islets. For the Santo East Coast Road and South Coast Road
               Bridges, respectively, 196 (3 percent of the beneficiary
               population) and 533 (6 percent of the beneficiary population) live
               on off-shore islets.

    			  Appendix IV: MCC Cost Estimates for the Vanuatu Compact

               The reliability of MCC's cost estimates for the Vanuatu compact
               construction cannot be evaluated, because MCC was unable to fully
               provide the data that it used to produce them. MCC's contractor
               based its construction cost estimates for MCC's transportation
               infrastructure activities primarily on two sources. To build cost
               estimates for seven of the compact projects, representing 15
               percent of total compact cost, the contractor used a 2003 analysis
               it had performed for the ADB. To prepare cost estimates for the
               three projects on Santo and Efate, representing 56 percent of
               compact cost, the contractor used estimates prepared by a second
               contractor for the government of Vanuatu.^1

               o For the seven ADB projects, MCC's file of documents used in its
               due diligence included the original cost estimate but did not
               include an analysis that showed how these costs were updated to
               account for inflation and changes in the scope of the work. MCC's
               documentation states that these previous costs were multiplied by
               1.17 to account for inflation and also adjusted for changes in the
               scope.^2

               o For the three projects on Santo and Efate, MCC's file did not
               include any source information or analysis. MCC's contractor later
               told us that it derived the costs of the projects on Santo and
               Efate from an existing report by a second contractor to the
               government of Vanuatu, dated approximately 2004. MCC's contractor
               told us that because MCC's scope of work was more realistic than
               that proposed in 2004, which included realigning portions of the
               roadways, the MCC cost estimate was prepared anew using source
               information from the previous report. We requested, but did not
               receive, a copy of this report to the government of Vanuatu and a
               copy of the analysis showing how MCC developed its cost estimates
               using this source data. According to MCC officials, the government
               of Vanuatu was not willing to provide the report to MCC for our
               review.

               According to previous GAO work, high-quality, reliable cost
               estimates should be accurate, comprehensive, well-documented, and
               validated.^3 MCC cost estimates used in the economic analyses have
               shortcomings in each of these areas.

    ^1The remaining 29 percent of costs are for warehouses, and for oversight
    and administrative functions.

    ^2MCC's contractor provided us with spreadsheet calculations of the costs
    for five of the compact projects for MCC. The sheets included estimated
    quantities and unit costs but did not indicate how these quantities and
    costs were derived.

    ^3We developed these criteria as part of a forthcoming publication on best
    practices in cost estimation and implemented the criteria in a recent
    report. See GAO, Telecommunications: GSA Has Accumulated Adequate Funding
    for Transition to New Contracts but Needs Cost Estimation Policy,
    [38]GAO-07-268 (Washington, D.C.: February 23, 2007).

               o Accurate. The source data for the estimates is from 2003 and
               approximately 2004, and changes in the cost environment beyond
               general inflation may not be accounted for. MCC also incorrectly
               calculated the length of the Santo East Coast Road as 70
               kilometers in its economic analyses, instead of the 55 kilometers
               found by a PWD survey, according to a PWD official. Other elements
               of this criterion cannot be independently assessed without the
               source data for the cost estimates used in the model.

               o Comprehensive. We could not assess the cost estimates against
               this criterion without the source data for the estimates used in
               the model.

               o Well-documented. MCC did not fully document any cost estimation
               procedures and checks that it performed as part of its due
               diligence. The due diligence documents we received from MCC
               contain a narrative describing the cost estimate procedures, but
               no documentation showing the analysis that MCC's contractor
               performed to update previous cost estimates.

               o Validated. MCC's due diligence documentation contains no
               evidence that it obtained an independent cost estimate to verify
               the work of its contractor in updating its own estimate for the
               ADB projects and reworking the estimates of another contractor for
               the Efate and Santo projects. An independent cost estimate
               provides the estimator with an unbiased test of the reasonableness
               of the estimate and reduces the cost risk associated with the
               project by demonstrating that alternate methods generate similar
               results.

    			  Appendix V: Illustrative Alternative Calculations of Vanuatu
    			  Compact Impact

               This appendix describes the results from several alternative
               scenarios, which disaggregate the effects of the various types of
               benefits on the overall ERR, GDP and per capita income in order to
               illustrate the significance of each of the components.

               MCC reported different values of summary statistics in the
               investment memo dated November 2005 as compared with the final
               analysis that reported more recent data (see table 5).

    Table 5: Summary of Compact Impacts as Presented by MCC, with GAO
    Recalculations

                                           Increase in level
                                           of GDP with MCC in  Increase in level
                                           year 5 vs. GDP      of real income per
                                           without MCC in year capita in 2010 vs.
                            Compact ERR    5                   2005 baseline
    MCC's anticipated    24.7 percent   3.2 percent         3.9 percent^a
    effect as presented
    in investment memo
    MCC's anticipated    24.2 percent^b 3.2 percent         3.7 percent^c
    effect using
    corrected or updated
    data

    Source: GAO analysis of MCC data.

    aDerived from MCC's own data and calculations.

    bIn its updated and final April 2006 economic analysis, MCC adjusted the
    24.7 percent reported in its congressional notification downward slightly
    to 24.2 percent.

    cGAO calculation after correcting MCC calculation errors.

    To illustrate the maximum impact of certain areas of risk on projected
    compact benefits, we modified the model using MCC data for different
    scenarios. We phased costs and benefits to mirror more accurately the rate
    of construction and accrual of potential benefits. Phasing costs has the
    effect of increasing ERR, while phasing benefits decreases it; the net
    effect is to decrease ERR. Within this modified framework that includes
    phasing, we eliminated each main type of benefit--direct and induced--one
    at a time to understand their relative importance.

               o For "no induced benefits," we eliminated all induced benefits
               from expanded tourism, agriculture, or land development as a
               result of the compact but did not change the direct benefits.

               o For "efficiency gains are not monetized," we eliminated all
               direct benefits such as cost savings for existing users of a road
               or benefits from newly generated traffic, which are attributable
               to the project, but did not change the induced benefits.

               Finally, we examined the impact of poor maintenance performance
               after compact expiration in 2011 on total benefits from the phased
               project. We modeled this by eliminating all future large
               maintenance expenditures and taking away a fraction of the growing
               expected benefits each year, so that at the end of the 20-year
               period, 2027, benefits basically return to their 2012 level.

               These scenarios present illustrative cases and should be
               considered as extremes of a possible range of cases that capture
               uncertainty effects on summary statistics, such as the ERR, GDP,
               and income per capita. MCC has observed that overall benefits may
               understate the true impact of the compact because spillover
               effects, such as easier access to schools and medical facilities
               due to the improved infrastructure, are not captured by these
               measures. These effects represent long-term benefits to society as
               whole, which are not accounted for by the analysis performed by
               either MCC or GAO.

    Table 6: Summary of Compact ERR, GDP, and Per Capita Income impacts under
    Alternative Scenarios of Accounting for Risks to Benefits

                                          Increase in GDP    Increase in real
                                          with MCC in year 5 income per capita
                                          relative to GDP    level in year 5
                                          without MCC in     relative to 2005
    GAO analysis^a        Compact ERR  year 5             baseline
    (1) Costs are phased  16.5 percent 1.1 percent        1.4 percent
    over 3 years and
    benefits are phased
    over 5 years
    Costs are phased over 3 years and benefits are phased
    over 5 years, and
    (2) induced benefits  5.5 percent  0.7 percent        0.8 percent
    are not realized^b
    (3) efficiency gains  11.8 percent 0.8 percent        1.0 percent
    are not monetized^c
    (4) large-scale       13.8 percent 1.1 percent        1.3 percent
    maintenance is not
    undertaken^d

    Source: GAO analysis of MCC data.

    aIn our analysis, benefits start in 2010 and are phased in equal
    increments over 5 years, from 2010 to 2014, with phasing completed by year
    5. Costs are phased over 3 years to reflect projected timing of
    construction.

    bIn addition to phasing benefits and costs, we eliminated induced effects
    of the project on agriculture, tourism, fisheries, and the development of
    subdivided beachfront land.

    cIn addition to phasing benefits and costs, we eliminated road user cost
    savings and savings from wasted surface trips, lost trips, longer
    diversions, and enforced longer trips from road closures.

    dIn addition to phasing benefits and costs, we assumed that total benefits
    will increase, peak, and decrease such that their value in 2027 will equal
    their original value in 2012. The large capital outlays for road
    rehabilitation in 2017 and 2026 in Santo and Efate have been eliminated.

    Table 7 reports the ERR for individual projects as modified in our
    scenarios.

    Table 7: Summary of Selected Compact Project Impacts under Alternative
    Scenarios Accounting for Risks to Benefits

                                                 Costs are phased over 3 years,
                                                benefits are phased over 5 years,
                                                               and
                                 Costs are
                                 phased over 3
                                 years and      Induced  Efficiency  Large-scale
                    MCC's        benefits are   benefits gains are   maintenance
                    anticipated  phased over 5  are not  not         is not
    Projects     ERR          years          realized monetized   undertaken
    Overall      24.2 percent 16.5 percent   5.5      11.8        13.8 percent
    compact                                  percent  percent
    Efate: Ring  20.6 percent 13.6 percent   negative 10.8        11.2 percent
    Road                                              percent
    Santo: Port  33.8 percent 22.6 percent   10.3     15.5        19.4 percent
    Olry Road                                percent  percent
    Santo: South 24.3 percent 14.3 percent   -4.0     12.4        10.8 percent
    Coast                                    percent  percent
    Bridges
    Tanna:       17.7 percent 12.1percent    8.6      -0.3        8.4 percent
    Whitesands                               percent  percent
    Road
    Malekula:    22.5 percent 15.1 percent   15.1     negative    14.3 percent
    Lits Lits                                percent
    Road
    Malekula:    11.4 percent 6.3 percent    1.1      6.1 percent 2.4 percent
    South West                               percent
    Bay Airstrip
    Pentecost:   15.6 percent 8.9 percent    5.6      4.4 percent 4.7 percent
    Loltong                                  percent
    Wharf/N-S
    Road

    Source: GAO analysis of MCC data.

    Appendix VI: Comments from the Millennium Challenge Corporation

    Note: GAO comments supplementing those in the report text appear at the
    end of this appendix.

    See comment 6.

    See comment 5.

    See comment 4.

    See comment 3.

    See comment 2.

    See comment 1.

    See comment 11.

    See comment 10.

    See comment 9.

    See comment 8.

    See comment 7.

    See comment 16.

    See comment 15.

    See comment 14.

    See comment 13.

    See comment 12.

    See comment 18.

    See comment 17.

    The following are GAO's comments on the Millennium Challenge Corporation
    letter dated June 8, 2007.

    GAO Comments

               1. MCC quotes from its statement in the compact and our statement
               in the text of the draft report. However, the figure we cite for
               per capita income in 2015--37 percent--originates in MCC's
               investment memo, not in our analysis.
               2. MCC notes that it emphasized repeatedly that its key statements
               in question were factually correct and that it did not intend to
               overstate benefits. At issue in this finding are both the facts
               represented by MCC's data and calculations and MCC's
               representation of these data and calculations in its public
               statements. As our report notes, MCC's portrayal of the compact's
               projected impact on income and economic growth can be understood
               as cumulative only by reviewing supporting source documents and
               spreadsheets, which are not publicly available. Our report does
               not assert or imply that MCC's portrayal was intentionally
               misleading; we did not determine through our audit, or address in
               the report, the reasons for the gap between MCC's portrayal and
               its underlying analysis.
               3. MCC comments that our assertion about its portrayal of income
               benefits is itself likely to mislead. In response to this comment,
               and as part of our final review process, we clarified the terms
               and presentation of our analysis of MCC's statements and data and
               made modifications as appropriate.
               4. MCC asserts that it "stated that per capita income would
               increase by a cumulative amount of 15.4% by 2010 relative to the
               2005 baseline" and that this is how it understood the underlying
               data in 2005. However, MCC documents in 2005 and later do not
               state that the portrayed effect on per capita income is
               cumulative. These documents include MCC's November 2005 Investment
               Memo; the March 2006 Vanuatu compact; the January 4 and March 7,
               2006, congressional notifications; MCC press release; and the
               April 2007 MCC Annual Report for 2006. We asked MCC to provide any
               publicly available document that discloses that the per capita
               income effect is cumulative; however, MCC did not provide any such
               document.
               5. MCC comments that its "underlying data do not support the
               conclusion that cumulative growth over 5 years would be only
               3.9%." Our report does not make any assertions about the
               cumulative growth rate in income per capita. As stated in figure
               7, we graph the increase in the level of income per capita in each
               year relative to the 2005 baseline. In 2010, this increase is 3.9
               percent.
               6. We agree with MCC that "the use of five-year cumulative figures
               is misleading." Since MCC's public documents do not state that the
               increase in per capita income is cumulative, we maintain that
               MCC's statements regarding the increase in average per capita
               income are inconsistent with its data.
               7. MCC comments that "the data also indicate that the effect on
               the per capita GDP growth rate will be moderate but rising, albeit
               much smaller than three percentage points." (We understand this
               statement to mean that the effect on aggregate GDP growth will be
               smaller than 3 percentage points, given that the sentences that
               follow the statement discuss aggregate, not per capita, GDP.) As
               the note to figure 9 of our report indicates, we agree with MCC
               that its data indicate that the compact will have a positive,
               albeit small, effect on GDP growth rate. MCC provided us with its
               computation showing that the compact's effect on GDP growth rate
               will increase over a 20-year period and will reach 0.54 percentage
               points in year 20--not the 3 percent growth rate increase implied
               by its portrayal of the compact's projected results. Based on
               MCC's calculation, the compact will enable the Vanuatu economy to
               double in size in 20 years. Without the compact, the Vanuatu
               economy will double in size in 24 years.
               8. MCC comments that "GAO's presentation suggests that [near and
               non-poor] households (local business people, government personnel,
               transport providers, and tourism service providers) could feasibly
               be excluded and that this would be desirable." We are not
               suggesting that such people could be feasibly excluded or that
               this would be desirable. Our report notes, "Given that many of
               MCC's benefits flow to the expatriate-dominated tourism sector,
               MCC makes an appropriate assumption that the local population will
               not receive all the benefits." Our report's analysis indicates
               that there will be poor, rural beneficiaries in addition to other
               beneficiaries but observes that MCC has not quantified what
               portion of the compact's benefits the poor, rural beneficiaries
               will receive.
               9. MCC states that we argue that the total number of rural
               beneficiaries should be reduced by one-third. In fact, our report
               observes that the total number of beneficiaries may be overstated.
               MCC specifically disputes our observation that households living
               in the hinterland or on off-shore islets may not be beneficiaries.
               However, of the 13,619 persons in the catchment areas that we
               question, 6,889 (51 percent) reside not in the hinterland or
               off-shore islets but near the Efate tourism center and capital of
               Port Vila and along a portion of the Ring Road that is already
               paved. Given that MCC's benefits in Efate are primarily based on
               increased tourist activity and road user cost savings from the
               improved road, it is not clear how construction of a road
               elsewhere on the island will lead to benefits when the existing
               paved road already provides access to Port Vila. As documented in
               appendix III, we do question whether inhabitants of off-shore
               islets and of villages more than 10 kilometers from MCC projects
               would benefit fully from MCC's projects. MCC includes these people
               in its count of beneficiaries without accounting for, and
               quantifying, the likely decrease in benefits that will accrue to a
               person who lives both across an occasionally impassable river and
               more than 10 kilometers from an MCC project bridge. As MCC's
               internal analysis states regarding the Santo South Coast Road,
               "the project does not entail a general improvement of the road,
               only of river crossings. Neither would it give all-weather access
               all the way to Tasiriki, and so to the west coast. It must be
               said, therefore, that the scale of these benefits are more than
               usually doubtful." Elsewhere in the report, it notes that the
               "benefits to the west coast [Santo] population will be slight."
               10. MCC asserts that its definition of "local population" does not
               include urban dwellers. Although we asked MCC to support this
               assertion, we received no additional documents. In separate
               comments, MCC added that although its documentation does not
               contain a very detailed reference specifically saying exactly how
               many business people, transport providers and tourism operators
               are foreigners and urban, MCC believes it is reasonable to
               conclude that urban dwellers are included in these categories.
               However, we note that not every urban dweller is a business
               person, transport provider, or tourism operator and that some
               urban beneficiaries would therefore fall into the "local
               population" that receives an estimated 43 percent of benefits. On
               the basis of MCC's definitions of "local population" and other
               terms, we determined that "local population" includes residents of
               nonrural areas. MCC's definition of the term local population
               states, "We have combined three categories: local producers, local
               consumers and inhabitants of remote communities. In reality these
               are the same people and we have labeled them simply `Local
               population.'" MCC defines local producers as including
               "landowners; existing and potential lessees of land; processors of
               primary produce, chiefly for export; and the owners and operators
               of tourist facilities." This definition does not distinguish
               between rural and urban dwellers. Further, given that expatriates
               dominate in Vanuatu's formal economy sectors and that the urban
               center of Port Vila is the center of Vanuatu's tourist industry,
               it is likely that "processors of primary produce, chiefly for
               export; and the owners and operators of tourist facilities"
               include urban dwellers and those that are not poor. Finally, MCC's
               definition of "inhabitants of remote communities" includes the
               poor inhabitants of the "informal urban settlements" on the
               outskirts of Port Vila. Although these people come from rural
               areas, they are now in an urban environment and included within
               MCC's definition of the local population.
               11. MCC states that it has "a reasonable expectation that the
               majority of the 65,000 [beneficiaries] are poor." However, MCC's
               public documents use the phrase "65,000 poor, rural inhabitants,"
               without acknowledging that not all of the 65,000 beneficiaries
               live in poverty. We agree that MCC would be correct to suggest
               that the majority of the projected beneficiaries are poor, as our
               report states: "The poverty level, defined as having an income of
               one U.S. dollar per day, is 51 percent in rural areas."
               12. About program risks in general, MCC states that it differs
               from us significantly regarding the precise nature and severity of
               risks. As we note in our report, we offer several alternative
               scenarios to illustrate the maximum impact of individual areas of
               risk on projected compact benefits. These scenarios examine the
               effect of individual risk areas and do not consider the
               possibility of projects' failing to achieve certain benefits in
               multiple areas simultaneously.
               13. We agree with MCC that attempting to quantify induced benefits
               is important and that our fieldwork generally affirmed MCC's
               assumptions. We maintain that quantifying induced benefits
               constitutes a risk to projected impact, because realization of
               such benefits depends on beneficiaries' response to the
               opportunities that the compact provides.
               14. We asked MCC to support its assertion that "it is highly
               plausible in a poor country that the vast majority of [time and
               cost] savings would be used to generate income." We did not
               receive additional documentation supporting this assertion.
               However, MCC stated that its assertion is based on extensive field
               consultations with beneficiary groups. MCC's documentation of its
               methodology for assessing efficiency gains states that its
               valuation of the average value of passengers' time is "extremely
               crude." MCC's documentation adds: "The very concept of `working
               hours' in a largely subsistence economy is open to question. And
               what is the value of tourists' time to the national economy? It is
               arguable that faster journey times allow tourists more time to
               spend on things other than travel. But there is no research to
               support a more sophisticated approach to valuing time."
               15. MCC notes other factors that may benefit the economy. As our
               report indicates (see footnote 15), we agree that "other benefits
               may accrue to Vanuatu from the MCC compact. Increased economic
               activity in tourism may have spillover benefits for other sectors
               of the economy, and the welfare of Vanuatu's citizens may improve
               due to increased access to health care and educational
               opportunities."
               16. MCC believes that we have overstated the risk from poor
               maintenance of the infrastructure investment and have not
               accounted for Vanuatu's increasing its maintenance spending by
               $4.6 million or the assistance and maintenance equipment MCC is
               providing to Vanuatu. We have added additional information to our
               report regarding MCC's program to improve maintenance capability.
               Nevertheless, maintenance remains a significant risk. As we noted
               in our report, Vanuatu's record on transport infrastructure
               maintenance is poor. Other donors, including officials from the
               ADB and the World Bank, identified maintenance as a significant
               challenge.
               17. MCC states that "the projects that were under consideration in
               the compact were at an advanced stage in preparation, either at
               preliminary design or final design stage," not at the alternatives
               analysis stage. Our previous report, which cited a study by
               Flyvbjerg, Holm, and Buhl as part of a larger report on improving
               cost and benefit analysis for transportation projects, used the
               term "alternatives analysis stage"; however, the study uses the
               term "time of decision to build" to define the point of
               comparison. MCC's "time of decision to build" is when it has
               completed its due diligence process and decided to sign a compact.
               As such, the basis for comparison in the study by Flyvbjerg et al.
               and in our report is identical. To help clarify the basis of
               comparison, we have replaced "alternatives analysis stage" with
               "time of decision to build." MCC states that it provided us the
               final cost estimates, including price and physical contingencies,
               for each subproject within the compact. However, as our report
               notes, we did not receive full documentation of the cost estimates
               and contingencies that MCC used in developing its economic
               analyses. We received no supporting documentation for the cost and
               contingency estimates for the Santo and Efate projects, which
               account for 56 percent of the total compact budget. A key document
               that MCC's contractor told us it used as a basis for developing
               the Santo and Efate costs was not available to us because the
               government of Vanuatu would not provide it to MCC. The
               documentation of MCC's cost estimates and contingencies for
               specific projects that we received shows a design contingency of
               20 percent. (One project--MCC's smallest project, the $400,000
               South West Bay airstrip--adds an additional 10 percent
               construction contingency.) This 20 percent contingency is slightly
               less than the average cost overrun found by Flyvbjerg et al. MCC
               may complete the projects at, under, or above a 20 percent
               contingency. However, cost overruns remain a risk to project
               benefits, and we have presented them as such.
               18. See comment 2.

    Appendix VII: GAO Contact and Staff Acknowledgments

    GAO Contact

    David B. Gootnick, Director, 202-512-3149

    Staff Acknowledgments

    In addition to the person named above, Emil Friberg, Jr. (Assistant
    Director), Gergana Danailova-Trainor, Reid Lowe, Angie Nichols-Friedman,
    Michael Simon, and Seyda Wentworth made key contributions to this report.
    Also, David Dornisch, Etana Finkler, Ernie Jackson, and Tom McCool
    provided technical assistance.

    (320451)

    GAO's Mission

    The Government Accountability Office, the audit, evaluation and
    investigative arm of Congress, exists to support Congress in meeting its
    constitutional responsibilities and to help improve the performance and
    accountability of the federal government for the American people. GAO
    examines the use of public funds; evaluates federal programs and policies;
    and provides analyses, recommendations, and other assistance to help
    Congress make informed oversight, policy, and funding decisions. GAO's
    commitment to good government is reflected in its core values of
    accountability, integrity, and reliability.

    Obtaining Copies of GAO Reports and Testimony

    The fastest and easiest way to obtain copies of GAO documents at no cost
    is through GAO's Web site ( [39]www.gao.gov ). Each weekday, GAO posts
    newly released reports, testimony, and correspondence on its Web site. To
    have GAO e-mail you a list of newly posted products every afternoon, go to
    [40]www.gao.gov and select "Subscribe to Updates."

    Order by Mail or Phone

    The first copy of each printed report is free. Additional copies are $2
    each. A check or money order should be made out to the Superintendent of
    Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
    copies mailed to a single address are discounted 25 percent. Orders should
    be sent to:

    U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
    D.C. 20548

    To order by Phone: Voice: (202) 512-6000
    TDD: (202) 512-2537
    Fax: (202) 512-6061

    To Report Fraud, Waste, and Abuse in Federal Programs

    Contact:

    Web site: [41]www.gao.gov/fraudnet/fraudnet.htm
    E-mail: [42]fraudnet@gao.gov
    Automated answering system: (800) 424-5454 or (202) 512-7470

    Congressional Relations

    Gloria Jarmon, Managing Director, [43]JarmonG@gao.gov (202) 512-4400 U.S.
    Government Accountability Office, 441 G Street NW, Room 7125 Washington,
    D.C. 20548

    Public Affairs

    Paul Anderson, Managing Director, [44]AndersonP1@gao.gov (202) 512-4800
    U.S. Government Accountability Office, 441 G Street NW, Room 7149
    Washington, D.C. 20548

    [45]www.gao.gov/cgi-bin/getrpt?GAO-07-909 .

    To view the full product, including the scope
    and methodology, click on the link above.

    For more information, contact David Gootnick, (202) 512-3149,
    gootnickd@gao.gov.

    Highlights of [46]GAO-07-909 , a report to the Chairman, Committee on
    Foreign Affairs, House of Representatives

    July 2007

    MILLENNIUM CHALLENGE CORPORATION

    Vanuatu Compact Overstates Projected Program Impact

    In January 2004, Congress established the Millennium Challenge Corporation
    (MCC) for foreign assistance. Congress has appropriated almost $6 billion
    to MCC. As of March 2007, MCC had signed almost $3 billion in compacts
    with 11 countries, including a 5-year, $65.7 million compact with Vanuatu.
    MCC states that the Vanuatu compact will have a transformational effect on
    the country's economy, increasing per capita income and GDP and benefiting
    65,000 poor, rural people. GAO examined (1) MCC's methods of projecting
    economic benefits, (2) MCC's portrayal and analysis of the projected
    benefits, and (3) risks that may affect the compact's impact. GAO reviewed
    MCC's analyses and met with officials and business owners in Vanuatu as
    well as with other donors.

    [47]What GAO Recommends

    GAO recommends that the Chief Executive Officer of MCC (1) revise the
    public reporting of the Vanuatu compact's projected impact, (2) assess
    whether similar reporting in other compacts accurately reflects underlying
    analyses, and (3) improve its economic analyses by more fully accounting
    for risks to project benefits. MCC did not directly address our
    recommendations but commented that it had not intended to make misleading
    statements and that its portrayal of projected results was factual and
    consistent with underlying data.

    MCC projects that the Vanuatu compact's transportation infrastructure
    projects will provide direct benefits such as reduced transportation costs
    and induced benefits from growth in tourism and agriculture. MCC estimated
    the costs and benefits over 20 years, with benefits beginning in full in
    2008 or 2009 and growing each year, and it counted poor, rural
    beneficiaries by defining the area where benefits were likely to accrue.
    Using projected benefits and costs, MCC calculated the compact's economic
    rate of return (ERR) and its effects on Vanuatu's gross domestic product
    (GDP) and per capita income.

    MCC's portrayal of the projected impact does not reflect its underlying
    data. MCC states that per capita income will increase by approximately
    $200, or 15 percent, by 2010 and by $488, or 37 percent, by 2015. However,
    MCC's underlying data show that these figures represent the sum of
    individual years' gains in per capita income relative to 2005 and that
    actual gains will be $51, or 3.9 percent, in 2010 and $61, or 4.6 percent,
    in 2015. MCC also states that GDP will increase by an additional 3 percent
    a year, but its data show that after GDP growth of 6 percent in 2007, the
    economy's growth will continue at about 3 percent, as it would without the
    compact. MCC states that the compact will benefit approximately 65,000
    poor, rural inhabitants, but this statement does not identify the
    financial benefits that accrue to the rural poor or reflect its own
    analysis that 57 percent of benefits go to others.

    We identified five key risks that could affect the compact's projected
    impacts. (1) Cost estimate contingencies may not be sufficient to cover
    project overruns. (2) Compact benefits will likely accrue more slowly than
    MCC projected. (3) Benefit estimates assume continued maintenance, but
    MCC's ability to ensure maintenance will end in 2011, and Vanuatu's
    maintenance record is poor. (4) Induced benefits depend on businesses' and
    residents' response to new opportunities. (5) Efficiency gains, such as
    time saved in transit, may not increase per capita income. Our analysis of
    these areas of risk illustrates the extent that MCC's projections are
    dependent on assumptions of immediate realization of benefits, long-term
    maintenance, realization of induced benefits, and benefits from efficiency
    gains.

    Vanuatu Compact's Impact on Per Capita Income According to MCC Statement
    vs. MCC Data

    References

    Visible links
      35. http://www.gao.gov/cgi-bin/getrpt?GAO-06-805
      36. http://www.gao.gov/cgi-bin/getrpt?GAO-07-357
      37. http://www.gao.gov/cgi-bin/getrpt?GAO-05-172
      38. http://www.gao.gov/cgi-bin/getrpt?GAO-07-268
      39. http://www.gao.gov/
      40. http://www.gao.gov/
      41. http://www.gao.gov/fraudnet/fraudnet.htm
      42. mailto:fraudnet@gao.gov
      43. mailto:JarmonG@gao.gov
      44. mailto:AndersonP1@gao.gov
      45. http://www.gao.gov/cgi-bin/getrpt?GAO-07-909
      46. http://www.gao.gov/cgi-bin/getrpt?GAO-07-909
    *** End of document. ***