Millennium Challenge Corporation: Vanuatu Compact Overstates	 
Projected Program Impact (11-JUL-07, GAO-07-909).		 
                                                                 
In January 2004, Congress established the Millennium Challenge	 
Corporation (MCC) for foreign assistance. Congress has		 
appropriated almost $6 billion to MCC. As of March 2007, MCC had 
signed almost $3 billion in compacts with 11 countries, including
a 5-year, $65.7 million compact with Vanuatu. MCC states that the
Vanuatu compact will have a transformational effect on the	 
country's economy, increasing per capita income and GDP and	 
benefiting 65,000 poor, rural people. GAO examined (1) MCC's	 
methods of projecting economic benefits, (2) MCC's portrayal and 
analysis of the projected benefits, and (3) risks that may affect
the compact's impact. GAO reviewed MCC's analyses and met with	 
officials and business owners in Vanuatu as well as with other	 
donors. 							 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-07-909 					        
    ACCNO:   A72527						        
  TITLE:     Millennium Challenge Corporation: Vanuatu Compact	      
Overstates Projected Program Impact				 
     DATE:   07/11/2007 
  SUBJECT:   Budgeting						 
	     Cost analysis					 
	     Cost control					 
	     Economic growth					 
	     Economically depressed areas			 
	     Financial statements				 
	     Impacted areas					 
	     Income statistics					 
	     Program evaluation 				 
	     Federal corporations				 
	     Federal aid to foreign countries			 
	     Risk management					 
	     Cost estimates					 
	     Vanuatu						 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-07-909

   

     * [1]Results in Brief
     * [2]Background

          * [3]Description of Vanuatu
          * [4]Vanuatu's Economy
          * [5]MCC's Compact with Vanuatu

     * [6]MCC Projected Compact's Impact Using Estimates of Benefits,

          * [7]MCC Calculated Benefits, Costs, and Beneficiaries

               * [8]Benefits
               * [9]Costs
               * [10]Beneficiaries

          * [11]MCC Used Projected Benefits and Costs to Determine ERR and E

     * [12]MCC's Data Do Not Support Its Portrayal of Compact Benefits

          * [13]Portrayal of Per Capita Income Benefit Suggests Larger Effec
          * [14]Portrayal of GDP Benefit Differs from Underlying Analysis
          * [15]Portrayal of Poverty Reduction Does Not Quantify Benefits to
          * [16]Estimates of Compact Benefits Are Generally Reasonable, but

     * [17]Several Risks May Lead to Reduced Project Benefits

          * [18]Construction Costs May Exceed Contingencies
          * [19]Compact Benefits Are Likely to Accrue More Slowly Than Proje
          * [20]Project Maintenance after Compact Expiration Cannot Be Ensur
          * [21]Induced Benefits Require Tourism Providers and Agricultural
          * [22]Efficiency Gains May Not Cause Measurable Change in Per Capi

     * [23]Conclusions
     * [24]Recommendations
     * [25]Agency Comments and Our Evaluation

          * [26]Compact Impact on Incomes and Growth
          * [27]Compact Beneficiaries
          * [28]Program Risks
          * [29]Per Capita Income Impact
          * [30]Calculation of Beneficiaries

     * [31]GAO Comments
     * [32]GAO Contact
     * [33]Staff Acknowledgments

          * [34]Order by Mail or Phone

Report to the Chairman, Committee on Foreign Affairs, House of
Representatives

United States Government Accountability Office

GAO

July 2007

MILLENNIUM CHALLENGE CORPORATION

Vanuatu Compact Overstates Projected Program Impact

GAO-07-909

Contents

Letter 1

Results in Brief 3
Background 6
MCC Projected Compact's Impact Using Estimates of Benefits, Costs, and
Catchment Area 12
MCC's Data Do Not Support Its Portrayal of Compact Benefits 17
Several Risks May Lead to Reduced Project Benefits 24
Conclusions 29
Recommendations 30
Agency Comments and Our Evaluation 30
Appendix I Objectives, Scope, and Methodology 35
Appendix II Prior Development Assistance to Vanuatu 38
Appendix III MCC Estimates of Per Capita Income Impact and Poor, Rural
Beneficiaries 40
Appendix IV MCC Cost Estimates for the Vanuatu Compact 47
Appendix V Illustrative Alternative Calculations of Vanuatu Compact Impact
49
Appendix VI Comments from the Millennium Challenge Corporation 52
GAO Comments 57
Appendix VII GAO Contact and Staff Acknowledgments 63

Tables

Table 1: Average Annual Contributions of Top Five Donors to Vanuatu in
2004 and 2005 38
Table 2: Number of Poor, Rural Beneficiaries of Vanuatu Compact, as
Calculated by MCC 43
Table 3: Effect of Calculation and Data Errors on Beneficiary Population
in Efate and Santo 44
Table 4: Effect of MCC Assumptions on Beneficiary Population in Efate and
Santo 44
Table 5: Summary of Compact Impacts as Presented by MCC, with GAO
Recalculations 49
Table 6: Summary of Compact ERR, GDP, and Per Capita Income impacts under
Alternative Scenarios of Accounting for Risks to Benefits 50
Table 7: Summary of Selected Compact Project Impacts under Alternative
Scenarios Accounting for Risks to Benefits 51

Figures

Figure 1: Location of Vanuatu 7
Figure 2: Development and Implementation of Vanuatu Compact 9
Figure 3: MCC Vanuatu Projects by Size and Location 11
Figure 4: MCC Statement of Impacts in March 2006 Congressional
Notification 12
Figure 5: MCC's Logic Model for the Vanuatu Compact 14
Figure 6: MCC Benefits Calculations as Source of Statements about Impact
of Vanuatu Compact 17
Figure 7: Vanuatu Compact's Projected Impact on Real Per Capita Income
According to MCC Statement and MCC Data Relative to 2005 Per Capita Income
19
Figure 8: MCC Methodology for Projecting Vanuatu Compact's Impact on Real
Per Capita Income 20
Figure 9: Vanuatu GDP Growth with and without MCC Compact 21
Figure 10: MCC Analysis of Distribution of Vanuatu Compact Benefits 22
Figure 11: MCC Compact's Projected Impact on Vanuatu Per Capita Income
with and without Adjusted Population Estimates 41
Figure 12: MCC Compact's Projected Impact on Vanuatu Per Capita Income in
Real Terms Relative to Per Capita Income without MCC 42
Figure 13: MCC Efate Ring Road Catchment Area 45

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

Abbreviations

ADB Asian Development Bank
CEO Chief Executive Officer
ERR economic rate of return
GDP gross domestic product
IMF International Monetary Fund
MCA Millennium Challenge Account
MCC Millennium Challenge Corporation
PWD Public Works Department

United States Government Accountability Office
Washington, DC 20548

July 11, 2007

The Honorable Tom Lantos
Chairman
Committee on Foreign Affairs
House of Representatives

Dear Mr. Lantos:

In January 2004, Congress established the Millennium Challenge Corporation
(MCC) to administer the Millennium Challenge Account (MCA) for foreign
assistance. MCC's mission is to reduce poverty by supporting sustainable,
transformative economic growth in developing countries that create and
maintain sound policy environments. MCC carries out this mission by
funding projects or activities in developing countries that have
demonstrated a commitment to ruling justly and democratically, encouraging
economic freedom, and investing in people. Congress appropriated almost $6
billion for fiscal years 2004 to 2007 to MCC, and the President has
requested an additional $3 billion in MCC funding for fiscal year 2008.

As of March 2007, MCC had signed compacts with 11 countries, totaling
approximately $3 billion, including a 5-year, $65.7 million compact with
the Pacific island nation of Vanuatu.^1 Although MCC's compact with
Vanuatu is its smallest compact monetarily, it provides by far the largest
amount relative to the country's population and gross domestic product
(GDP).^2 In a process known as due diligence, MCC analyzed Vanuatu's
proposal for compact assistance to determine the compact's expected
economic rate of return (ERR)^3 and impact on poverty reduction and
economic growth. After completing its due diligence analysis, MCC stated
in its compact and in its notification to Congress that it expects the
Vanuatu compact to have a transformational impact on Vanuatu's economic
development--an effect MCC defines as "a dramatic and long-lasting impact
on poverty reduction through sustainable economic growth."^4 MCC states
that its compacts will provide or contribute to a transformational impact
in 5 of its 11 compacts.^5

^1An MCC compact is an agreement between the U.S. government, acting
through MCC, and the government of a country eligible for MCA assistance.

^2MCC's $65.7 million compact with Vanuatu provides $317 per capita; in
contrast, MCC's $547 million compact with Ghana--its largest
compact--provides $25 per capita. The amounts provided per capita by the
11 compacts signed to date range from $6 for Madagascar to $317 for
Vanuatu.

^3Project cash flows are determined by comparing program spending against
future expected increases in value added or income. The internal rate of
return is calculated for these cash flows to summarize the economic
impact. MCC refers to this internal rate of return as the economic rate of
return.

At your request, we reviewed MCC's economic analyses of the Vanuatu
compact. Specifically, we examined

           o MCC's methods of projecting the compact's economic benefits and
           methods of calculating the benefits,

           o MCC's portrayal and analysis of the projected benefits, and

           o risks that could affect the compact's impact on poverty
           reduction and economic growth.

           We reviewed MCC's record of due diligence and supplemented this
           review with interviews with MCC officials to identify MCC's logic,
           data, methods, and assumptions for determining the compact's
           projected costs and benefits, and to identify the projected
           effects on GDP, per capita income, and poverty. We evaluated MCC's
           statements about the compact's impacts in its notification to
           Congress, in the Vanuatu compact, and in its "investment memo"^6
           by comparing these statements with the underlying analyses and
           data used to support them. We could not validate most of
			  
^4Millennium Challenge Account, Best Practices in Compact Development
(Washington, D.C.: 2006).

^5For example, in Nicaragua, MCC expects that the compact will transform
project areas into an engine of economic growth; in El Salvador, MCC
states that the compact provides an historic opportunity to transform the
country's economic development; and in Armenia, MCC is undertaking road
and irrigation projects to transform the economic performance of Armenia's
agricultural sector.

^6The "investment memo" is an MCC internal document prepared by MCC's
compact assessment team and submitted to MCC's investment
committee--consisting of MCC's Chief Executive Officer (CEO), vice
presidents, and other senior officials. The committee reviews the memo and
decides whether to recommend proceeding to compact negotiations.

           MCC's underlying data and assumptions because the data were not
           available or could not be checked within the time frames of our
           engagement. We examined MCC's methodologies and checked the
           analyses for calculation errors. Further, we identified risks to
           MCC's compact results based on our review of MCC's internal
           documentation, donor reporting, and academic development
           literature. We interviewed Vanuatu and MCC officials and contacted
           the contractor that assisted in MCC's analyses. We also
           interviewed interested parties such as tourism and agriculture
           business owners in Vanuatu. We focused our analysis and field work
           on MCC's three transportation infrastructure projects on Vanuatu's
           two most populous islands, Santo and Efate, which represent 56
           percent of compact cost. We modeled several areas of project risks
           to illustrate their maximum impact on the economic analyses of
           ERR, GDP, and per capita income. In modeling these risks, we used
           the data from MCC's economic analyses; we did not validate these
           data. We conducted our review from August 2006 through May 2007 in
           accordance with generally accepted government auditing standards.
			  
			  Results in Brief

           MCC projected the impact of the Vanuatu compact by estimating the
           program's benefits, costs, and beneficiaries and calculating the
           compact's effect on per capita income, GDP, and poverty reduction.
           According to MCC, transportation infrastructure improvements will
           provide direct benefits from construction spending in the local
           economy, reduced transportation costs, and improved services. The
           improved infrastructure will also provide induced benefits from
           growth in Vanuatu's tourism and agriculture sectors. MCC estimated
           the value of these benefits over a 20-year period, beginning in
           full in 2008 or 2009 and growing each year. MCC developed its
           project cost estimates based on existing cost estimates prepared
           for the government of Vanuatu and for another donor. To determine
           the number of poor, rural beneficiaries, MCC defined a catchment
           area--the geographic area in which benefits may be expected to
           accrue--using maps of Vanuatu and data from the most recent
           Vanuatu census. Using its projected benefit and cost data, MCC
           calculated (1) the compact's ERR, comparing projected benefits to
           projected costs; (2) per capita income, determining the total
           benefits and dividing the total value by Vanuatu's baseline
           population; and (3) the compact's effect on Vanuatu's GDP by
           computing the total benefits added to the economy.

           MCC's portrayal of the Vanuatu compact's impact does not reflect
           the data and analysis underlying its projections of the compact's
           benefits. MCC states that as a result of the compact, per capita
           income will increase by approximately $200, or 15 percent, by 2010
           and $488, or 37 percent, by 2015. This statement suggests that per
           capita income in 2010 and 2015 will be, respectively, 15 percent
           and 37 percent higher than without the compact. However, MCC's
           underlying data show that these figures represent the sum of gains
           in per capita income for individual years relative to 2005 rather
           than actual, or net, annual gains as of 2010 or 2015. For example,
           MCC sums the per capita income gains relative to 2005 for 2006 to
           2010, averaging 3 percent, as 15 percent in 2010, without stating
           that these gains are cumulative. Our analysis of MCC's data shows
           that actual gains in per capita income, relative to income in
           2005, would be $51, or 3.9 percent, in 2010 and $61, or 4.6
           percent, in 2015. Likewise, MCC states that Vanuatu's GDP will
           increase by "an additional 3 percent a year." However, MCC's
           underlying data and calculations show that while the level of
           Vanuatu's GDP will grow by 6 percent in 2007, the economy's growth
           rate in subsequent years continues at approximately 3 percent,
           MCC's assumed rate without the compact. Regarding MCC's portrayal
           of the compact's impact on poverty reduction, MCC's publicly
           available documents state that the compact is expected to benefit
           approximately 65,000 poor, rural inhabitants "living nearby and
           using the roads to access markets and social services." However,
           MCC's underlying documentation shows that these 65,000
           beneficiaries will not receive the majority of the benefits but
           will instead share the 43 percent of the compact's monetary
           benefits that MCC expects to go to the local population; the
           remaining 57 percent will go to beneficiaries such as tourism
           services providers, transport companies, and local businesses.
           Finally, although MCC's estimates of compact benefits and
           beneficiaries are generally reasonable, some of the calculations
           and assumptions it used are problematic. Correcting MCC's
           calculations of per capita income benefits slightly reduces the
           estimated benefit, and correcting MCC's calculations and fully
           discounting some of its assumptions regarding the count of
           beneficiaries would reduce estimated beneficiaries on Santo and
           Efate by about one-third.

           We identified five key risks that could affect the Vanuatu
           compact's projected impact on poverty reduction and economic
           growth.

           o The contingencies included in MCC's calculations of construction
           costs may not be sufficient to cover potential cost overruns. We
           received MCC cost estimate documentation for 5 of MCC's 11
           construction projects showing that MCC's estimates for these 5
           include design contingencies of 20 percent; however, a previous
           study found average cost overruns of more than 20 percent in
           transportation infrastructure projects in other countries.
           Further, the risk of excessive cost overruns is significant in a
           small country such as Vanuatu. Any construction cost overrun could
           cause MCC to reduce the compact's scope and therefore its
           benefits.

           o Although the compact's benefits are projected to begin shortly
           after completion of the projects, some of these benefits will
           likely accrue more slowly. For example, according to agricultural
           and timber producers, their businesses will likely respond
           gradually to any increased market opportunities.

           o Whereas the benefit projections assume continued maintenance of
           completed projects, MCC's ability to ensure such maintenance will
           end in 2011. Moreover, previous donors to Vanuatu have found
           Vanuatu's record of maintaining donor projects to be poor. Reduced
           maintenance would lead to reduced benefits from the project.

           o The projected induced benefits from expanded tourism and
           agriculture depend on businesses and rural inhabitants responding
           to opportunities created by improved infrastructure.

           o Efficiency gains that MCC counts as direct benefits, such as
           time saved in transit, may not be put to economic use and result
           in increased per capita income, as MCC projects.

           Our analysis of these areas of risk illustrates the extent to
           which MCC's benefit projections are dependent on assumptions of
           immediate realization of benefits, successful long-term
           maintenance, realization of induced benefits, and benefits from
           efficiency gains. Accounting for these risks can reduce overall
           compact ERR from 24.2 percent, as projected by MCC, to between 5.5
           percent and 16.5 percent, and some projects may have a negative
           ERR.^7

           To help better express and determine the impact of its compacts,
           this report recommends that the Chief Executive Officer (CEO) of
           MCC (1) revise the public reporting of the projected impact of the
           Vanuatu compact, (2) assess whether similar statements in other
           compacts accurately reflect underlying data, and (3) improve MCC's
           economic analysis by phasing costs and benefits and more fully
           accounting for risks to project benefits.
			  
^7We modeled risk scenarios by first assuming the phasing of costs and
benefits. We then combined this phasing with three additional models that
assumed a lack of maintenance, a lack of induced benefits, and a lack of
monetized efficiency gains. See appendix V for more information about our
methodology.

           We provided a draft of this report to MCC. In commenting on the
           draft, MCC did not directly acknowledge our recommendations. MCC
           acknowledged that its use of projected cumulative compact impact
           on income and growth was misleading, but asserted that it had no
           intention to mislead and that its portrayal of projected compact
           benefits was factually correct. MCC questioned our finding that
           its underlying data and analysis do not support its portrayal of
           compact benefits and our characterization of the program's risks.
           In response to MCC's comments, we clarified the terms and
           presentation of our analysis of MCC's statements and data and
           provided additional information on the basis for our findings. We
           also provided technical corrections where appropriate.
			  
			  Background
			  
			  Description of Vanuatu

           Vanuatu consists of 83 islands spread over hundreds of miles of
           ocean in the South Pacific, 1,300 miles northeast of Sydney,
           Australia (see fig. 1). About 39 percent of the population is
           concentrated on the islands of Santo and Efate. Vanuatu's capital,
           Port Vila, is on Efate, and Vanuatu's only other urban center,
           Luganville, is on Santo. The country has three official
           languages--English, French, and Bislama--but more than 100 other
           dialects are also spoken. Traditional custom chiefs have a
           significant role in the state particularly in rural areas. Civil
           unrest has caused occasional disruptions; for example, riots
           erupted in Port Vila over the misappropriation of assets from the
           Vanuatu National Provident Fund in 1998, and an ethnic conflict on
           Efate led Vanuatu's parliamentary government to declare a state of
           emergency in March 2007.

           Figure 1: Location of Vanuatu
			  
			  Vanuatu's Economy

           In the past decade, Vanuatu's real GDP growth averaged 2 percent,
           although more rapid population growth led to a decline in per
           capita GDP over the same period. Average growth of real GDP per
           capita was negative from 1993 to 2005. In its economic analyses,
           MCC used a baseline 2005 Vanuatu per capita income of $1,326. An
           estimated 40 percent of Vanuatu's population of about 207,000 has
           an income below the international poverty line of $1 per day.
           Agriculture and tourism are the principal productive sectors of
           Vanuatu's economy, contributing approximately 15 percent and 19
           percent to GDP, respectively. Although agriculture represents a
           relatively small share of Vanuatu's overall economy, approximately
           80 percent of Vanuatu's residents live in rural areas and depend
           on subsistence agriculture for food and shelter, selling surplus
           commodities to generate cash for school fees, transportation,
           consumer goods, and services. Copra (dried coconut) is the main
           cash crop; kava and cacao are also grown. The tourism sector is
           dominated by expatriates of foreign countries living in Vanuatu,
           who also predominate in other formal sectors of the economy such
           as plantation agriculture and retail trade. According to the Asian
           Development Bank (ADB), tourism is one of the most promising
           sectors in Vanuatu in terms of its potential for earning foreign
           exchange and generating employment. According to a survey
           conducted by the Vanuatu National Statistics office, the largest
           markets for Vanuatu tourism are Australia and New Zealand. Despite
           prior donor efforts, private sector development in Vanuatu's
           tourism and agricultural sectors faces challenges, including
           political uncertainty, and high costs of doing business. (See app.
           II for a summary of donor efforts and Vanuatu's development
           challenges.)
			  
			  MCC's Compact with Vanuatu

           On May 6, 2004, MCC determined that Vanuatu was eligible to submit
           a compact proposal for Millennium Challenge Account (MCA)
           funding.^8 Vanuatu's proposal identified transportation
           infrastructure as a key constraint to private-sector development.
           Based on the analysis performed during approximately 5 months of
           due diligence,^9 MCC signed a 5-year, $65.7 million compact with
           the government of Vanuatu on March 2, 2006, with entry into
           force^10 on April 28, 2006.^11 In keeping with its statutory
           requirements, MCC submitted a congressional notification of the
           compact signing on March 7, 2006.
			  
^8The Millennium Challenge Act of 2003 requires MCC to determine whether
countries are eligible for MCA assistance each fiscal year. Countries with
per capita income at or below a set threshold may be selected as eligible
for assistance if they meet MCC indicator criteria and are not statutorily
barred from receiving U.S. assistance. MCC uses 16 indicators divided into
three categories: Ruling Justly, Encouraging Economic Freedom, and
Investing in People. To be eligible for MCA assistance, countries must
score above the median relative to their peers on at least half of the
indicators in each category and above the median on the indicator for
combating corruption. GAO, Millennium Challenge Corporation: Compact
Implementation Structures Are Being Established; Framework for Measuring
Results Needs Improvement, [35]GAO-06-805 (Washington, D.C.: July 28,
2006).

           Figure 2 illustrates the chronology of the development and
           implementation of the Vanuatu proposal and compact.

Figure 2: Development and Implementation of Vanuatu Compact

^9Prior to the due diligence process, a transaction team conducts a
preliminary assessment of each proposal and reports its findings in an
internal "opportunity memo" to the MCC investment committee. If the
opportunity memo is approved, the team launches a detailed due diligence
review, using MCC's published proposal guidelines and criteria to analyze
project costs and benefits and assess the proposal's consultative process,
rationale, environmental and social impact, and sustainability. The
transaction team makes recommendations based on its assessment of the
proposal in an investment memo to the MCC investment committee.

^10MCC and the country's accountable entity must complete supplemental
agreements, including a disbursement agreement and a procurement
agreement, before the compact enters into force and funds are disbursed.

^11By U.S. law, all compacts must end within 5 years of entry into force.

The $65.7 million Vanuatu compact includes $54.5 million for the
rehabilitation or construction of 11 transportation infrastructure assets
on 8 of Vanuatu's 83 islands, including roads, wharves, an airstrip, and
warehouses (see fig. 3). The compact also includes $6.2 million for an
institutional strengthening program to increase the capacity of the
Vanuatu Public Works Department (PWD) to maintain transportation
infrastructure.^12 The remaining $5 million is for program management and
monitoring and evaluation. More than half of the compact, $37 million, is
budgeted for three road projects on Santo and Efate islands. On both
islands, the compact will upgrade existing roads, while on Santo the
compact also includes five new bridges on an existing road. To oversee and
manage the compact programs, the Vanuatu government has established
MCA-Vanuatu, an independent entity housed within the Vanuatu Ministry of
Finance and Economic Management.

^12The institutional strengthening program includes $5.74 million for
equipment purchases; of this amount, $1.4 million is provided directly to
PWD and the remainder will purchase equipment for the use of the MCC
construction contractor, to be turned over to the PWD in specified
condition 4 years later.

Figure 3: MCC Vanuatu Projects by Size and Location

The compact and the congressional notification state that the compact will
have a transformational impact on Vanuatu's economic development,
increasing average per capita income by approximately $200--15 percent--by
2010 and increasing total GDP by "an additional 3 percent a year." The
investment memo further quantifies the per capita income increase as
$488--37 percent--by 2015. The compact and the congressional notification
further state that the compact will provide benefits to approximately
65,000 poor, rural inhabitants (see fig. 4).

Figure 4: MCC Statement of Impacts in March 2006 Congressional
Notification

MCC Projected Compact's Impact Using Estimates of Benefits, Costs, and Catchment
Area

To project the Vanuatu compact program's benefits, costs, and
beneficiaries and calculate the compact's impact on per capita income,
GDP, and poverty reduction, MCC made site visits to Vanuatu and reviewed
available documents to gather needed data. MCC determined that investments
in transportation infrastructure will lead to increases in incomes and
used the available data and made assumptions to determine the compact's
benefits and costs. MCC also estimated the number of beneficiaries within
a defined catchment area--the geographic area in which benefits may be
expected to accrue. MCC then used the benefits and costs to calculate
summary statistics about the compact's ERR and effects on Vanuatu's GDP
and per capita income.

MCC Calculated Benefits, Costs, and Beneficiaries

To prepare its economic analyses of the compact program's benefits, costs,
and number of beneficiaries, MCC sent a team of employees and contractors
to Vanuatu. The contractor MCC retained had previously served as a
consultant to the ADB in Vanuatu and had prepared a multiyear study of
projects proposed to the ADB as part of its Vanuatu Outer Islands
Infrastructure Development Project in 2000 to 2003. Seven of MCC's 11
transportation infrastructure projects were initially developed as part of
the ADB study. While in Vanuatu, the MCC team met with officials
representing other donors, Vanuatu's Department of Economic and Sectoral
Planning, the National Statistics Office, and the Vanuatu PWD, among
others. The MCC team also met with a number of interested and informed
parties representing the transportation, tourism, and agriculture sectors
and reviewed a number of documents to gather the information needed for
its economic analyses.^13

  Benefits

MCC's compact aims to benefit poor, rural agricultural producers and
providers of tourism-related goods and services by reducing transportation
costs and improving the reliability of access to transportation services.
MCC's logic model for the compact posits that improvements in
infrastructure will lead to (1) direct benefits from increased
transportation reliability, construction spending, and reduced
transportation costs and (2) induced benefits from greater tourism and
agricultural trade. The direct and induced benefits in turn lead to
increases in per capita income and GDP and reduction in poverty (see fig.
5).

^13MCC used Vanuatu statistical surveys such as the Vanuatu 1999 Household
Income and Expenditure Survey, 1993 Agricultural Survey, 2004 Tourist
Survey, and a monthly Hotel Occupancy Survey as baseline data sources.
Some data were updated to 2005 values from earlier data based on growth
factors. MCC assessed the data it used as the best available, but noted
that some data have shortcomings. Other data--such as the amount of
spending by expatriates in Vanuatu, and the percentage of tourist spending
that remains in the country--were not available, and MCC had to estimate
their value. Since signing the compact with Vanuatu, MCC has funded
additional surveys to improve the quality of baseline data and track key
compact performance indicators.

Figure 5: MCC's Logic Model for the Vanuatu Compact

MCC calculated several different direct and induced benefits for the
compact's projects over a 20-year period. Direct benefits include the
local value added of construction spending in the economy, reduced
spoilage of agricultural goods, time saved in transit on the improved
roads, and reduced user costs for operators of ships, aircraft, and
vehicles.^14 Induced benefits are those projected to result from Vanuatu's
response to new economic opportunities in tourism and agriculture. For
example, MCC assumes that the projects will cause Vanuatu tourism to grow
at 7 percent per year instead of the recent rate of 5 percent; tourists
will increase their daily spending; new hotel rooms will be constructed
and hotel occupancy will increase; and crop, livestock, and fisheries
production will increase. Benefits from the completed projects are counted
as beginning in full in 2008 or 2009 and are assumed to increase a minimum
of 3 percent every year thereafter. MCC expects the benefits of the
compact to flow from different sources, depending on the project and its
location. In Efate, the Ring Road is expected to provide direct benefits
from decreased road user costs and induced benefits through tourism and
foreign resident spending. In Santo, MCC anticipates benefits from all of
these efforts, as well as the induced benefit of increased agricultural
production. On other islands, where tourism is not as developed, the
benefits primarily derive from user cost savings and increased
agriculture.^15

^14MCC determined reduced road user costs by means of (1) traffic counts
prepared by Vanuatu's PWD and MCA-Vanuatu for the MCC and (2) an
estimation of road user costs per kilometer based on the Highway Design
and Maintenance Standards Model (HDM)-III developed by the World Bank.
HDM-III is a software package that provides the economic decision criteria
for evaluating road construction or maintenance strategies. MCC determined
the costs to road users by multiplying the average annual daily traffic on
the road by the length of the road and by the average road user cost per
kilometer. Road user cost includes vehicle operating costs and the value
of passengers' time. The difference in user cost before and after
construction provides the benefit.

  Costs

To calculate compact construction and maintenance costs, MCC used existing
data from previous contractor studies and data from the Vanuatu PWD. In
MCC's economic model, construction costs are assumed to be incurred in the
first year after compact signing. Although listed on the cost side of the
model, MCC also counted the effect of the construction spending on the
economy as a benefit in the year it takes place.^16 According to MCC's
contractor, it used two different primary sources in developing the MCC
cost estimates:

           o The contractor used its own 2003 analysis for the ADB projects
           and updated its analysis to account for inflation and changes in
           project scope.

           o The contractor used a cost analysis prepared by a different
           contractor for the government of Vanuatu, dated approximately
           2004, as a basis for MCC's cost estimates for the Santo and Efate
           projects. MCC's contractor then prepared a new estimate because
           MCC's projects had a different scope of work. We requested from
           MCC a copy of the original report to the government of Vanuatu.
           According to MCC officials, MCC did not have it and the government
           of Vanuatu was not willing to provide the report to MCC for our
           review. These Santo and Efate projects represent 56 percent of
           compact cost.

^15Although not included in the economic analyses, other benefits may
accrue to Vanuatu from the MCC compact. Increased economic activity in
tourism may have spillover benefits for other sectors of the economy, and
the welfare of Vanuatu's citizens may improve due to increased access to
health care and educational opportunities.

^16MCC assumed that 16 percent of the total cost of construction would be
captured in the local economy and counted this as a benefit.

           MCC assumed a cost for continued annual maintenance based on past
           estimated maintenance costs provided by the PWD. MCC applied
           assumptions and estimated maintenance costs based on incomplete
           information, because the PWD did not track maintenance costs in
           its budget. For some projects, MCC assumed that maintenance costs
           would decrease once the construction project was complete. MCC
           also estimated and included the cost of a periodic major
           reinvestment for some projects. For example, on the Efate Ring
           Road, MCC assumed a cost for rehabilitation in 2017 and 2026.
			  
           Beneficiaries

           MCC assessed the number of poor, rural beneficiaries by
           determining the catchment area--the geographic area in which
           benefits may be expected to accrue. For example, MCA-Vanuatu
           officials told us that they defined the catchment area in Efate as
           consisting of villages within 3 miles of the rehabilitated Efate
           Ring Road. MCC used Vanuatu maps to identify villages in the
           catchment area and used the 1999 Vanuatu National Population and
           Housing Census to determine the number of persons living in those
           villages. In all, MCC calculated that approximately 65,000 poor,
           rural people on the eight islands would benefit from MCC projects.
			  
			  MCC Used Projected Benefits and Costs to Determine ERR and Effects
			  on GDP and Per Capita Income
  
           MCC used its projection of costs and benefits over a 20-year
           period as the basis for calculating three summaries of the
           compact's impact: its ERR, effect on per capita income, and effect
           on GDP (see fig. 6). The compact's ERR reflects the ratio of the
           benefits of the compact in relation to its costs, expressed as a
           percentage. For the Vanuatu compact, MCC reported an overall
           compact ERR of 24.7 percent over 20 years.^17
			  
^17In its final April 2006 economic analysis, MCC adjusted this
calculation downward slightly to 24.2 percent.

           Figure 6: MCC Benefits Calculations as Source of Statements about
           Impact of Vanuatu Compact

           MCC also prepared a sensitivity analysis to assess how a range of
           possible outcomes would affect compact results. MCC's tests
           included

           o a 1-year delay of the start date for accrued benefits;

           o a 20 percent increase of all costs;

           o a 20 percent decrease of all benefits; and

           o a "stress test," with a 20 percent increase of all costs and a
           20 percent decrease of all benefits.

           MCC tested a best-case scenario based on a 10 percent increase in
           benefits and a 10 percent decrease in costs. For the overall
           compact, MCC calculated a best-case ERR of 30.2 percent and
           worst-case of 13.9 percent. MCC also used the cost and benefit
           data for the compact projects to determine the projects' expected
           impacts on Vanuatu's per capita income and GDP.
			  
			  MCC's Data Do Not Support Its Portrayal of Compact Benefits

           The anticipated benefits of the Vanuatu compact that MCC stated in
           March 2006 in the compact and in its Congressional Notification do
           not accurately reflect MCC's supporting data. MCC projects an
           increase in per capita income of $200 by 2010, but this increase
           is cumulative; MCC's underlying analysis shows a projected
           increase of $51 in 2010. Similarly, while MCC's statement about
           the compact's transformational effect on GDP could be interpreted
           as increasing the growth rate, the projected effect on GDP
           represents an increase to the level, not the growth rate, of GDP.
           The supporting data show the growth rate remaining roughly the
           same as it would be without the compact. In addition, the 65,000
           rural poor cited by MCC may share less than half of the compact's
           benefits with others in Vanuatu who are not poor or not rural. In
           addition, although MCC's estimates of project benefits and
           beneficiaries are generally reasonable, MCC made some calculation
           errors and questionable assumptions in developing these estimates.
           Correcting MCC's calculations of the per capita income benefit
           slightly reduces the benefit to $49, or 3.7 percent, in 2010;
           correcting calculations and fully discounting some of MCC's
           assumptions regarding beneficiaries would reduce the estimate of
           beneficiaries on Santo and Efate by about one-third.
			  
			  Portrayal of Per Capita Income Benefit Suggests Larger Effect
			  Than Data Support

           MCC's portrayal of the compact's projected impact suggests a
           greater effect on Vanuatu's per capita income than its analysis
           supports. In its publicly available documents such as the Vanuatu
           compact, and the congressional notification, issued in March 2006,
           MCC states that the transportation infrastructure project "is
           expected to have a transformational impact . . . increasing
           average income per capita (in real terms) by approximately $200,
           or 15 percent of current income per capita, by 2010." In addition,
           MCC's investment memo states that the compact will cause per
           capita income to increase by $488, or 37 percent, by 2015.

           MCC's statements suggest that as a result of the program, average
           incomes in Vanuatu will be 15 percent higher in 2010 and 37
           percent higher in 2015 than they would be without the compact.
           However, MCC's underlying data show that these percentages
           represent the sum of increases from per capita income in 2005 that
           MCC projects for each year. For example, according to MCC's data,
           Vanuatu's per capita income in a given year between 2006 and 2010
           will range from about 2 percent to almost 4 percent higher than in
           2005; however, MCC sums these percentages as 15 percent, without
           stating that this percentage is a cumulative increase from 2005.
           Our analysis of MCC's data shows that actual gains in per capita
           income, relative to income in 2005, would be $51, or 3.9 percent,
           in 2010 and $61, or 4.6 percent, in 2015 (see fig. 7).

Figure 7: Vanuatu Compact's Projected Impact on Real Per Capita Income
According to MCC Statement and MCC Data Relative to 2005 Per Capita Income

Note: MCC's statement: "Increasing average income per capita (in real
terms) by approximately $200 or 15 percent of current income per capita by
2010" and by $488--37 percent--by 2015.

Figure 8 further illustrates MCC's methodology in projecting the compact's
impact on per capita income levels for 2010 and 2015.

Figure 8: MCC Methodology for Projecting Vanuatu Compact's Impact on Real
Per Capita Income

Portrayal of GDP Benefit Differs from Underlying Analysis

Like its portrayal of the per capita income benefit, MCC's portrayal of
the program's GDP impact differs from that supported by the underlying
data. In the compact and the 2006 Congressional Notification, MCC states
that the compact will have a transformational effect on Vanuatu's economy,
causing GDP to "increase by an additional 3 percent a year." Given the GDP
growth rate of about 3 percent that MCC expects in Vanuatu without the
compact, MCC's statement of a transformational effect suggests that the
GDP growth rate will rise to about 6 percent. However, MCC's underlying
data show that although Vanuatu's GDP growth rate will rise to about 6
percent in 2007, in subsequent years the GDP growth rate will revert to
roughly the rate MCC assumes would occur without the compact,
approximately 3 percent (see fig. 9). Although MCC's data show that the
compact will result in a higher level (i.e., dollar value) of GDP, the
data do not show a transformational increase to the GDP growth rate.

Figure 9: Vanuatu GDP Growth with and without MCC Compact

Notes:

According to MCC, "GDP is expected to increase by an additional 3 percent
a year as a result of the MCA program."

According to MCC data, the compact will have a small impact on GDP growth
rate in later years. In 2010 to 2015, the GDP growth rate resulting from
the compact will be 3.1 percent, compared with 3 percent without the
compact.

Portrayal of Poverty Reduction Does Not Quantify Benefits to Rural Poor

MCC's portrayal of the compact's projected impact on poverty does not
identify the proportion of the financial benefits that will accrue to the
rural poor. In the compact and the Congressional Notification, MCC states
that the program is expected to benefit "approximately 65,000 poor, rural
inhabitants living nearby and using the roads to access markets and social
services." However, in its underlying documentation, MCC specifies that 43
percent of the monetary benefits are expected to go to the local
population. The remaining 57 percent of the benefits are expected to
accrue to other beneficiaries, including expatriate tourism services
providers, transport providers, government, and local businesses (see fig.
10). Given that many of MCC's benefits flow to the expatriate-dominated
tourism sector, MCC makes an appropriate assumption that the local
population will not receive all the benefits. However, the compact,
congressional notifications, and other publicly available MCC documents do
not provide this information.

Figure 10: MCC Analysis of Distribution of Vanuatu Compact Benefits

Note: MCC defines "local population" as comprising local producers, local
consumers, and inhabitants of rural communities.

Although MCC expects that 43 percent of monetary benefits will go to the
local population, MCC does not establish the proportion of
local-population benefits that will go to the rural poor. Because MCC
defines the local population as "local producers, local consumers and
inhabitants of remote communities," the 65,000 poor, rural beneficiaries
that MCC projects may share local-population benefits with those who are
urban and are not poor.

Estimates of Compact Benefits Are Generally Reasonable, but Some Calculations
and Assumptions Are Problematic

MCC's documentation states that it used conservative assumptions in
developing its benefit estimates. Although our fieldwork and meetings in
Vanuatu generally affirmed MCC's assumptions about benefits, many
participants in our Vanuatu discussion groups noted that the positive
impacts of the MCC compact are contingent on other factors, such as the
development of tourist facilities and activities. However, our review of
MCC's analyses identified some calculation errors in its determination of
the compact's impact on per capita income and estimation of the number of
compact beneficiaries, as well as questionable assumptions in regard to
the beneficiary population. (See app. III for a detailed discussion of
these calculation errors and the effect of MCC's assumptions.)

           o Calculation errors. To determine per capita income effects, MCC
           incorrectly adjusted the value of a benefit stream for inflation
           that was already presented in real (i.e., inflation-adjusted)
           terms. In addition, MCC failed to account for population growth in
           projecting per capita income effects from the compact. Although
           these errors largely cancel each other, correcting them reduces
           MCC's projection of the per capita income benefit slightly, from
           $51 to $49 in 2010 and from $61 to $57 in 2015. Correcting
           calculation errors significantly increases the number of rural
           beneficiaries; MCC's estimate of compact beneficiaries in Santo
           and Efate did not take into account population growth since the
           1999 census.

           o Assumptions. Our analysis of MCC's data for Santo and Efate
           indicates that MCC's count of beneficiaries may be overestimated.
           MCC's count of poor, rural beneficiaries includes all rural
           inhabitants in the catchment area, indicating that MCC assumes
           that all rural inhabitants are poor. However, according to ADB
           reporting, rural poverty in Vanuatu is widespread but not
           universal. The poverty level, defined as having an income of one
           U.S. dollar per day, is 51 percent in rural areas.^18 Further, in
           defining the catchment areas in Efate and Santo, MCC assumed that
           residents of villages near existing paved portions of the Ring
           Road not improved by MCC, as well as those on off-shore islets,
           would benefit fully from the compact.

           Correcting calculation errors and fully discounting MCC's
           assumption including these residents in the catchment area would
           reduce the beneficiary count on Efate and Santo by 32
           percent--from 26,553, as stated by MCC, to 18,070--indicating that
           MCC may overestimate the compact's beneficiaries.
			  
^18In its Vanuatu Monitoring and Evaluation Plan, MCC is using the poverty
measure proposed by Vanuatu, the fraction of individuals with monthly cash
income less than 20,000 vatu (approximately $185 U.S. dollars or $6 per
day).

           Several Risks May Lead to Reduced Project Benefits

           We identified five key risks that may affect the Vanuatu compact's
           projected impact on poverty reduction and economic growth. First,
           the contingencies included in MCC's calculations of construction
           costs may not be sufficient to cover average transportation
           project overruns. Second, although the compact's benefits are
           projected to begin shortly after completion of the projects, some
           of these benefits are likely to accrue more slowly. Third, while
           the benefit projections assume continued maintenance of completed
           projects, MCC's ability to ensure such maintenance will end in
           2011, and Vanuatu's record of road maintenance is poor. Fourth,
           the projected induced benefits from expanded tourism and
           agriculture depend on businesses and rural inhabitants responding
           to opportunities created by improved infrastructure. Fifth,
           efficiency gains, such as time saved in transit, may not result in
           increased per capita income, as MCC projects. Our analysis of
           these areas of risk illustrates the extent to which MCC's
           projections of benefits are dependent on assumptions of immediate
           realization of benefits, successful long-term maintenance,
           realization of induced benefits, and benefits from efficiency
           gains.^19
			  
			  Construction Costs May Exceed Contingencies

           Although MCC considered the risk of construction cost increases,
           the contingencies used in its calculations may not be sufficient
           to cover actual construction costs. We received documentation from
           MCC's contractor of its MCC cost estimates for only 5 of MCC's 11
           construction projects. For these five, it used a design
           contingency of 20 percent.^20 In its due diligence book, MCC
           states that its cost estimates include physical contingencies,
           with an average value of 15 percent, and price contingencies, with
           an average value of 12 percent. However, cost overruns of more
           than 20 percent occur in many transportation projects.^21 For
           example, a study of more than 250 transportation projects in
           Europe, North America, and elsewhere found that costs for all
           projects were 28 percent higher, on average, than forecasted at
           the time of decision to build, while road projects averaged
           escalations of 20.4 percent.^22 Further, as MCC's analysis notes,
           the risk of excessive cost overruns is significant in a small
           country such as Vanuatu, because (1) few comparable projects have
           been undertaken in Vanuatu and, consequently, little previous cost
           information is available and (2) no local contractors are capable
           of undertaking a project of this size, and foreign contractors may
           apply large margins to cover unknown factors.^23 Any construction
           cost overrun must be made up within the Vanuatu compact budget by
           reducing the scope, and therefore the benefits, of the compact
           projects.^24 Reduced project benefits would in turn reduce the
           compact's ERR and effects on per capita income and GDP.
			  
^19We modeled risk scenarios with alternative methods of looking at
compact benefits that assume phasing of costs and benefits and phasing
coupled with lack of maintenance, lack of induced benefits, and lack of
monetized efficiency gains. See appendix V for more information about our
methodology.

^20For one of these five, the $400,000 South West Bay airstrip project,
the estimate also included an additional 10 percent construction
contingency.			  

           In addition, because MCC was unable to provide the data that it
           used to produce the estimates for the Santo and Efate projects,
           the project costs may be at further risk from unchecked
           assumptions and data (see app. IV).
			  
^21GAO recently reported this occurring as part of U.S. Agency for
International Development (USAID)-funded road construction in a
tsunami-affected area of Indonesia. In Indonesia, construction cost per
mile increased by 75 percent, USAID reduced the length of road to be built
by more than one third and the agency may extend the planned completion
date by 5 months. GAO, Foreign Assistance: USAID Signature Tsunami
Reconstruction Efforts in Indonesia and Sri Lanka Exceed Initial Cost and
Schedule Estimates and Face Further Risks, [36]GAO-07-357 (Washington,
D.C.: February 28, 2007).

^22Bent Flyvbjerg, Mette Skamris Holm, and Soren Buhl, "Underestimating
Costs in Public Works Projects: Error or Lie?," Journal of the American
Planning Association, Vol. 68, No. 3 (2002), cited in GAO, Highway and
Transit Investments: Options for Improving Information on Projects'
Benefits and Costs and Increasing Accountability for Results,
[37]GAO-05-172 (Washington, D.C.: January 24, 2005).

^23MCC cites the "design-construct" contract proposed for the MCA program,
which will include design and construction of all the projects as one
package, as key to mitigating this risk. However, MCC's analysis also
recognized that nonconstruction-related issues (such as access to parts of
the project site) have the potential to delay the contractor and increase
costs and that such issues can be significant for major road upgrade
projects where the competing interests of the contractor, adjacent
villages, and the general public must be balanced. MCC's analysis states
that, to help manage the risk of project-related disputes and delays, MCC
plans to have experienced consultants work with local PWD staff who have
an understanding of the social and cultural issues.

^24According to the compact, the government of Vanuatu must pay any
environmental mitigation and remediation costs in excess of the budget.

           Compact Benefits Are Likely to Accrue More Slowly Than Projected

           Although MCC's analysis assumes compact benefits from 2008 or
           2009--shortly after the end of project construction--we found that
           benefits are likely to accrue more slowly. Our document review and
           discussions with tourism services providers and agricultural and
           timber producers suggest that these businesses will likely react
           gradually to any increased market opportunities resulting from
           MCC's projects, in part because of constraints to expanding
           economic activity.^25 For example:

           o According to tourism officials and business owners, several
           types of activities need to progress to enable industry growth.
           Factors needed to foster tourism growth in Vanuatu include
           marketing Vanuatu as a tourist destination; promoting different
           types of products to different markets; improving domestic and
           international air and sea access; developing or upgrading the
           electric power, water, and road infrastructures; and recruiting
           and training workers. Tourism service providers in Santo and Efate
           identified improving air capacity as an important need.

           o According to an official from the fisheries department in Santo,
           besides rough roads, the lack of ice-making equipment and lack of
           feed are important barriers to developing Santo's small-scale
           aquaculture.

           o Timber production expands slowly; for example, a timber company
           owner in Santo stated that newly planted trees could not be
           harvested for about 15 years.

           o In both Efate and Santo, feeder roads are in worse condition
           than the project roads and are critical to agriculture
           shipments.^26

           These constraints suggest that future benefits related to tourism
           and agriculture will be phased in once transportation
           infrastructure is improved. Moreover, MCC assumes all construction
           costs will be incurred in the first year, instead of phasing these
           costs over the multiyear construction schedule. Our analysis shows
           that if costs are phased over 3 years and benefits are phased over
           5 years, overall compact ERR declines from 24.2 percent, as
           projected by MCC, to 16.5 percent (see app. V).
			  
^25Benefits from construction activities may also be reduced by a delayed
procurement. MCA-Vanuatu officials initially told us they anticipated
issuing an invitation for bid to contractors by the end of February 2007.
As of May 2007, the invitation had not yet been issued. MCC currently
expects construction to begin in 2008, further reducing the likelihood of
benefits starting in 2007 as MCC anticipated in its analyses.

^26It is unclear whether responsibility for feeder roads lies with the
government at the central or provincial level; neither takes
responsibility. The Santo PWD expects to focus on maintaining feeder roads
once the East Coast Road is done.

           Project Maintenance after Compact Expiration Cannot Be Ensured

           Uncertainty about the maintenance of completed transportation
           infrastructure projects after 2011 may affect the compact's
           projected benefits. According to World Bank and ADB officials,
           continuing donor involvement is needed to ensure the maintenance
           and sustainability of completed projects. In addition, during our
           visits to Efate and Santo, tourism and agriculture business
           representatives cited continued road maintenance as a critical
           concern. However, although MCC has budgeted $6.2 million for
           institutional strengthening of the Vanuatu PWD, MCC has no means
           of ensuring the maintenance of completed projects after the
           compact expires in 2011; the Millennium Challenge Act limits
           compacts to 5 years.

           Although the conditions precedent^27 to the Vanuatu compact
           require the government's commitment to ongoing project
           maintenance, World Bank and ADB officials told us, based on their
           experience with Pacific countries, that covenants such as these
           are difficult to enforce after the project is completed and often
           are not effective in ensuring sustainability. According to Vanuatu
           government officials, funds from vehicle registration and related
           fees are available for maintenance of the completed projects;
           however, the government has not dedicated these funds for this
           purpose. In addition, according to donor reporting, the government
           has failed to sustain maintenance of previous donor projects,
           primarily because of a lack of funds and a shortage of resources,
           skills, and capabilities. Some of the department's equipment on
           Santo Island is more than 27 years old, and newer equipment
           remains idle for long periods because spare parts are scarce;
           overall, the department lacks adequate equipment for Santo's
           1,000-kilometer road network. Although the compact provides funds
           for equipment and for technical assistance to increase capacity,
           MCC cannot ensure the maintenance of infrastructure after the
           compact ends.
			  
^27Conditions precedent are specific steps that must be completed by the
compact country prior to MCC's providing a disbursement. According to MCC
officials, the government has thus far met or exceeded the requirements of
these conditions.

           Poor maintenance performance will reduce the benefits projected in
           the MCC compact.^28 Our analysis shows that with phasing of costs
           and benefits, no large periodic maintenance expenditures, and
           inadequate maintenance performance, overall compact ERR decreases
           from 24.2 percent, as projected by MCC, to 16.5 percent with
           phasing and to 13.8 percent without maintenance.^29
			  
			  Induced Benefits Require Tourism Providers and Agricultural
			  Producers to Respond to Opportunities

           The compact's induced benefits depend on the response of Vanuatu
           tourism providers and agricultural producers. However, constraints
           affecting these economic sectors--such as the air-passenger
           capacity limitations that affect Vanuatu's tourism--may prevent
           the sector from expanding as MCC projects, affecting the
           reliability of MCC's estimate of program benefits.

           Some development assistance organizations do not include induced
           benefits in calculations of ERR. In particular, World Bank
           officials told us they have not considered induced benefits in
           economic analyses of infrastructure for 10 years, because they
           deem such benefits too conjectural and subject to manipulation.^30
           For example, if a projected road is assumed to transform an area's
           agricultural production, it is easy to show large benefits and
           high ERR.^31
			  
^28In technical comments on a draft of this report, the World Bank noted
that initial construction quality and subsequent traffic can affect a
road's projected benefits and, if not included in the investment analysis,
can have important implications for the actual ERR that may be equal to,
or larger than, the impact of poor maintenance. The World Bank also noted
that weak governance and political corruption can also have a large impact
on a road's sustainability.

^29This simulation results in a modest impact on ERR because it reflects
both a reduction in future benefits and a reduction in future costs for
road maintenance.

^30In technical comments on a draft of this report, the World Bank noted
that it measures induced benefits directly, comparing base traffic levels
with expected growth in demand for transport as a result of investments.
This method results in relatively straightforward and transparent
calculations. The World Bank observed that models using other, more
indirect approaches to estimating traffic, such as examining the impact of
increased agricultural production, require careful consideration and that
it has generally used such methods less often over the last 10 years
because the results are sometimes unclear and sensitive to assumptions.
Typically, it uses such methods only when a road project is expected to
have significant restructuring effects--such as a completely new road or
bridge in an urban environment--that could drastically change land use
patterns.			  

           Limited response to the compact by tourism providers and
           agricultural producers would have a significant impact on compact
           benefits. For example, our analysis shows that the phasing of
           costs and benefits reduces the overall compact ERR from 24.2
           percent to 16.5 percent, and the omission of induced benefits
           (i.e., no producer response) further reduces the compact ERR to
           5.5 percent. Two projects would have a negative ERR.
			  
			  Efficiency Gains May Not Cause Measurable Change in Per Capita Income

           MCC counts efficiency gains--such as time saved because of better
           roads--as compact benefits. However, although efficiency gains
           could improve social welfare, they may not lead to changes in per
           capita income or GDP or be directly measurable as net additions to
           the economy.

           Excluding efficiency gains from ERR calculations reduces the
           compact's overall ERR. For example, our analysis shows that the
           phasing of costs and benefits and the omission of efficiency gains
           from the Vanuatu ERR calculation causes the overall compact ERR to
           decline from 24.2 percent to 16.5 percent with phasing and to 11.8
           percent without calculating efficiency gains. This analysis also
           results in a negative ERR for two projects (see app. V).^32
			  
			  Conclusions

           MCC obtained the input of knowledgeable stakeholders in projecting
           the benefits, costs, and number of beneficiaries of its compact
           with Vanuatu, which addresses one of Vanuatu's primary constraints
           to economic growth. However, MCC's public statements about these
           benefits--particularly its projection of the compact's effect on
           per capita income--suggest greater impacts than MCC's underlying
           data and analysis support. MCC's statements can be understood only
           by reviewing supporting source documents and spreadsheets, which
           are not publicly available. These gaps could lead to unrealistic
           expectations about the compact's effect within Vanuatu; for
           example, by suggesting that per capita incomes will increase so
           quickly, MCC suggests that its compact will achieve sustainable
           growth in a way that other donors to Vanuatu have not been able to
           achieve. Further, these gaps between MCC's statements and
           underlying analysis raise questions about other compacts'
           projections of transformative impact on country economies or
           economic sectors. Without an accurate representation of the
           compacts' projected benefits, the extent to which the compacts
           further MCC's goals of poverty reduction, economic growth, and
           transformative development cannot be accurately evaluated.
           Further, MCC's economic analyses for the Vanuatu compact did not
           fully consider the phasing of costs and benefits, such as the time
           required to improve the infrastructure and for the economy to
           respond to the opportunities from the improved infrastructure.
           Additionally, MCC's analysis did not fully account for risks that
           could substantially reduce compact benefits.
			  
^31However, the lack of consideration of induced benefits is not without
its flaws. World Bank officials told us that in the fast-growing economy
of Vietnam, the World Bank funded a road on the basis of current use of
traffic without projecting the gains to the economy. The road is now very
congested, and the project was underinvested. According to the World Bank,
consideration of projected traffic increases--that is, induced
traffic--are taken into account only when the project is not justified on
the basis of existing traffic alone. However, rapid traffic growth is
considered exceptional.

^32The ERR in this case does not take into account opportunity costs and
is therefore an internal rate of return.

           Recommendations

           We recommend that the CEO of MCC take the following actions:

           o revise the public reporting of the Vanuatu compact's projected
           impact to clearly represent the underlying data and analysis;

           o assess whether similar statements in other compacts accurately
           reflect the underlying data and analysis; and

           o improve economic analysis by phasing the costs and benefits in
           compact ERR calculations and by more fully accounting for risks
           such as those related to continuing maintenance, induced benefits,
           and monetized efficiency gains as part of sensitivity analysis.
			  
			  Agency Comments and Our Evaluation

           MCC provided written comments regarding a draft of this report. We
           have reprinted MCC's comments, with our responses, in appendix VI
           and incorporated technical comments from MCC where appropriate.
           Although MCC did not directly acknowledge our recommendations, it
           questioned our finding of a gap between its portrayal of the
           compact's benefits and its underlying analysis. In addition, MCC
           responded to our discussion of beneficiary numbers and program
           risks. Following is a summary of MCC's comments and our evaluation
           of these comments.
			  
			  Compact Impact on Incomes and Growth

           MCC acknowledged that its use of projected cumulative compact
           impact on income and growth was misleading, but it asserted that
           (1) it had no intention to mislead and (2) its portrayal of
           projected compact benefits was factually correct and consistent
           with its underlying data. Our report does not state or imply that
           MCC intended to mislead; we have not determined why MCC made its
           statements portraying projected benefits on income and growth.
           Further, MCC's portrayal of the projected benefits could be
           considered factually correct and consistent with the underlying
           data only if the reader knows that the portrayal represents
           cumulative income and growth over 5 years. However, MCC's public
           documents and communications to Congress never describe the
           benefits it projects as being cumulative over 5 years. At issue in
           this finding are both the facts represented by MCC's data and
           calculations and MCC's representation of these data and
           calculations in its public statements. As our report notes, MCC's
           statements can be understood accurately only by reviewing
           supporting source documents and spreadsheets, which are not
           publicly available.

           In addition, MCC's portrayal of the compact's effect in cumulative
           terms is not consistent with its description of per capita income
           in Vanuatu in annex I of the compact, which states that per capita
           income in Vanuatu declined by 15.4 percent between 1994 and 2003.
           This calculation is consistent with the World Bank's data, which
           show Vanuatu's per capita income in 2003 as approximately 15.4
           percent less than its per capita income in 1994--a comparison of
           2003 and 1994 income levels, not a reflection of cumulative
           decline. If the methodology MCC uses to describe past per capita
           income in Vanuatu is used to describe the compact's projected
           future impact, then the projected increase in Vanuatu's per capita
           income in 2010 should be portrayed as 3.9 percent, not the
           cumulative 15 percent that MCC presents in its congressional
           notifications and public documents.

           MCC also commented that its data on the compact's effect on
           Vanuatu's GDP are consistent with its assessment that GDP will be
           perpetually 3 percent higher with the MCC investment than without
           it. We agree that MCC's data reflect this, but MCC's portrayal of
           this benefit in its public statements implies an effect on the
           growth rate, rather than the level, of Vanuatu's GDP. MCC couples
           its portrayal of the GDP benefit with a description of the
           compact's impact as "transformational"--that is, as MCC defines
           the term, having "a dramatic and long-lasting impact on poverty
           reduction through sustainable economic growth." However, although
           MCC's data show projected benefits to Vanuatu's economy, including
           an increase in the level of GDP, the projected change to the
           overall Vanuatu economy, as reflected by the GDP growth rate, is
           smaller than 3 percent. In 2010-2015, for example, the GDP growth
           rate increase resulting from the compact will be 0.1 percent.
			  
			  Compact Beneficiaries

           MCC commented that our report suggests that (1) nonpoor households
           could feasibly be excluded from the count of beneficiaries, (2)
           persons living on off-shore islets and in the hinterland should
           not be counted as beneficiaries, and (3) MCC's definition of the
           local beneficiary population includes urban dwellers. Regarding
           nonpoor households, we do not suggest that they could be excluded
           or that this would be desirable. Our report indicates that MCC's
           analysis shows there will be poor, rural beneficiaries as well as
           other beneficiaries and that MCC has not quantified what portion
           of the compact's benefits poor, rural beneficiaries will receive.
           Regarding off-shore and hinterland beneficiaries, we do question
           whether inhabitants of off-shore islets and of villages more than
           10 kilometers from MCC projects on Santo and Efate would benefit
           fully from MCC's projects. However, these persons are less than
           half of the total number we question. In Santo and Efate, 51
           percent of the catchment area population are not inhabitants of
           the hinterland or off-shore islets, but rather residents of Efate,
           near the tourism center and capital of Port Vila and along a
           portion of the Ring Road that is already paved. It is not clear
           how construction of a road elsewhere on the island will benefit
           these residents when they already have access to Port Vila via the
           existing paved road. Regarding the definition of "local
           population"--which MCC's underlying documentation says will
           receive 43 percent of compact benefits--MCC's written comments
           asserted that its definition of this population does not include
           urban dwellers. Although we asked MCC to support this assertion,
           MCC did not provide any additional documents. In separate
           comments, MCC added that although its documentation does not
           contain a very detailed reference specifically saying exactly how
           many business people, transport providers and tourism operators
           are foreigners and urban, MCC believes it reasonable to conclude
           that urban dwellers are included in these categories. However, we
           note that not every urban dweller is a business person, transport
           provider, or tourism operator and that some urban beneficiaries
           would therefore fall into the "local population" that receives an
           estimated 43 percent of benefits. Further, MCC's definition of the
           term in its internal documents includes persons who are not poor
           and not rural.
			  
			  Program Risks

           MCC commented that it undertakes sensitivity analysis for all of
           its ERR calculations, although it may focus on risks other than
           those we identified. In addition, MCC states that it differs
           significantly from us regarding the precise nature and severity of
           program risks in general. As our report notes, we offer several
           alternative scenarios to illustrate the maximum impact of certain
           areas of risk on projected compact benefits. The scenarios look at
           the effect of individual risk areas, rather than the possibility
           of projects' failing to achieve certain benefits in multiple areas
           simultaneously. Regarding induced benefits and efficiency gains,
           we note that MCC's documentation of its economic analysis
           questions the valuing of efficiency gains as induced benefits to
           the Vanuatu economy. Regarding maintenance, MCC states that its
           compact provides a significant amount of money to assist the
           Vanuatu PWD in providing timely and adequate maintenance. However,
           as our report notes, Vanuatu's record of maintenance is poor, and
           achieving a change in maintenance performance through donor
           assistance would show a significant break from past experience.
           Finally, regarding contingencies, MCC questions our use of the
           Flyvbjerg, Holm, and Buhl study as a basis for evaluating its use
           of contingencies and states that it provided us with price and
           contingency information to support its use of contingencies. The
           Flyvbjerg, Holm, and Buhl study and our report use the same basis
           of comparison for evaluating the adequacy of contingencies, and we
           have updated the language of our report to make this clear. In
           addition, as our report notes, we did not receive full
           documentation of the cost estimates and contingencies that MCC
           used in developing its economic analyses. We received no
           supporting documentation for the cost and contingency estimates
           for the Santo and Efate projects, which account for 56 percent of
           the total compact budget. A key document that MCC's contractor
           said it used as a basis for developing the Santo and Efate costs
           was unavailable to us, because the government of Vanuatu would not
           provide it to MCC. Cost overruns remain a risk to project
           benefits, and we have presented them as such.

           As agreed with your offices, unless you publicly announce the
           contents of this report earlier, we plan no further distribution
           until 30 days from the report date. At that time, we will send
           copies of this report to interested congressional committees as
           well as to the CEO of MCC and the Secretary of the Treasury. We
           will make copies available to others on request. In addition, this
           report will be available at no charge on the GAO Web site at
           http://www.gao.gov.

           If you or your staff have any questions about this report, please
           contact David Gootnick at (202) 512-3149 or [email protected].
           Contact points for our Offices of Congressional Relations and
           Public Affairs may be found on the last page of this report. GAO
           staff who made major contributions to this report are listed in
           appendix VII.

           Sincerely yours,

           David Gootnick
			  Director
			  International Affairs and Trade
			  
			  Appendix I: Objectives, Scope, and Methodology

           At the request of the Chairman of the House Committee on Foreign
           Affairs, we examined the Millennium Challenge Corporation's (MCC)
           economic analyses of the Vanuatu compact. Specifically, we
           examined

           o MCC's methods of projecting the compact's economic benefits and
           methods of calculating the benefits,

           o MCC's portrayal and analysis of the projected benefits, and

           o risks that could affect the compact's impact on poverty
           reduction and economic growth.

           To accomplish our objectives, we reviewed MCC's record of due
           diligence, including its opportunity memo, investment memo, due
           diligence book, economic analysis spreadsheets, and contractor
           reports. These documents are restricted from public dissemination
           based on MCC policy, but MCC made them available to us for
           analysis. We also reviewed the supporting documents used in MCC's
           due diligence process, such as previous donor reports and studies.
           We supplemented this review with interviews with MCC officials in
           Washington, D.C., and MCA-Vanuatu officials in Vanuatu. We then
           used this information to (1) summarize the process that MCC used
           to develop its economic model and (2) identify the logic, data,
           methods, and assumptions used to determine the compact's projected
           costs and benefits, number of beneficiaries, and effects on per
           capita income, gross domestic product (GDP), and poverty.

           We evaluated MCC's statements of compact impact in its public
           documents such as the congressional notification and the compact
           itself--as well as MCC's internal investment memo--by comparing
           these statements with the underlying analyses and data used to
           support them. We analyzed the November 2005 investment memo
           spreadsheet containing the source data and calculations used for
           MCC's statements about per capita income and GDP and reviewed its
           contents and formulas to determine the methodology and source data
           for MCC's statements and to assess the accuracy of the
           calculations. We then used MCC's own data and formulas from the
           November 2005 investment memo to determine MCC's impact in
           individual years and compare the results with the statements made
           in the Vanuatu compact and congressional notification. We could
           not validate most of MCC's underlying data and assumptions,
           because some data, such as construction and maintenance cost data,
           were not available or could not be checked within the time frames
           of our engagement. Further, we used the calculations and formulas
           from MCC's November 2005 investment memo, corrected population
           figures, and the data from MCC's April 2006 final economic
           analysis to recalculate MCC's statements about per capita income
           impacts. We reviewed MCC's economic analysis spreadsheet to find
           the amount of benefits that MCC's analysis assumed would flow to
           poor, rural beneficiaries and reviewed spreadsheets provided by
           MCA-Vanuatu to determine the assumptions, methodology and data
           used to calculate the beneficiary population in Santo and Efate.
           We compared the spreadsheet data with Vanuatu village maps and
           population figures from the 1999 National Population and Housing
           Census provided by MCA-Vanuatu to evaluate MCC's calculations and
           assumptions about the villages included as beneficiaries.

           We identified risks to MCC's compact results based on our review
           of MCC's internal documentation and donor and academic literature.
           We also met with Vanuatu, MCC, and contractor officials and
           interested parties such as tourism and agriculture business owners
           in Vanuatu. We focused our field work on the three projects
           representing more than half of the compact budget and on Vanuatu's
           two most populous islands, Santo and Efate. We examined MCC's due
           diligence documentation and contacted MCC's contractor to
           determine the methodology used to estimate construction costs. We
           then compared MCC's procedures to findings and GAO's best practice
           criteria for cost estimation. We compared MCC's assumptions about
           maintenance continuing for 20 years to previous donor experiences
           gleaned from donor reporting or interviews with the Asian
           Development Bank, Australian Agency for International Development,
           European Union, International Monetary Fund, New Zealand Agency
           for International Development, and World Bank. We determined the
           need for the phasing of benefits based on discussions with
           knowledgeable agriculture and tourism representatives in Vanuatu
           and a review of MCC's source documentation. We further compared
           MCC's treatment of induced benefits to the approach taken by the
           World Bank and MCC's treatment of direct benefits to economic
           principles about the monetization of efficiency gains. We modeled
           risk scenarios that provide alternative methods of looking at
           compact benefits using MCC's April 2006 economic model
           spreadsheet. In modeling these risks, we used the data from MCC's
           economic analyses; we did not validate these data. In these
           scenarios, we determined the effect of the scenario on the
           economic rate of return by assuming the phasing in of costs over 3
           years and of benefits over 5 years, first determining the effect
           of phasing alone, then examining the phasing of costs and benefits
           coupled with the following:

           o Lack of maintenance--eliminating all future large maintenance
           expenditures and taking away a fraction of the growing expected
           benefits each year, so that at the end of the 20-year period,
           2027, benefits basically return to their 2012 level.

           o Lack of induced benefits--deleting those benefits from the cost
           model.

           o Lack of monetized efficiency gains--deleting those benefits from
           the cost model.
			  
			  Appendix II: Prior Development Assistance to Vanuatu 

           Of Pacific island countries, Vanuatu receives among the largest
           amounts of aid; however, Vanuatu's economic performance and social
           development since 1990 have lagged behind other countries in the
           region. Vanuatu depends on donor assistance for its development
           budget; from its gaining independence, in 1980, to 2002, aid
           receipts averaged around 17 percent of GDP. In 1997, Vanuatu
           initiated structural adjustment efforts with ADB assistance
           designed to stimulate the private sector and help realize the
           country's growth potential in agriculture and tourism. However,
           these reforms have had a limited impact on growth in
           private-sector investment and GDP. In 2003, the government of
           Vanuatu prepared the Priorities and Action Agenda to focus
           available resources, including donor assistance, on Vanuatu's
           development priorities. The current strategies of Vanuatu's major
           donors--which include Australia, France, New Zealand, the European
           Union, Japan, and MCC--are aimed at strengthening Vanuatu's
           productive sectors and supporting private sector-led development
           and are based on the Vanuatu government's stated goals. MCC
           anticipates that its average annual assistance level of $13.1
           million over the next 5 years will position the United States as
           one of Vanuatu's top two official donors (see table 1).

           Table 1: Average Annual Contributions of Top Five Donors to
           Vanuatu in 2004 and 2005
			  
(Dollars in millions)                             
Donor                 Average annual contribution 
Australia                                   $19.1 
France                                       $5.3 
New Zealand                                  $4.8 
European Union                               $4.6 
Japan                                        $3.5 

           Source: Organization for Economic Cooperation and Development.

           According to the ADB, Vanuatu's development budget has
           concentrated on building new capital infrastructure; however,
           according to donor reporting, these assets have not been
           adequately maintained. From 1996 to 2004, Vanuatu received about
           $35 million in assistance for transportation infrastructure
           projects including the construction and rehabilitation of roads,
           bridges, and wharfs, and the extension and upgrading of airport
           terminals and runways.

           According to studies by the World Bank and ADB, Vanuatu faces
           significant development challenges. Some of Vanuatu's challenges
           are shared by other small states and island economies,^1 while
           others are either the result of, or exacerbated by, political
           instability and, according to ADB, inappropriate policies or
           regulations. Like other small island economies, Vanuatu's
           agriculture and tourism sectors are sensitive to extreme weather
           conditions and natural disasters. For example, frequent cyclones
           cause production shocks and restrict the range of viable crops and
           tree species. Vanuatu's narrow production base also leaves the
           economy open to economic shocks from changes in commodity prices,
           especially for copra. Despite these constraints, Vanuatu has a
           wealth of natural resources and growth opportunities in
           agriculture, fisheries, and tourism. However, growth has been
           hindered by substantial barriers to private-sector development,
           including political uncertainty,^2 high costs of doing business,
           poor and costly infrastructure, lack of a secured transactions
           framework,^3 and issues with land tenure.
			  
^1According to the World Bank, some of the characteristics shared by small
states such as Vanuatu include remoteness and insularity, susceptibility
to natural disasters, limited institutional capacity, limited
diversification, reliance on external trade, and limited access to
external capital. Commonwealth Secretariat/World Bank, Small States:
Meeting Challenges in the Global Economy (Washington, D.C.: 2000).

^2Beginning in the early 1990s, Vanuatu has witnessed a series of
short-lived coalition governments, comprising numerous parties and
involving frequent changes of ministers. There have been eleven
governments in power since 1990, with most lasting only about 1 year. This
constant turnover has delayed reforms as coordination of policies and
obtaining consensus for needed reforms has proved difficult. International
Monetary Fund, Vanuatu: Selected Issues, IMF Country Report No. 07/93
(Washington, D.C.: 2007).

^3The legal framework for secured transactions is incomplete. According to
the ADB, the legal system does not allow for property to be used as
collateral for loans, particularly outside of Port Vila and Luganville.
Thus borrowers can neither purchase property on credit nor obtain loans
against their own assets. This accounts for high borrowing costs, higher
interest rates, smaller loans offered, and loans of shorter maturity,
which limit the private sector's access to credit and entrepreneurship.
IMF, Vanuatu: Selected Issues.

           Appendix III: MCC Estimates of Per Capita Income Impact
           and Poor, Rural Beneficiaries
			  
			  Per Capita Income Impact

           In its economic analysis for Vanuatu, MCC did not correctly
           account for population growth in projecting per capita income and
           incorrectly deflated the value of the benefit stream that was
           already in real terms. When the benefit stream is no longer
           deflated, its value increases. However, when population growth is
           accounted for, the per capita income benefit is reduced. MCC
           assumes that the population is growing by approximately 2.6
           percent each year, but MCC's analysis of cumulative income per
           capita benefits for 2006 to 2015 is based on 2005 population
           estimates, rather than on an estimate of population that increases
           each year. Figure 11 contrasts the estimated increase in income
           per capita over the 2005 baseline in the given year with and
           without our corrections to deflating of benefits and to population
           growth. The errors largely cancel out, reducing the per capita
           income benefit projected by MCC from $51 (3.9 percent) to $49 (3.7
           percent) in 2010 and from $61 (4.6 percent) to $57 (4.3 percent)
           in 2015.

           Figure 11: MCC Compact's Projected Impact on Vanuatu Per Capita
           Income with and without Adjusted Population Estimates

           Note: In addition to adjusting for population growth and
           eliminating the deflating of benefits already in real terms, our
           revised projections of per capita income benefits use April 2006
           data from MCC, including a benefit start date of 2007 for
           construction spending, a benefit start date of 2008 for some
           projects, and updated numeric values for costs and benefits.

           The impact of the MCC compact on per capita income is a relatively
           small addition to Vanuatu's expected per capita income without
           MCC. Figure 12 shows, in dollar terms, MCC's projected impact on
           Vanuatu's per capita income compared with the expected levels of
           per capita income, using MCC's assumptions of 3 percent annual
           growth in GDP and 2.6 percent in population.

           Figure 12: MCC Compact's Projected Impact on Vanuatu Per Capita
           Income in Real Terms Relative to Per Capita Income without MCC

           Note: Assumes 0.4 percent growth in per capita income, based on
           MCC's assumed GDP growth of 3 percent without the compact and
           MCC's assumed Vanuatu population growth rate of 2.6 percent.
			  
			  Calculation of Beneficiaries

           MCC calculated approximately 65,000 beneficiaries on the eight
           islands receiving MCC projects (see table 2).^1 However, MCC's
           determination of beneficiary numbers in Santo and Efate contained
           calculation errors and relied on problematic assumptions that
           villages near portions of the road not improved by MCC, as well as
           those on off-shore islets, would fully benefit from the compact.
           Correcting the calculation and data errors and fully discounting
           MCC's assumptions would reduce the beneficiary count on Efate and
           Santo by 32 percent, from MCC's stated 26,553 to 18,070.
			  
^1Before signing the compact, MCC determined the number of potential
beneficiaries but did not quantify targets for the compact's effect on
poverty. MCC has funded an additional survey that will provide updated
baselines and the basis for establishing these targets.

           Table 2: Number of Poor, Rural Beneficiaries of Vanuatu Compact,
           as Calculated by MCC


Island              Project                        Number of beneficiaries 
Efate               Round Island Road                               13,819 
Santo               Port Olry Road                                   7,404 
Santo               South Coast Road Bridges                         5,330 
Subtotal, Santo and                                                 26,553 
Efate                                                                      
Other projects                                                             
Tanna               Whitesands Road                                  6,190 
Malekula            Lits Lits Road                                   5,676 
Malekula            Southwest Bay Airstrip                           4,570 
Pentecost           Loltong Wharf and North-South                    7,858 
                       Road                                                   
Epi                 Lamen Bay Wharf                                  1,560 
Malo                Malo Roads Upgrading                             3,480 
Ambae               Ambae Roads Reconstruction                       9,340 
Total                                                               65,227 			  

           Source: GAO analysis of MCC data.

           Our examination of the calculation of MCC's beneficiary estimate
           for the Efate Ring Road, Santo East Coast Road, and Santo South
           Coast Road Bridges shows that MCC's estimate of rural
           beneficiaries in Santo and Efate would be increased by
           approximately 5,100 persons, or 19 percent, if adjustments were
           made to correct data errors^2 (see table 3).
			  
^2We identified the following errors: (1) MCC's beneficiary estimates use
data from the 1999 census and have not been updated to 2005. This results
in an underestimate of the beneficiary population. (2) In the data
provided to us, the total of the populations of villages included in the
MCC analysis does not match the beneficiary count in the monitoring and
evaluation plan. (3) For the Efate Ring Road project, MCC's data
double-counted some villages. (4) For all three projects, the 1999
population figures used for some villages do not match the populations
shown in the 1999 Vanuatu census. For the Santo East Coast Road, the
population figures do not match for 53 of 56 villages in the catchment
area.

Table 3: Effect of Calculation and Data Errors on Beneficiary Population
in Efate and Santo

                                                           Santo South        
                                     Efate Ring Santo East  Coast Road        
                                           Road Coast Road     Bridges  Total 
MCC Statement: beneficiary count                                           
in monitoring and evaluation plan     13,819      7,404       5,330 26,553 
GAO recalculation of MCC data^a       15,439      7,518       8,732 31,689 
Difference                             1,620        114       3,402  5,136 

Source: GAO analysis of MCC data.

aGAO's recalculation uses 1999 Vanuatu census data, removes double-counted
villages, and updates the population to 2005 using MCC's assumed
population growth factor in rural areas of 2 percent per year.

MCC also made problematic assumptions about the geographic reach of
project benefits that increased the number of beneficiaries by including
more rural areas as benefiting from the project. Fully discounting these
assumptions would reduce the beneficiary count from our adjusted total of
31,689 to 18,070 (see table 4).

Table 4: Effect of MCC Assumptions on Beneficiary Population in Efate and
Santo

                                                         Santo South          
                                   Efate Ring Santo East  Coast Road          
                                         Road Coast Road     Bridges    Total 
GAO calculation using MCC data      15,439      7,518       8,732   31,689 
Discounting population of areas    (6,889)          0     (3,222) (10,111) 
not adjacent to MCC road                                                   
project                                                                    
Discounting population of          (2,779)      (196)       (533)  (3,508) 
off-shore islets                                                           
Total^a                              5,770      7,322       4,978   18,070 

Source: GAO analysis of MCC data.

aBecause of rounding, numbers in columns may not sum to totals.

In estimating beneficiaries of both the Efate Ring Road and the Santo
South Coast Road Bridges projects, MCC included the populations of
villages not located near MCC's road projects and populations that face
geographic barriers to using the projects. These population counts were
not prorated or adjusted to account for the barriers that some villages
face in realizing benefits from MCC's projects.

           o In Efate, MCC assumed that the populations of villages located
           along an already improved portion of the road adjacent to Port
           Vila in the Mele, Eratap, and Erakor areas--which will not be
           addressed by the MCC compact--would benefit from construction of
           the road elsewhere on the island (see fig. 13). MCA-Vanuatu stated
           that the populations of these villages were included because they
           own land where the road will be constructed and most of the
           villagers' gardens are located in these areas. (The source for
           this assumption is not documented.) The inhabitants of these
           villages represent 45 percent of the total number (15,439) of
           poor, rural Efate beneficiaries. MCC also did not account for
           "most" villagers and instead included the entire village
           population in these areas in its totals.

           Figure 13: MCC Efate Ring Road Catchment Area

           o For the Santo South Coast Road Bridges, MCC included villages in
           West Santo more than 10 kilometers from the bridge projects and
           villages across a river that vehicles can cross only by fording;
           after heavy rains, the river becomes impassable. MCC's compact
           program does not include a bridge across this river. Using a 2005
           population estimate, we determined that MCC included 3,222 people
           in the count of beneficiaries of South Coast Road Bridges based on
           this assumption--37 percent of the total number (8,732) of South
           Coast Road Bridges beneficiaries.

           For all three projects, MCC assumed that inhabitants of off-shore
           islets would fully benefit, although they are not connected to the
           improved roads and must travel by boat to reach them. MCC
           officials told us that they expect that residents of off-shore
           islets will benefit from increased tourism spending after
           construction of roads and would benefit from decreased
           transportation costs after coming to the mainland. Using 2005
           population estimates, we determined that 2,779 persons (18 percent
           of the Efate Ring Road beneficiary population) live on off-shore
           islets. For the Santo East Coast Road and South Coast Road
           Bridges, respectively, 196 (3 percent of the beneficiary
           population) and 533 (6 percent of the beneficiary population) live
           on off-shore islets.
			  
			  Appendix IV: MCC Cost Estimates for the Vanuatu Compact

           The reliability of MCC's cost estimates for the Vanuatu compact
           construction cannot be evaluated, because MCC was unable to fully
           provide the data that it used to produce them. MCC's contractor
           based its construction cost estimates for MCC's transportation
           infrastructure activities primarily on two sources. To build cost
           estimates for seven of the compact projects, representing 15
           percent of total compact cost, the contractor used a 2003 analysis
           it had performed for the ADB. To prepare cost estimates for the
           three projects on Santo and Efate, representing 56 percent of
           compact cost, the contractor used estimates prepared by a second
           contractor for the government of Vanuatu.^1

           o For the seven ADB projects, MCC's file of documents used in its
           due diligence included the original cost estimate but did not
           include an analysis that showed how these costs were updated to
           account for inflation and changes in the scope of the work. MCC's
           documentation states that these previous costs were multiplied by
           1.17 to account for inflation and also adjusted for changes in the
           scope.^2

           o For the three projects on Santo and Efate, MCC's file did not
           include any source information or analysis. MCC's contractor later
           told us that it derived the costs of the projects on Santo and
           Efate from an existing report by a second contractor to the
           government of Vanuatu, dated approximately 2004. MCC's contractor
           told us that because MCC's scope of work was more realistic than
           that proposed in 2004, which included realigning portions of the
           roadways, the MCC cost estimate was prepared anew using source
           information from the previous report. We requested, but did not
           receive, a copy of this report to the government of Vanuatu and a
           copy of the analysis showing how MCC developed its cost estimates
           using this source data. According to MCC officials, the government
           of Vanuatu was not willing to provide the report to MCC for our
           review.

           According to previous GAO work, high-quality, reliable cost
           estimates should be accurate, comprehensive, well-documented, and
           validated.^3 MCC cost estimates used in the economic analyses have
           shortcomings in each of these areas.
			  
^1The remaining 29 percent of costs are for warehouses, and for oversight
and administrative functions.

^2MCC's contractor provided us with spreadsheet calculations of the costs
for five of the compact projects for MCC. The sheets included estimated
quantities and unit costs but did not indicate how these quantities and
costs were derived.

^3We developed these criteria as part of a forthcoming publication on best
practices in cost estimation and implemented the criteria in a recent
report. See GAO, Telecommunications: GSA Has Accumulated Adequate Funding
for Transition to New Contracts but Needs Cost Estimation Policy,
[38]GAO-07-268 (Washington, D.C.: February 23, 2007).

           o Accurate. The source data for the estimates is from 2003 and
           approximately 2004, and changes in the cost environment beyond
           general inflation may not be accounted for. MCC also incorrectly
           calculated the length of the Santo East Coast Road as 70
           kilometers in its economic analyses, instead of the 55 kilometers
           found by a PWD survey, according to a PWD official. Other elements
           of this criterion cannot be independently assessed without the
           source data for the cost estimates used in the model.

           o Comprehensive. We could not assess the cost estimates against
           this criterion without the source data for the estimates used in
           the model.

           o Well-documented. MCC did not fully document any cost estimation
           procedures and checks that it performed as part of its due
           diligence. The due diligence documents we received from MCC
           contain a narrative describing the cost estimate procedures, but
           no documentation showing the analysis that MCC's contractor
           performed to update previous cost estimates.

           o Validated. MCC's due diligence documentation contains no
           evidence that it obtained an independent cost estimate to verify
           the work of its contractor in updating its own estimate for the
           ADB projects and reworking the estimates of another contractor for
           the Efate and Santo projects. An independent cost estimate
           provides the estimator with an unbiased test of the reasonableness
           of the estimate and reduces the cost risk associated with the
           project by demonstrating that alternate methods generate similar
           results.
			  
			  Appendix V: Illustrative Alternative Calculations of Vanuatu
			  Compact Impact

           This appendix describes the results from several alternative
           scenarios, which disaggregate the effects of the various types of
           benefits on the overall ERR, GDP and per capita income in order to
           illustrate the significance of each of the components.

           MCC reported different values of summary statistics in the
           investment memo dated November 2005 as compared with the final
           analysis that reported more recent data (see table 5).
			  
Table 5: Summary of Compact Impacts as Presented by MCC, with GAO
Recalculations

                                       Increase in level                      
                                       of GDP with MCC in  Increase in level  
                                       year 5 vs. GDP      of real income per 
                                       without MCC in year capita in 2010 vs. 
                        Compact ERR    5                   2005 baseline      
MCC's anticipated    24.7 percent   3.2 percent         3.9 percent^a      
effect as presented                                                        
in investment memo                                                         
MCC's anticipated    24.2 percent^b 3.2 percent         3.7 percent^c      
effect using                                                               
corrected or updated                                                       
data                                                                       

Source: GAO analysis of MCC data.

aDerived from MCC's own data and calculations.

bIn its updated and final April 2006 economic analysis, MCC adjusted the
24.7 percent reported in its congressional notification downward slightly
to 24.2 percent.

cGAO calculation after correcting MCC calculation errors.			  

To illustrate the maximum impact of certain areas of risk on projected
compact benefits, we modified the model using MCC data for different
scenarios. We phased costs and benefits to mirror more accurately the rate
of construction and accrual of potential benefits. Phasing costs has the
effect of increasing ERR, while phasing benefits decreases it; the net
effect is to decrease ERR. Within this modified framework that includes
phasing, we eliminated each main type of benefit--direct and induced--one
at a time to understand their relative importance.

           o For "no induced benefits," we eliminated all induced benefits
           from expanded tourism, agriculture, or land development as a
           result of the compact but did not change the direct benefits.

           o For "efficiency gains are not monetized," we eliminated all
           direct benefits such as cost savings for existing users of a road
           or benefits from newly generated traffic, which are attributable
           to the project, but did not change the induced benefits.

           Finally, we examined the impact of poor maintenance performance
           after compact expiration in 2011 on total benefits from the phased
           project. We modeled this by eliminating all future large
           maintenance expenditures and taking away a fraction of the growing
           expected benefits each year, so that at the end of the 20-year
           period, 2027, benefits basically return to their 2012 level.

           These scenarios present illustrative cases and should be
           considered as extremes of a possible range of cases that capture
           uncertainty effects on summary statistics, such as the ERR, GDP,
           and income per capita. MCC has observed that overall benefits may
           understate the true impact of the compact because spillover
           effects, such as easier access to schools and medical facilities
           due to the improved infrastructure, are not captured by these
           measures. These effects represent long-term benefits to society as
           whole, which are not accounted for by the analysis performed by
           either MCC or GAO.

Table 6: Summary of Compact ERR, GDP, and Per Capita Income impacts under
Alternative Scenarios of Accounting for Risks to Benefits

                                      Increase in GDP    Increase in real     
                                      with MCC in year 5 income per capita    
                                      relative to GDP    level in year 5      
                                      without MCC in     relative to 2005     
GAO analysis^a        Compact ERR  year 5             baseline             
(1) Costs are phased  16.5 percent 1.1 percent        1.4 percent          
over 3 years and                                                           
benefits are phased                                                        
over 5 years                                                               
Costs are phased over 3 years and benefits are phased                      
over 5 years, and                                                          
(2) induced benefits  5.5 percent  0.7 percent        0.8 percent          
are not realized^b                                                         
(3) efficiency gains  11.8 percent 0.8 percent        1.0 percent          
are not monetized^c                                                        
(4) large-scale       13.8 percent 1.1 percent        1.3 percent          
maintenance is not                                                         
undertaken^d                                                               

Source: GAO analysis of MCC data.

aIn our analysis, benefits start in 2010 and are phased in equal
increments over 5 years, from 2010 to 2014, with phasing completed by year
5. Costs are phased over 3 years to reflect projected timing of
construction.

bIn addition to phasing benefits and costs, we eliminated induced effects
of the project on agriculture, tourism, fisheries, and the development of
subdivided beachfront land.

cIn addition to phasing benefits and costs, we eliminated road user cost
savings and savings from wasted surface trips, lost trips, longer
diversions, and enforced longer trips from road closures.

dIn addition to phasing benefits and costs, we assumed that total benefits
will increase, peak, and decrease such that their value in 2027 will equal
their original value in 2012. The large capital outlays for road
rehabilitation in 2017 and 2026 in Santo and Efate have been eliminated.

Table 7 reports the ERR for individual projects as modified in our
scenarios.

Table 7: Summary of Selected Compact Project Impacts under Alternative
Scenarios Accounting for Risks to Benefits

                                             Costs are phased over 3 years,
                                            benefits are phased over 5 years,
                                                           and
                             Costs are                                        
                             phased over 3                                    
                             years and      Induced  Efficiency  Large-scale  
                MCC's        benefits are   benefits gains are   maintenance  
                anticipated  phased over 5  are not  not         is not       
Projects     ERR          years          realized monetized   undertaken   
Overall      24.2 percent 16.5 percent   5.5      11.8        13.8 percent 
compact                                  percent  percent                  
Efate: Ring  20.6 percent 13.6 percent   negative 10.8        11.2 percent 
Road                                              percent                  
Santo: Port  33.8 percent 22.6 percent   10.3     15.5        19.4 percent 
Olry Road                                percent  percent                  
Santo: South 24.3 percent 14.3 percent   -4.0     12.4        10.8 percent 
Coast                                    percent  percent                  
Bridges                                                                    
Tanna:       17.7 percent 12.1percent    8.6      -0.3        8.4 percent  
Whitesands                               percent  percent                  
Road                                                                       
Malekula:    22.5 percent 15.1 percent   15.1     negative    14.3 percent 
Lits Lits                                percent                           
Road                                                                       
Malekula:    11.4 percent 6.3 percent    1.1      6.1 percent 2.4 percent  
South West                               percent                           
Bay Airstrip                                                               
Pentecost:   15.6 percent 8.9 percent    5.6      4.4 percent 4.7 percent  
Loltong                                  percent                           
Wharf/N-S                                                                  
Road                                                                       

Source: GAO analysis of MCC data.

Appendix VI: Comments from the Millennium Challenge Corporation

Note: GAO comments supplementing those in the report text appear at the
end of this appendix.

See comment 6.

See comment 5.

See comment 4.

See comment 3.

See comment 2.

See comment 1.

See comment 11.

See comment 10.

See comment 9.

See comment 8.

See comment 7.

See comment 16.

See comment 15.

See comment 14.

See comment 13.

See comment 12.

See comment 18.

See comment 17.

The following are GAO's comments on the Millennium Challenge Corporation
letter dated June 8, 2007.

GAO Comments

           1. MCC quotes from its statement in the compact and our statement
           in the text of the draft report. However, the figure we cite for
           per capita income in 2015--37 percent--originates in MCC's
           investment memo, not in our analysis.
           2. MCC notes that it emphasized repeatedly that its key statements
           in question were factually correct and that it did not intend to
           overstate benefits. At issue in this finding are both the facts
           represented by MCC's data and calculations and MCC's
           representation of these data and calculations in its public
           statements. As our report notes, MCC's portrayal of the compact's
           projected impact on income and economic growth can be understood
           as cumulative only by reviewing supporting source documents and
           spreadsheets, which are not publicly available. Our report does
           not assert or imply that MCC's portrayal was intentionally
           misleading; we did not determine through our audit, or address in
           the report, the reasons for the gap between MCC's portrayal and
           its underlying analysis.
           3. MCC comments that our assertion about its portrayal of income
           benefits is itself likely to mislead. In response to this comment,
           and as part of our final review process, we clarified the terms
           and presentation of our analysis of MCC's statements and data and
           made modifications as appropriate.
           4. MCC asserts that it "stated that per capita income would
           increase by a cumulative amount of 15.4% by 2010 relative to the
           2005 baseline" and that this is how it understood the underlying
           data in 2005. However, MCC documents in 2005 and later do not
           state that the portrayed effect on per capita income is
           cumulative. These documents include MCC's November 2005 Investment
           Memo; the March 2006 Vanuatu compact; the January 4 and March 7,
           2006, congressional notifications; MCC press release; and the
           April 2007 MCC Annual Report for 2006. We asked MCC to provide any
           publicly available document that discloses that the per capita
           income effect is cumulative; however, MCC did not provide any such
           document.
           5. MCC comments that its "underlying data do not support the
           conclusion that cumulative growth over 5 years would be only
           3.9%." Our report does not make any assertions about the
           cumulative growth rate in income per capita. As stated in figure
           7, we graph the increase in the level of income per capita in each
           year relative to the 2005 baseline. In 2010, this increase is 3.9
           percent.
           6. We agree with MCC that "the use of five-year cumulative figures
           is misleading." Since MCC's public documents do not state that the
           increase in per capita income is cumulative, we maintain that
           MCC's statements regarding the increase in average per capita
           income are inconsistent with its data.
           7. MCC comments that "the data also indicate that the effect on
           the per capita GDP growth rate will be moderate but rising, albeit
           much smaller than three percentage points." (We understand this
           statement to mean that the effect on aggregate GDP growth will be
           smaller than 3 percentage points, given that the sentences that
           follow the statement discuss aggregate, not per capita, GDP.) As
           the note to figure 9 of our report indicates, we agree with MCC
           that its data indicate that the compact will have a positive,
           albeit small, effect on GDP growth rate. MCC provided us with its
           computation showing that the compact's effect on GDP growth rate
           will increase over a 20-year period and will reach 0.54 percentage
           points in year 20--not the 3 percent growth rate increase implied
           by its portrayal of the compact's projected results. Based on
           MCC's calculation, the compact will enable the Vanuatu economy to
           double in size in 20 years. Without the compact, the Vanuatu
           economy will double in size in 24 years.
           8. MCC comments that "GAO's presentation suggests that [near and
           non-poor] households (local business people, government personnel,
           transport providers, and tourism service providers) could feasibly
           be excluded and that this would be desirable." We are not
           suggesting that such people could be feasibly excluded or that
           this would be desirable. Our report notes, "Given that many of
           MCC's benefits flow to the expatriate-dominated tourism sector,
           MCC makes an appropriate assumption that the local population will
           not receive all the benefits." Our report's analysis indicates
           that there will be poor, rural beneficiaries in addition to other
           beneficiaries but observes that MCC has not quantified what
           portion of the compact's benefits the poor, rural beneficiaries
           will receive.
           9. MCC states that we argue that the total number of rural
           beneficiaries should be reduced by one-third. In fact, our report
           observes that the total number of beneficiaries may be overstated.
           MCC specifically disputes our observation that households living
           in the hinterland or on off-shore islets may not be beneficiaries.
           However, of the 13,619 persons in the catchment areas that we
           question, 6,889 (51 percent) reside not in the hinterland or
           off-shore islets but near the Efate tourism center and capital of
           Port Vila and along a portion of the Ring Road that is already
           paved. Given that MCC's benefits in Efate are primarily based on
           increased tourist activity and road user cost savings from the
           improved road, it is not clear how construction of a road
           elsewhere on the island will lead to benefits when the existing
           paved road already provides access to Port Vila. As documented in
           appendix III, we do question whether inhabitants of off-shore
           islets and of villages more than 10 kilometers from MCC projects
           would benefit fully from MCC's projects. MCC includes these people
           in its count of beneficiaries without accounting for, and
           quantifying, the likely decrease in benefits that will accrue to a
           person who lives both across an occasionally impassable river and
           more than 10 kilometers from an MCC project bridge. As MCC's
           internal analysis states regarding the Santo South Coast Road,
           "the project does not entail a general improvement of the road,
           only of river crossings. Neither would it give all-weather access
           all the way to Tasiriki, and so to the west coast. It must be
           said, therefore, that the scale of these benefits are more than
           usually doubtful." Elsewhere in the report, it notes that the
           "benefits to the west coast [Santo] population will be slight."
           10. MCC asserts that its definition of "local population" does not
           include urban dwellers. Although we asked MCC to support this
           assertion, we received no additional documents. In separate
           comments, MCC added that although its documentation does not
           contain a very detailed reference specifically saying exactly how
           many business people, transport providers and tourism operators
           are foreigners and urban, MCC believes it is reasonable to
           conclude that urban dwellers are included in these categories.
           However, we note that not every urban dweller is a business
           person, transport provider, or tourism operator and that some
           urban beneficiaries would therefore fall into the "local
           population" that receives an estimated 43 percent of benefits. On
           the basis of MCC's definitions of "local population" and other
           terms, we determined that "local population" includes residents of
           nonrural areas. MCC's definition of the term local population
           states, "We have combined three categories: local producers, local
           consumers and inhabitants of remote communities. In reality these
           are the same people and we have labeled them simply `Local
           population.'" MCC defines local producers as including
           "landowners; existing and potential lessees of land; processors of
           primary produce, chiefly for export; and the owners and operators
           of tourist facilities." This definition does not distinguish
           between rural and urban dwellers. Further, given that expatriates
           dominate in Vanuatu's formal economy sectors and that the urban
           center of Port Vila is the center of Vanuatu's tourist industry,
           it is likely that "processors of primary produce, chiefly for
           export; and the owners and operators of tourist facilities"
           include urban dwellers and those that are not poor. Finally, MCC's
           definition of "inhabitants of remote communities" includes the
           poor inhabitants of the "informal urban settlements" on the
           outskirts of Port Vila. Although these people come from rural
           areas, they are now in an urban environment and included within
           MCC's definition of the local population.
           11. MCC states that it has "a reasonable expectation that the
           majority of the 65,000 [beneficiaries] are poor." However, MCC's
           public documents use the phrase "65,000 poor, rural inhabitants,"
           without acknowledging that not all of the 65,000 beneficiaries
           live in poverty. We agree that MCC would be correct to suggest
           that the majority of the projected beneficiaries are poor, as our
           report states: "The poverty level, defined as having an income of
           one U.S. dollar per day, is 51 percent in rural areas."
           12. About program risks in general, MCC states that it differs
           from us significantly regarding the precise nature and severity of
           risks. As we note in our report, we offer several alternative
           scenarios to illustrate the maximum impact of individual areas of
           risk on projected compact benefits. These scenarios examine the
           effect of individual risk areas and do not consider the
           possibility of projects' failing to achieve certain benefits in
           multiple areas simultaneously.
           13. We agree with MCC that attempting to quantify induced benefits
           is important and that our fieldwork generally affirmed MCC's
           assumptions. We maintain that quantifying induced benefits
           constitutes a risk to projected impact, because realization of
           such benefits depends on beneficiaries' response to the
           opportunities that the compact provides.
           14. We asked MCC to support its assertion that "it is highly
           plausible in a poor country that the vast majority of [time and
           cost] savings would be used to generate income." We did not
           receive additional documentation supporting this assertion.
           However, MCC stated that its assertion is based on extensive field
           consultations with beneficiary groups. MCC's documentation of its
           methodology for assessing efficiency gains states that its
           valuation of the average value of passengers' time is "extremely
           crude." MCC's documentation adds: "The very concept of `working
           hours' in a largely subsistence economy is open to question. And
           what is the value of tourists' time to the national economy? It is
           arguable that faster journey times allow tourists more time to
           spend on things other than travel. But there is no research to
           support a more sophisticated approach to valuing time."
           15. MCC notes other factors that may benefit the economy. As our
           report indicates (see footnote 15), we agree that "other benefits
           may accrue to Vanuatu from the MCC compact. Increased economic
           activity in tourism may have spillover benefits for other sectors
           of the economy, and the welfare of Vanuatu's citizens may improve
           due to increased access to health care and educational
           opportunities."
           16. MCC believes that we have overstated the risk from poor
           maintenance of the infrastructure investment and have not
           accounted for Vanuatu's increasing its maintenance spending by
           $4.6 million or the assistance and maintenance equipment MCC is
           providing to Vanuatu. We have added additional information to our
           report regarding MCC's program to improve maintenance capability.
           Nevertheless, maintenance remains a significant risk. As we noted
           in our report, Vanuatu's record on transport infrastructure
           maintenance is poor. Other donors, including officials from the
           ADB and the World Bank, identified maintenance as a significant
           challenge.
           17. MCC states that "the projects that were under consideration in
           the compact were at an advanced stage in preparation, either at
           preliminary design or final design stage," not at the alternatives
           analysis stage. Our previous report, which cited a study by
           Flyvbjerg, Holm, and Buhl as part of a larger report on improving
           cost and benefit analysis for transportation projects, used the
           term "alternatives analysis stage"; however, the study uses the
           term "time of decision to build" to define the point of
           comparison. MCC's "time of decision to build" is when it has
           completed its due diligence process and decided to sign a compact.
           As such, the basis for comparison in the study by Flyvbjerg et al.
           and in our report is identical. To help clarify the basis of
           comparison, we have replaced "alternatives analysis stage" with
           "time of decision to build." MCC states that it provided us the
           final cost estimates, including price and physical contingencies,
           for each subproject within the compact. However, as our report
           notes, we did not receive full documentation of the cost estimates
           and contingencies that MCC used in developing its economic
           analyses. We received no supporting documentation for the cost and
           contingency estimates for the Santo and Efate projects, which
           account for 56 percent of the total compact budget. A key document
           that MCC's contractor told us it used as a basis for developing
           the Santo and Efate costs was not available to us because the
           government of Vanuatu would not provide it to MCC. The
           documentation of MCC's cost estimates and contingencies for
           specific projects that we received shows a design contingency of
           20 percent. (One project--MCC's smallest project, the $400,000
           South West Bay airstrip--adds an additional 10 percent
           construction contingency.) This 20 percent contingency is slightly
           less than the average cost overrun found by Flyvbjerg et al. MCC
           may complete the projects at, under, or above a 20 percent
           contingency. However, cost overruns remain a risk to project
           benefits, and we have presented them as such.
           18. See comment 2.

Appendix VII: GAO Contact and Staff Acknowledgments

GAO Contact

David B. Gootnick, Director, 202-512-3149

Staff Acknowledgments

In addition to the person named above, Emil Friberg, Jr. (Assistant
Director), Gergana Danailova-Trainor, Reid Lowe, Angie Nichols-Friedman,
Michael Simon, and Seyda Wentworth made key contributions to this report.
Also, David Dornisch, Etana Finkler, Ernie Jackson, and Tom McCool
provided technical assistance.

(320451)

GAO's Mission

The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( [39]www.gao.gov ). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
[40]www.gao.gov and select "Subscribe to Updates."

Order by Mail or Phone

The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:

U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548

To order by Phone: Voice: (202) 512-6000
TDD: (202) 512-2537
Fax: (202) 512-6061

To Report Fraud, Waste, and Abuse in Federal Programs

Contact:

Web site: [41]www.gao.gov/fraudnet/fraudnet.htm
E-mail: [42][email protected]
Automated answering system: (800) 424-5454 or (202) 512-7470

Congressional Relations

Gloria Jarmon, Managing Director, [43][email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548

Public Affairs

Paul Anderson, Managing Director, [44][email protected] (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548

[45]www.gao.gov/cgi-bin/getrpt?GAO-07-909 .

To view the full product, including the scope
and methodology, click on the link above.

For more information, contact David Gootnick, (202) 512-3149,
[email protected].

Highlights of [46]GAO-07-909 , a report to the Chairman, Committee on
Foreign Affairs, House of Representatives

July 2007

MILLENNIUM CHALLENGE CORPORATION

Vanuatu Compact Overstates Projected Program Impact

In January 2004, Congress established the Millennium Challenge Corporation
(MCC) for foreign assistance. Congress has appropriated almost $6 billion
to MCC. As of March 2007, MCC had signed almost $3 billion in compacts
with 11 countries, including a 5-year, $65.7 million compact with Vanuatu.
MCC states that the Vanuatu compact will have a transformational effect on
the country's economy, increasing per capita income and GDP and benefiting
65,000 poor, rural people. GAO examined (1) MCC's methods of projecting
economic benefits, (2) MCC's portrayal and analysis of the projected
benefits, and (3) risks that may affect the compact's impact. GAO reviewed
MCC's analyses and met with officials and business owners in Vanuatu as
well as with other donors.

[47]What GAO Recommends

GAO recommends that the Chief Executive Officer of MCC (1) revise the
public reporting of the Vanuatu compact's projected impact, (2) assess
whether similar reporting in other compacts accurately reflects underlying
analyses, and (3) improve its economic analyses by more fully accounting
for risks to project benefits. MCC did not directly address our
recommendations but commented that it had not intended to make misleading
statements and that its portrayal of projected results was factual and
consistent with underlying data.

MCC projects that the Vanuatu compact's transportation infrastructure
projects will provide direct benefits such as reduced transportation costs
and induced benefits from growth in tourism and agriculture. MCC estimated
the costs and benefits over 20 years, with benefits beginning in full in
2008 or 2009 and growing each year, and it counted poor, rural
beneficiaries by defining the area where benefits were likely to accrue.
Using projected benefits and costs, MCC calculated the compact's economic
rate of return (ERR) and its effects on Vanuatu's gross domestic product
(GDP) and per capita income.

MCC's portrayal of the projected impact does not reflect its underlying
data. MCC states that per capita income will increase by approximately
$200, or 15 percent, by 2010 and by $488, or 37 percent, by 2015. However,
MCC's underlying data show that these figures represent the sum of
individual years' gains in per capita income relative to 2005 and that
actual gains will be $51, or 3.9 percent, in 2010 and $61, or 4.6 percent,
in 2015. MCC also states that GDP will increase by an additional 3 percent
a year, but its data show that after GDP growth of 6 percent in 2007, the
economy's growth will continue at about 3 percent, as it would without the
compact. MCC states that the compact will benefit approximately 65,000
poor, rural inhabitants, but this statement does not identify the
financial benefits that accrue to the rural poor or reflect its own
analysis that 57 percent of benefits go to others.

We identified five key risks that could affect the compact's projected
impacts. (1) Cost estimate contingencies may not be sufficient to cover
project overruns. (2) Compact benefits will likely accrue more slowly than
MCC projected. (3) Benefit estimates assume continued maintenance, but
MCC's ability to ensure maintenance will end in 2011, and Vanuatu's
maintenance record is poor. (4) Induced benefits depend on businesses' and
residents' response to new opportunities. (5) Efficiency gains, such as
time saved in transit, may not increase per capita income. Our analysis of
these areas of risk illustrates the extent that MCC's projections are
dependent on assumptions of immediate realization of benefits, long-term
maintenance, realization of induced benefits, and benefits from efficiency
gains.

Vanuatu Compact's Impact on Per Capita Income According to MCC Statement
vs. MCC Data

References

Visible links
  35. http://www.gao.gov/cgi-bin/getrpt?GAO-06-805
  36. http://www.gao.gov/cgi-bin/getrpt?GAO-07-357
  37. http://www.gao.gov/cgi-bin/getrpt?GAO-05-172
  38. http://www.gao.gov/cgi-bin/getrpt?GAO-07-268
  39. http://www.gao.gov/
  40. http://www.gao.gov/
  41. http://www.gao.gov/fraudnet/fraudnet.htm
  42. mailto:[email protected]
  43. mailto:[email protected]
  44. mailto:[email protected]
  45. http://www.gao.gov/cgi-bin/getrpt?GAO-07-909
  46. http://www.gao.gov/cgi-bin/getrpt?GAO-07-909
*** End of document. ***