Millennium Challenge Corporation: Projected Impact of Vanuatu
Compact Is Overstated (26-JUL-07, GAO-07-1122T).
In January 2004, Congress established the Millennium Challenge
Corporation (MCC) for foreign assistance. Congress has
appropriated almost $6 billion to MCC. As of March 2007, MCC had
signed almost $3 billion in compacts with 11 countries, including
a 5-year, $65.7 million compact with Vanuatu. MCC states that the
Vanuatu compact will have a transformational effect on the
country's economy, increasing per capita income and GDP and
benefiting 65,000 poor, rural people. This testimony summarizes a
July 2007 report (GAO-07-909) examining (1) MCC's methods of
projecting economic benefits, (2) MCC's portrayal and analysis of
the projected benefits, and (3) risks that may affect the
compact's impact. To address these objectives, GAO reviewed MCC's
analyses and met with officials and business owners in Vanuatu as
well as with other donors. In its July 2007 report, GAO
recommended that the Chief Executive Officer of MCC revise the
public reporting of the Vanuatu compact's projected impact;
assess whether similar reporting in other compacts accurately
reflects underlying analyses; and improve its economic analyses
by more fully accounting for risks to project benefits. MCC did
not directly address GAO's recommendations but commented that it
had not intended to make misleading statements and that its
portrayal of projected results was factual and consistent with
underlying data.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-07-1122T
ACCNO: A73390
TITLE: Millennium Challenge Corporation: Projected Impact of
Vanuatu Compact Is Overstated
DATE: 07/26/2007
SUBJECT: Budgeting
Cost analysis
Cost control
Economic growth
Economically depressed areas
Federal aid to foreign countries
Federal corporations
Financial statements
Impacted areas
Income statistics
Program evaluation
Risk management
Cost estimates
Vanuatu
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-07-1122T
* [1]Summary
* [2]Background
* [3]MCC Projected Compact's Impact Using Estimates of Benefits,
* [4]MCC's Data Do Not Support Its Portrayal of Compact Benefits
* [5]Several Risks May Lead to Reduced Project Benefits
* [6]Conclusions
* [7]Recommendations
* [8]GAO Contact and Staff Acknowledgments
* [9]Order by Mail or Phone
Testimony
Before the Subcommittee on Asia, the Pacific, and the Global Environment,
Committee on Foreign Affairs, House of Representatives
United States Government Accountability Office
GAO
For Release on Delivery
Expected at 2 p.m. EDT
Thursday, July 26, 2007
MILLENNIUM CHALLENGE CORPORATION
Projected Impact of Vanuatu Compact Is Overstated
Statement of David B. Gootnick, Director
International Affairs and Trade
GAO-07-1122T
Mr. Chairman and Members of the Subcommittee:
Thank you for the opportunity to discuss our recent work regarding the
Millennium Challenge Corporation's (MCC) compact with Vanuatu.1
In January 2004, Congress established MCC to administer the Millennium
Challenge Account for foreign assistance. MCC's mission is to reduce
poverty by supporting sustainable, transformative economic growth in
developing countries that have demonstrated a commitment to ruling justly
and democratically, encouraging economic freedom, and investing in people.
Congress appropriated almost $6 billion to MCC for fiscal years 2004
through 2007, and the President has requested an additional $3 billion in
MCC funding for fiscal year 2008. As of March 2007, MCC had signed 11
compacts totaling approximately $3 billion.2 MCC's 5-year, $65.7 million
compact with Vanuatu focuses on increasing economic activity and incomes
in rural areas through investments in transportation infrastructure.
Although MCC's Vanuatu compact is its smallest compact monetarily, it
provides by far the largest amount relative to the country's population
and gross domestic product (GDP).3
Publicly available documents show that MCC expects its compacts to
significantly benefit the countries' economies. In its Vanuatu compact and
its March 2006 congressional notification, MCC states that it expects the
compact to have a "transformational" impact--that is, as MCC defines it,
"a dramatic and long-lasting impact on poverty reduction through
sustainable economic growth."4 Using its projected benefit and cost data,
MCC calculated of the compact's expected economic rate of return (ERR)5
and impact on poverty reduction and economic growth. MCC states that its
compacts will provide or contribute to a transformational impact in 5 of
its 11 compacts.6
1GAO, Millennium Challenge Corporation: Vanuatu Compact Overstates
Projected Program Impact, [10]GAO-07-909 (Washington, D.C.: July 2007).
2An MCC compact is an agreement between the U.S. government, acting
through MCC, and the government of a country eligible for MCA assistance.
In June 2007, the MCC board approved a $362.6 million compact with Lesotho
and a $506.9 million compact with Mozambique.
3MCC's $65.7 million compact with Vanuatu provides $317 per capita; in
contrast, MCC's $547 million compact with Ghana--its largest
compact--provides $25 per capita. The amounts provided per capita by the
11 compacts signed to date range from $6 for Madagascar to $317 for
Vanuatu.
4Millennium Challenge Account, Best Practices in Compact Development
(Washington, D.C.: 2006).
In my testimony today, I will address (1) MCC's methods of projecting and
calculating the Vanuatu compact's impact on poverty reduction and economic
growth, (2) MCC's portrayal and analysis of the Vanuatu compact's
projected impact, and (3) risks that could affect the Vanuatu compact's
actual impact. This statement summarizes the findings in our report
released today.
In our report, we addressed our first and second objectives by evaluating
MCC's economic analysis of the Vanuatu compact proposal and MCC's public
statements about the compact's impacts. We could not validate most of
MCC's underlying data and assumptions, because the data were not available
or could not be checked within the time frames of our engagement. To
address our third objective, we identified risks to MCC's compact results,
based on our review of MCC's internal documentation, donor reporting, and
academic literature. To illustrate the impact of these risks on MCC's
economic analyses of ERR, GDP, and per capita income, we modeled the risks
using the data from MCC's economic analyses; however, we did not validate
these data. We focused our analysis and field work on MCC's three
transportation infrastructure projects on Vanuatu's two most populous
islands, Santo and Efate, which represent 56 percent of compact cost. We
interviewed Vanuatu and MCC officials and interested parties such as
tourism and agriculture business owners and contacted MCC's contractor. We
conducted this work from August 2006 through May 2007 in accordance with
generally accepted government auditing standards.
5Project cash flows are determined by comparing program spending against
future expected increases in value added or income. The internal rate of
return is calculated for these cash flows to summarize the economic
impact. MCC refers to this internal rate of return as the economic rate of
return.
6For example, in Nicaragua, MCC expects that the compact will transform
project areas into an engine of economic growth; in El Salvador, MCC
states that the compact provides an historic opportunity to transform the
country's economic development; and in Armenia, MCC is undertaking road
and irrigation projects to transform the economic performance of Armenia's
agricultural sector.
Summary
MCC projected the Vanuatu compact's impact by estimating the program's
benefits, costs, and beneficiaries and calculating the compact's effect on
per capita income, GDP, and poverty reduction. According to MCC,
transportation infrastructure improvements will provide direct benefits,
such as construction spending in the local economy, reduced transportation
costs, and improved services, as well as induced benefits from growth in
Vanuatu's tourism and agriculture sectors. MCC estimated the value of
these benefits over a 20-year period, beginning in full in 2008 or 2009
and growing each year. MCC developed its project cost estimates based on
existing cost estimates prepared for the government of Vanuatu and for
another donor. To determine the number of poor, rural beneficiaries, MCC
defined a catchment area--the geographic area in which benefits may be
expected to accrue--using maps of Vanuatu and data from the most recent
Vanuatu census. Using its projected benefit and cost data, MCC calculated
the compact's ERR by comparing projected benefits with projected costs;
calculated the compact's impact on per capita income by determining the
total benefits and dividing the total value by Vanuatu's baseline
population; and calculated the compact's impact on Vanuatu's GDP by
computing the total benefits added to the economy.
In the compact and the congressional notification, MCC portrays projected
impacts on per capita income and GDP that do not reflect the underlying
data and analysis, which are not publicly available. Also, MCC does not
establish the proportion of monetary benefits that will accrue to the
rural poor.
o Per capita income. MCC states that as a result of the compact,
per capita income will increase by approximately $200, or 15
percent, by 2010 and $488, or 37 percent, by 2015. This statement
suggests that per capita income in 2010 and 2015 will be,
respectively, 15 percent and 37 percent higher than without the
compact. However, MCC's data show that these percentages represent
sums of per capita income gains for individual years. The actual
gains in per capita income, relative to income in 2005, would be
$51, or 3.9 percent, in 2010 and $61, or 4.6 percent, in 2015.
o GDP. MCC states that Vanuatu's GDP will increase by "an
additional 3 percent a year." However, MCC's underlying data and
calculations show that although the level of Vanuatu's GDP will
grow by 6 percent in 2007, the economy's growth rate in subsequent
years will continue at approximately 3 percent, the growth rate
that MCC assumes would occur without the compact.
o Poverty reduction. MCC states that the compact is expected to
benefit approximately 65,000 poor, rural inhabitants "living
nearby and using the roads to access markets and social services."
According to MCC's underlying documentation, 57 percent of the
compact's monetary benefits will accrue to tourism services
providers, transport providers, government workers, and local
businesses and 43 percent of the benefits will go to the local
population--that is, local producers, local consumers, and
inhabitants of remote communities. However, MCC does not establish
the proportion of local-population benefits that will go to the
rural poor.
Our analysis shows five key areas of risk that may affect the
Vanuatu compact's actual impact on poverty reduction and economic
growth.
o Construction costs. The contingencies included in MCC's
calculations of construction costs may not be sufficient to cover
potential cost overruns. The risk of excessive cost overruns is
especially significant in a small country such as Vanuatu. Any
construction cost overrun could cause MCC to reduce the compact's
scope and therefore its benefits.
o Timing of benefits. Although MCC projects that the compact's
benefits will begin shortly after completion of the projects, some
benefits are likely to accrue more slowly. For example, according
to agricultural and timber producers, their businesses will likely
respond gradually to any increased market opportunities.
o Project maintenance. MCC's benefit projections assume continued
maintenance of completed projects; however, its ability to ensure
such maintenance will end in 2011. Moreover, previous donors to
Vanuatu have found the country's maintenance of donor projects to
be poor. Reduced maintenance would lead to reduced benefits from
the project.
o Induced benefits. MCC projects that induced benefits from
Vanuatu's tourism and agriculture--for example, increased tourist
traffic and agricultural trade--will lead to expansion of these
economic sectors. However, realization of such benefits depends on
businesses' and rural inhabitants' responses to opportunities
created by the compact's infrastructure improvements.
o Efficiency gains. MCC's projections count efficiency gains from
the infrastructure improvements, such as time saved in transit, as
direct benefits. However, such gains may not be put to economic
use or result in increased per capita income as MCC projects.
Accounting for these risks could reduce overall compact ERR from
24.2 percent, as projected by MCC, to between 5.5 percent and 16.5
percent.7
To help MCC better express and determine the impact of its
compacts, our report recommends that MCC's Chief Executive Officer
(CEO) (1) revise the public reporting of the projected impact of
the Vanuatu compact, (2) assess whether similar statements in
other compacts accurately reflect underlying data, and (3) improve
MCC's economic analysis by phasing costs and benefits and more
fully accounting for risks to project benefits. In comments on a
draft of our report, MCC responded that it had not intended to
make misleading statements and that its portrayal of projected
results was factual and consistent with underlying data.
Background
Vanuatu consists of 83 islands spread over hundreds of miles of
ocean in the South Pacific, 1,300 miles northeast of Sydney,
Australia. About 39 percent of the population is concentrated on
the islands of Santo and Efate. Vanuatu's capital, Port Vila, is
on Efate, and Vanuatu's only other urban center, Luganville, is on
Santo.
In the past decade, Vanuatu's real GDP growth averaged 2 percent,
although more rapid population growth led to a decline in per
capita GDP over the same period. Average growth of real GDP per
capita was negative from 1993 to 2005. An estimated 40 percent of
Vanuatu's population of about 207,000 has an income below the
international poverty line of $1 per day. Agriculture and tourism
are the principal productive sectors of Vanuatu's economy,
contributing approximately 15 percent and 19 percent to GDP,
respectively. Although agriculture represents a relatively small
share of Vanuatu's overall economy, approximately 80 percent of
Vanuatu's residents live in rural areas and depend on subsistence
agriculture for food and shelter. The tourism sector is dominated
by expatriates of foreign countries living in Vanuatu, who also
predominate in other formal sectors of the economy such as
plantation agriculture and retail trade.
On May 6, 2004, MCC determined that Vanuatu was eligible to submit
a compact proposal for Millennium Challenge Account funding.8
Vanuatu's proposal identified transportation infrastructure as a
key constraint to private-sector development. The timeline in
figure 1 shows the development and implementation of the Vanuatu
proposal and compact.
7MCC expresses the compact's ERR--the ratio of its benefits and costs--as
a percentage.
Figure 1: Development and Implementation of Vanuatu Compact
The $65.7 million Vanuatu compact includes $54.5 million for the
rehabilitation or construction of 11 transportation infrastructure assets
on 8 of Vanuatu's 83 islands, including roads, wharves, an airstrip, and
warehouses (see fig. 2). The compact also includes $6.2 million for an
institutional strengthening program to increase the capacity of the
Vanuatu Public Works Department (PWD) to maintain transportation
infrastructure.9 The remaining $5 million is for program management and
monitoring and evaluation. More than half of the compact, $37 million, is
budgeted for three road projects on Santo and Efate islands. The compact
provides for upgrading existing roads on both islands; the compact also
includes five new bridges for an existing road on Santo.10
8The Millennium Challenge Act of 2003 requires MCC to determine whether
countries are eligible for MCA assistance each fiscal year. Countries with
per capita income at or below a set threshold may be selected as eligible
for assistance if they meet MCC indicator criteria and are not statutorily
barred from receiving U.S. assistance. MCC uses 16 indicators divided into
three categories: Ruling Justly, Encouraging Economic Freedom, and
Investing in People. To be eligible for MCA assistance, countries must
score above the median relative to their peers on at least half of the
indicators in each category and above the median on the indicator for
combating corruption. GAO, Millennium Challenge Corporation: Compact
Implementation Structures Are Being Established; Framework for Measuring
Results Needs Improvement, [11]GAO-06-805 (Washington, D.C.: July 28,
2006).
9The institutional strengthening program includes $5.74 million for
equipment purchases; of this amount, $1.4 million is provided directly to
PWD and the remainder will purchase equipment for the use of the MCC
construction contractor, to be turned over to the PWD in specified
condition 4 years later.
10As of March 2007, MCC had disbursed $1.72 million in compact funds, or
about 16 percent of planned disbursements by that date.
Figure 2: MCC Vanuatu Projects by Size and Location
MCC's compact with Vanuatu and congressional notification state that the
compact will have a transformational impact on Vanuatu's economic
development, increasing average per capita income by approximately
$200--15 percent--by 2010 and increasing total GDP by "an additional 3
percent a year." MCC's investment memo further quantifies the per capita
income increase as $488--37 percent--by 2015.11 The compact and the
congressional notification also state that the compact will provide
benefits to approximately 65,000 poor, rural inhabitants (see fig. 3).
Figure 3: MCC Statement of Impacts in March 2006 Congressional
Notification
MCC Projected Compact's Impact Using Estimates of Benefits, Costs, and Catchment
Area
In projecting the impact of the Vanuatu compact, MCC estimated the
benefits and costs of the proposed infrastructure improvements. MCC also
estimated the number of beneficiaries within a defined catchment
area--that is, the geographic area in which benefits may be expected to
accrue. MCC used the estimated benefits and costs to calculate the
compact's ERR and impact on Vanuatu's GDP and per capita income.
MCC's analysis determined that the compact will reduce transportation
costs and improve the reliability of access to transportation services for
poor, rural agricultural producers and providers of tourism-related goods
and services and that these benefits will, in turn, lead to increases in
per capita income and GDP and reduction in poverty. MCC projects several
direct and induced benefits from the compact's infrastructure improvement
projects over a 20-year period, beginning in full in 2008 or 2009 and
increasing by at least 3 percent every year.
11The "investment memo" is an MCC internal document prepared by MCC's
compact assessment team and submitted to MCC's investment
committee--consisting of MCC's Chief Executive Officer (CEO), vice
presidents, and other senior officials. The committee reviews the memo and
decides whether to recommend proceeding to compact negotiations.
o Direct benefits. MCC projects that direct benefits will include,
for example, construction spending, reduced transportation costs,
and time saved in transit on the improved roads.
o Induced benefits. MCC projects that induced benefits from
tourism and agriculture will include, for example, increased
growth in Vanuatu tourism, tourist spending, and hotel occupancy
and increased crop, livestock, and fisheries production.
Figure 4 illustrates MCC's logic in projecting the compact's
impact.
Figure 4: MCC's Logic Model for the Vanuatu Compact
MCC expects compact benefits to flow from different sources,
depending on the project and its location. In Efate, the Ring Road
is expected to provide direct benefits from decreased road user
costs and induced benefits through tourism and foreign resident
spending. In Santo, MCC anticipates similar benefits as well as
the induced benefit of increased agricultural production. On other
islands, where tourism is not as developed, MCC expects benefits
to derive primarily from user cost savings and increased
agriculture.12
To calculate construction and maintenance costs13 for the
transportation infrastructure projects, MCC used existing cost
estimates prepared for the government of Vanuatu14 and for another
donor as well as data from the Vanuatu PWD.
To estimate the number of poor, rural beneficiaries, MCC used
Vanuatu maps to identify villages in the catchment area and used
the 1999 Vanuatu National Population and Housing Census to
determine the number of persons living in those villages. In all,
MCC calculated that approximately 65,000 poor, rural people on the
eight islands would benefit from MCC projects.
On the basis of the costs and benefits projected over a 20-year
period, MCC calculated three summaries of the compact's impact:
its ERR, effect on per capita income, and effect on GDP. MCC
projected an overall compact ERR of 24.7 percent over 20 years.15
In projecting the compact's impact on Vanuatu's per capita income,
MCC used a baseline per capita income of $1,326 for 2005.
MCC also prepared a sensitivity analysis to assess how a range of
possible outcomes would affect compact results. MCC's tests
included a 1-year delay of the start date for accrued benefits; a
20 percent increase of all costs; a 20 percent decrease of all
benefits; and a "stress test," with a 20 percent increase of all
costs and a 20 percent decrease of all benefits. MCC calculated a
best-case compact ERR of 30.2 percent and a worst-case compact ERR
of 13.9 percent.
12Benefits other than those included in its economic analysis may accrue
to Vanuatu as a result of the compact. For example, increased economic
activity in tourism may benefit other sectors of the economy and that the
welfare of Vanuatu's citizens may improve with increased access to health
care and educational opportunities.
13MCC's economic model assumes that construction costs are incurred in the
first year after compact signing and counts 16 percent of total
construction spending as a benefit to the local economy for that year.
14MCC's cost estimate for construction and maintenance of the projects on
Santo and Efate was based on an estimate prepared for the Vanuatu
government by a contractor in 2004. We asked MCC for a copy of the 2004
estimate; however, according to MCC officials, MCC did not have a copy and
the government was not willing to provide the estimate for our review.
15In its final April 2006 economic analysis, MCC adjusted this calculation
downward slightly to 24.2 percent.
MCC's Data Do Not Support Its Portrayal of Compact Benefits
MCC's public portrayal of the Vanuatu compact's projected effects
on per capita income and on GDP suggest greater impact than its
analysis supports. In addition, MCC's portrayal of the compact's
projected impact on poverty does not identify the proportion of
benefits that will accrue to the rural poor.
o Impact on per capita income. In the compact and the
congressional notification, MCC states that the transportation
infrastructure project is expected to increase "average income per
capita (in real terms) by approximately $200, or 15 percent of
current income per capita, by 2010." MCC's investment memo states
that the compact will cause per capita income to increase by $488,
or 37 percent, by 2015. These statements suggest that as a result
of the program, average incomes in Vanuatu will be 15 percent
higher in 2010 and 37 percent higher in 2015 than they would be
without the compact. However, MCC's underlying data show that
these percentages represent the sum of increases from per capita
income in 2005 that MCC projects for each year. For example,
according to MCC's data, Vanuatu's per capita income in a given
year between 2006 and 2010 will range from about 2 percent to
almost 4 percent higher than in 2005; in its statements, MCC sums
these percentages as 15 percent without stating that this
percentage is a cumulative increase from 2005. Our analysis of
MCC's data shows that actual gains in per capita income, relative
to income in 2005, would be $51, or 3.9 percent, in 2010 and $61,
or 4.6 percent, in 2015 (see fig. 5).
Figure 5: Vanuatu Compact's Projected Impact on Real Per Capita Income
According to MCC Statement and MCC Data Relative to 2005 Per Capita Income
Note: MCC's statement: "Increasing average income per capita (in real
terms) by approximately $200 or 15 percent of current income per capita by
2010" and by $488--37 percent--by 2015.
Figure 6 further illustrates MCC's methodology in projecting the compact's
impact on per capita income levels for 2010 and 2015.
Figure 6: MCC Methodology for Projecting Vanuatu Compact's Impact on Real
Per Capita Income
o Impact on GDP. Like its portrayal of the projected impact on per
capita income, MCC's portrayal of the projected impact on GDP is
not supported by the underlying data. In the compact and the 2006
congressional notification, MCC states that the compact will have
a transformational effect on Vanuatu's economy, causing GDP to
"increase by an additional 3 percent a year." Given the GDP growth
rate of about 3 percent that MCC expects in Vanuatu without the
compact, MCC's statement of a transformational effect suggests
that the GDP growth rate will rise to about 6 percent. However,
MCC's underlying data show that although Vanuatu's GDP growth rate
will rise to about 6 percent in 2007, in subsequent years the GDP
growth rate will revert to roughly the rate MCC assumes would
occur without the compact, approximately 3 percent (see fig. 7).
Although MCC's data show that the compact will result in a higher
level (i.e., dollar value) of GDP, the data do not show a
transformational increase to the GDP growth rate.
Figure 7: Vanuatu GDP Growth with and without MCC Compact
Notes:
According to MCC, "GDP is expected to increase by an additional 3
percent a year as a result of the MCA program."
According to MCC data, the compact will have a small impact on GDP
growth rate in later years. In 2010 to 2015, the GDP growth rate
resulting from the compact will be 3.1 percent, compared with 3
percent without the compact.
o Impact on poverty. MCC's portrayal of the compact's projected
impact on poverty does not identify the proportion of the
financial benefits that will accrue to the rural poor. In the
compact and the congressional notification, MCC states that the
program is expected to benefit "approximately 65,000 poor, rural
inhabitants living nearby and using the roads to access markets
and social services." In its underlying documentation, MCC expects
57 percent of the monetary benefits to accrue to other
beneficiaries, including expatriate tourism services providers,
transport providers, government, and local businesses; 43 percent
is expected to go to the local population, which MCC defines as
"local producers, local consumers and inhabitants of remote
communities" (see fig. 8). However, MCC does not establish the
proportion of local-population benefits that will go to the 65,000
poor, rural beneficiaries.16
Figure 8: MCC Analysis of Distribution of Vanuatu Compact Benefits
Note: MCC defines "local population" as comprising local
producers, local consumers, and inhabitants of rural communities.
Several Risks May Lead to Reduced Project Benefits
Our analysis shows that risks related to construction costs,
timing of benefits, project maintenance, induced benefits, and
efficiency gains may lessen the Vanuatu compact's projected impact
on poverty reduction and economic growth. Accounting for these
risks could reduce the overall compact ERR.
16Our review of MCC's analyses also identified some calculation errors in
MCC's determination of the compact's impact on per capita income and
estimation of the number of compact beneficiaries. In addition, we
identified questionable assumptions regarding the beneficiary population.
For example, MCC counted all residents of the catchment area as poor and
assumed that residents of off-shore islets and villages near paved
portions of the Efate Ring Road not improved by MCC would benefit from the
compact. Correcting these errors and fully discounting these assumptions
would reduce the beneficiary count on Efate and Santo by 32 percent--from
26,553, as stated by MCC, to 18,070--indicating that MCC may have
overestimated the compact's beneficiaries.
o Construction costs. Although MCC considered the risk of
construction cost increases, the contingencies used in its
calculations may not be sufficient to cover actual construction
costs. Cost estimate documentation for 5 of MCC's 11 construction
projects shows that these estimates include design contingencies
of 20 percent. However, cost overruns of more than 20 percent
occur in many transportation projects,17 and as MCC's analysis
notes, the risk of excessive cost overruns is significant in a
small country such as Vanuatu.18 Any construction cost overrun
must be made up within the Vanuatu compact budget by reducing the
scope, and therefore the benefits, of the compact projects;19
reduced project benefits would in turn reduce the compact's ERR
and effects on per capita income and GDP.
o Timing of benefits. Although MCC's analysis assumes compact
benefits from 2008 or 2009--shortly after the end of project
construction--we found that benefits are likely to accrue more
slowly. Our document review and discussions with tourism services
providers and agricultural and timber producers suggest that these
businesses will likely react gradually to any increased market
opportunities resulting from MCC's projects, in part because of
constraints to expanding economic activity.20 In addition, MCC
assumes that all construction spending will occur in the first
year, instead of phasing the benefits from this spending over the
multiyear construction schedule.
17A study of more than 250 transportation projects in Europe, North
America, and elsewhere found that costs for all projects were 28 percent
higher, on average, than forecasted at the time of decision to build,
while road projects averaged escalations of 20.4 percent. See Bent
Flyvbjerg, Mette Skamris Holm, and Soren Buhl, "Underestimating Costs in
Public Works Projects: Error or Lie?," Journal of the American Planning
Association, Vol. 68, No. 3 (2002), cited in GAO, Highway and Transit
Investments: Options for Improving Information on Projects' Benefits and
Costs and Increasing Accountability for Results, [18]GAO-05-172
(Washington, D.C.: January 24, 2005).
18MCC cites the "design-construct" contract proposed for the MCA program,
which will include design and construction of all the projects as one
package, as key to mitigating this risk. However, MCC's analysis also
recognized that nonconstruction-related issues (such as access to parts of
the project site) have the potential to delay the contractor and increase
costs and that such issues can be significant for major road upgrade
projects where the competing interests of the contractor, adjacent
villages, and the general public must be balanced. MCC's analysis states
that, to help manage the risk of project-related disputes and delays, MCC
plans to have experienced consultants work with local PWD staff who have
an understanding of the social and cultural issues.
19According to the compact, the government of Vanuatu must pay any
environmental mitigation and remediation costs in excess of the budget.
20Benefits from construction activities may also be reduced by a delayed
procurement. MCA-Vanuatu officials initially told us they anticipated
issuing an invitation for bid to contractors by the end of February 2007.
As of May 2007, the invitation had not yet been issued. MCC currently
expects construction to begin in 2008, further reducing the likelihood of
benefits starting in 2007 as MCC anticipated in its analyses.
o Project maintenance. Uncertainty about the maintenance of
completed transportation infrastructure projects after 2011 may
affect the compact's projected benefits. Vanuatu's record of road
maintenance is poor. According to World Bank and Asian Development
Bank officials, continuing donor involvement is needed to ensure
the maintenance and sustainability of completed projects. However,
although MCC has budgeted $6.2 million for institutional
strengthening of the Vanuatu PWD, MCC has no means of ensuring the
maintenance of completed projects after the compact expires in
2011; the Millennium Challenge Act limits compacts to 5 years.
Poor maintenance performance will reduce the benefits projected in
the MCC compact.
o Induced benefits. The compact's induced benefits depend on the
response of Vanuatu tourism providers and agricultural producers.
However, constraints affecting these economic sectors may prevent
the sector from expanding as MCC projects. Limited response to the
compact by tourism providers and agricultural producers would have
a significant impact on compact benefits.
o Efficiency gains. MCC counts efficiency gains--such as time
saved because of better roads--as compact benefits. However,
although efficiency gains could improve social welfare, they may
not lead to changes in per capita income or GDP or be directly
measurable as net additions to the economy.
Accounting for these risks could reduce the overall compact ERR
from 24.2 percent, as projected by MCC, to between 5.5 percent and
16.5 percent (see table 1).
Table 1: Summary of Compact ERR under Alternative Scenarios of
Accounting for Risks to Benefits
Compact ERR
MCC's anticipated effect 24.2 percent
GAO analysisa
(1) Costs are phased over 3 years and benefits are phased 16.5 percent
over 5 years
Costs are phased over 3 years and benefits are phased over 5
years, and
(2) induced benefits are not realizedb 5.5 percent
(3) efficiency gains are not monetizedc 11.8 percent
(4) large-scale maintenance is not undertakend 13.8 percent
Source: GAO analysis of MCC data.
aIn our analysis, benefits start in 2010 and are phased in equal
increments over 5 years, from 2010 to 2014, with phasing completed
by year 5. Costs are phased over 3 years to reflect projected
timing of construction.
bIn addition to phasing benefits and costs, we eliminated induced
effects of the project on agriculture, tourism, fisheries, and the
development of subdivided beachfront land.
cIn addition to phasing benefits and costs, we eliminated road
user cost savings and savings from reduction of wasted surface
trips, lost trips, longer diversions, and enforced longer trips
from road closures.
dIn addition to phasing benefits and costs, we assumed that total
benefits will increase, peak, and decrease such that their value
in 2027 will equal their original value in 2012. The large capital
outlays for road rehabilitation in 2017 and 2026 in Santo and
Efate have been eliminated.
Conclusions
MCC's public portrayal of the Vanuatu compact's projected
benefits--particularly the effect on per capita income--suggests a
greater impact than MCC's underlying data and analysis support and
can be understood only by reviewing source documents and
spreadsheets that are not publicly available. As a result, MCC's
statements may foster unrealistic expectations of the compact's
impact in Vanuatu. For example, by suggesting that per capita
incomes will increase so quickly, MCC suggests that its compact
will produce sustainable growth that other donors to Vanuatu have
not been able to achieve. The gaps between MCC's statements about,
and underlying analysis of, the Vanuatu compact also raise
questions about other MCC compacts' projections of a
transformational impact on country economies or economic sectors.
Without accurate portrayals of its compacts' projected benefits,
the extent to which MCC's compacts are likely to further its goals
of poverty reduction and economic growth cannot be accurately
evaluated. In addition, the economic analysis underlying MCC's
statements does not reflect the time required to improve Vanuatu's
transportation infrastructure and for the economy to respond and
does not fully account for other risks that could substantially
reduce compact benefits.
Recommendations
In our report, we recommend that the CEO of MCC take the following
actions:
o revise the public reporting of the Vanuatu compact's projected
impact to clearly represent the underlying data and analysis;
o assess whether similar statements in other compacts accurately
reflect the underlying data and analysis; and
o improve its economic analysis by phasing the costs and benefits
in compact ERR calculations and by more fully accounting for risks
such as those related to continuing maintenance, induced benefits,
and monetized efficiency gains as part of sensitivity analysis.
In comments on a draft of our report, MCC did not directly
acknowledge our recommendations. MCC acknowledged that its use of
projected cumulative compact impact on income and growth was
misleading but asserted that it had no intention to mislead and
that its portrayal of projected compact benefits was factually
correct. MCC questioned our finding that its underlying data and
analysis do not support its portrayal of compact benefits and our
characterization of the program's risks. (See app. VI of our
report for MCC comments and our response.21)
Mr. Chairman, this completes my prepared statement. I would be
happy to respond to any questions you or other Members of the
Subcommittee may have at this time.
GAO Contact and Staff Acknowledgments
For further information about this testimony, please contact me at
(202) 512-3149 or [email protected]. Contact points for our
Offices of Congressional Relations and Public Affairs may be found
on the last page of this statement. In addition to the person
named above, Emil Friberg, Jr. (Assistant Director), Gergana
Danailova-Trainor, Reid Lowe, Angie Nichols-Friedman, Michael
Simon, and Seyda Wentworth made key
contributions to this statement. Also, David Dornisch, Etana
Finkler, Ernie Jackson, and Tom McCool provided technical
assistance.
21 [19]GAO-07-909 .
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in
meeting its constitutional responsibilities and to help improve
the performance and accountability of the federal government for
the American people. GAO examines the use of public funds;
evaluates federal programs and policies; and provides analyses,
recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at
no cost is through GAO's Web site ( [12]www.gao.gov ). Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of
newly posted products every afternoon, go to [13]www.gao.gov and
select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies
are $2 each. A check or money order should be made out to the
Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are
discounted 25 percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
(202) 512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: [14]www.gao.gov/fraudnet/fraudnet.htm E-mail:
[15][email protected] Automated answering system: (800) 424-5454 or
(202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [16][email protected] (202)
512-4400 U.S. Government Accountability Office, 441 G Street NW,
Room 7125 Washington, D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [17][email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW,
Room 7149 Washington, D.C. 20548
(320524)
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
[20]www.gao.gov/cgi-bin/getrpt?GAO-07-1122T .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact David Gootnick, (202) 512-3149,
[email protected].
Highlights of [21]GAO-07-1122T , a testimony to the Subcommittee on Asia,
the Pacific, and the Global Environment, House of Representatives
July 26, 2007
MILLENNIUM CHALLENGE CORPORATION
Projected Impact of Vanuatu Compact Is Overstated
In January 2004, Congress established the Millennium Challenge Corporation
(MCC) for foreign assistance. Congress has appropriated almost $6 billion
to MCC. As of March 2007, MCC had signed almost $3 billion in compacts
with 11 countries, including a 5-year, $65.7 million compact with Vanuatu.
MCC states that the Vanuatu compact will have a transformational effect on
the country's economy, increasing per capita income and GDP and benefiting
65,000 poor, rural people. This testimony summarizes a July 2007 report (
[22]GAO-07-909 ) examining (1) MCC's methods of projecting economic
benefits, (2) MCC's portrayal and analysis of the projected benefits, and
(3) risks that may affect the compact's impact. To address these
objectives, GAO reviewed MCC's analyses and met with officials and
business owners in Vanuatu as well as with other donors.
In its July 2007 report, GAO recommended that the Chief Executive Officer
of MCC revise the public reporting of the Vanuatu compact's projected
impact; assess whether similar reporting in other compacts accurately
reflects underlying analyses; and improve its economic analyses by more
fully accounting for risks to project benefits. MCC did not directly
address GAO's recommendations but commented that it had not intended to
make misleading statements and that its portrayal of projected results was
factual and consistent with underlying data.
MCC projects that the Vanuatu compact's transportation infrastructure
projects will provide direct benefits such as reduced transportation costs
and induced benefits from growth in tourism and agriculture. MCC estimated
the costs and benefits over 20 years, with benefits beginning in full in
2008 or 2009 and growing each year, and it counted poor, rural
beneficiaries by defining the area where benefits were likely to accrue.
Using projected benefits and costs, MCC calculated the compact's economic
rate of return (ERR) and its effects on Vanuatu's gross domestic product
(GDP) and per capita income.
MCC's portrayal of the projected impact does not reflect its underlying
data. MCC states that per capita income will increase by approximately
$200, or 15 percent, by 2010 and by $488, or 37 percent, by 2015. However,
MCC's underlying data show that these figures represent the sum of
individual years' gains in per capita income relative to 2005 and that
actual gains will be $51, or 3.9 percent, in 2010 and $61, or 4.6 percent,
in 2015. MCC also states that GDP will increase by an additional 3 percent
a year, but its data show that after GDP growth of 6 percent in 2007, the
economy's growth will continue at about 3 percent, as it would without the
compact. MCC states that the compact will benefit approximately 65,000
poor, rural inhabitants, but this statement does not identify the
financial benefits that accrue to the rural poor or reflect its own
analysis that 57 percent of benefits go to others.
We identified five key risks that could affect the compact's projected
impacts. (1) Cost estimate contingencies may not be sufficient to cover
project overruns. (2) Compact benefits will likely accrue more slowly than
MCC projected. (3) Benefit estimates assume continued maintenance, but
MCC's ability to ensure maintenance will end in 2011, and Vanuatu's
maintenance record is poor. (4) Induced benefits depend on businesses' and
residents' response to new opportunities. (5) Efficiency gains, such as
time saved in transit, may not increase per capita income. Our analysis of
these areas of risk illustrates the extent that MCC's projections are
dependent on assumptions of immediate realization of benefits, long-term
maintenance, realization of induced benefits, and benefits from efficiency
gains.
Vanuatu Compact's Impact on Per Capita Income According to MCC Statement
vs. MCC Data
References
Visible links
10. http://www.gao.gov/cgi-bin/getrpt?GAO-07-909
11. http://www.gao.gov/cgi-bin/getrpt?GAO-06-805
12. http://www.gao.gov/
13. http://www.gao.gov/
14. http://www.gao.gov/fraudnet/fraudnet.htm
15. mailto:[email protected]
16. mailto:[email protected]
17. mailto:[email protected]
18. http://www.gao.gov/cgi-bin/getrpt?GAO-05-172
19. http://www.gao.gov/cgi-bin/getrpt?GAO-07-909
20. http://www.gao.gov/cgi-bin/getrpt?GAO-07-1122T
21. http://www.gao.gov/cgi-bin/getrpt?GAO-07-1122T
22. http://www.gao.gov/cgi-bin/getrpt?GAO-07-909
*** End of document. ***