    International Financial Crises: Challenges Remain in IMF's
    Ability to Anticipate, Prevent, and Resolve Financial Crises
    (16-JUN-03, GAO-03-734).

    Building on reform initiatives instituted after the Mexican
    financial crisis, the IMF implemented new initiatives in the
    mid-1990s to better anticipate, prevent, and resolve sovereign
    financial crises. GAO was asked to assess (1) the IMF's framework
    for anticipating financial crises, (2) the status of key IMF
    reform initiatives to prevent financial crises, and (3) new IMF
    proposals to resolve future financial crises.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-03-734
        ACCNO:   A07165
      TITLE:     International Financial Crises: Challenges Remain in
    IMF's Ability to Anticipate, Prevent, and Resolve Financial
    Crises
         DATE:   06/16/2003
      SUBJECT:   Financial management
    	     International economic relations
    	     Risk management
    	     Strategic planning
    	     International Monetary Fund

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-03-734

    a

    GAO United States General Accounting Office

    Report to the Chairman, Committee on Financial Services, and to the Vice
    Chairman, Joint Economic Committee, House of Representatives

    June 2003 INTERNATIONAL FINANCIAL CRISES Challenges Remain in IMF*s
    Ability to Anticipate, Prevent, and Resolve Financial Crises

    GAO- 03- 734

    While the Fund*s new vulnerability framework is more comprehensive than
    its previous efforts, it is too early to assess whether it will improve
    Fund efforts to anticipate crises. The new framework uses the Fund*s major
    forecasting tools, the World Economic Outlook (WEO) and the Early Warning
    System (EWS), which have not performed well in anticipating prior crises.
    The forecasting of crises has been historically difficult for all
    forecasters.

    The Fund, with the World Bank, has made progress in implementing
    initiatives to prevent crises, but several challenges remain. To obtain
    better information about country financial sector weaknesses, the Fund and
    Bank introduced the Financial Sector Assessment Program (FSAP) to report
    on member countries* financial sectors and the Reports on the Observance
    of Standards and Codes (ROSC) to assess member countries* adherence to 12
    standards. Assessments have not been completed in some major emerging
    market countries primarily because participation is voluntary, and use of
    this information has been mixed. For example, some private sector market
    participants have found the reports untimely, outdated, and dense.
    Participation Gaps by 33 Major Emerging Market Countries in Key
    Assessments, 1999- 2003

    Number of assessments The Fund is considering two approaches to
    restructuring unsustainable sovereign debt; however, there are significant
    challenges to implementing them. One approach involves creating an
    international legal framework that would allow a specified majority of a
    country*s external creditors to restructure most private sector loans.
    Under the second approach, the Fund is encouraging members to include
    renegotiation clauses in individual bonds. Many private sector
    representatives wish to maintain the existing process in which the Fund
    assists resolution by providing loans to some eligible members. In
    response to concerns that its resources may have unintended negative
    impacts during a crisis, the Fund has clarified and strengthened its
    criteria for lending to members experiencing crises. Building on reform
    initiatives instituted after the Mexican

    financial crisis, the IMF implemented new initiatives in the mid- 1990s to
    better anticipate, prevent, and resolve sovereign financial crises. GAO
    was asked to assess (1) the IMF*s framework for anticipating financial
    crises, (2) the status of key IMF reform initiatives to prevent financial
    crises, and (3) new IMF proposals to resolve

    future financial crises. GAO recommends that the Secretary of the Treasury
    instruct the U. S. Executive Director of the

    Fund to work with other Executive Board members to encourage the Fund to
    improve the timeliness of

    FSAP and ROSC reports;  expand the coverage,

    frequency, and publication of updates of participants* implementation of
    FSAP and ROSC recommendations;  improve the FSAP and ROSC reports*
    readability; and

     increase participation in the FSAP and all standards of the ROSC and
    consider making participation mandatory.

    Treasury, IMF, and the World Bank generally agreed with the report*s
    recommendations. The IMF stated

    that we gave WEO and EWS forecasts greater importance than is warranted in
    anticipating crises. However, we focused on the only mature and
    quantifiable elements of the vulnerability framework.

    www. gao. gov/ cgi- bin/ getrpt? GAO- 03- 734. To view the full report,
    including the scope and methodology, click on the link above. For more
    information, contact Joseph A. Christoff at (202) 512- 8979 or christoffj@
    gao. gov. Highlights of GAO- 03- 734, a report to

    Chairman, House Committee on Financial Services and Vice Chairman, House
    Joint Economic Committee June 2003

    INTERNATIONAL FINANCIAL CRISES

    Challenges Remain in IMF's Ability to Anticipate, Prevent, and Resolve
    Financial Crises

    Page i GAO- 03- 734 International Financial Crises

    Contents Letter 1

    Background 2 Results in Brief 4 Too Early to Determine If Fund*s New
    Process for Anticipating

    Financial Crises Improves on Past Efforts 7 Some Progress in Implementing
    Crisis Prevention Initiatives, but

    Challenges Remain 15 The Fund*s Efforts to Better Resolve Future Financial
    Crises Face

    Significant Challenges 24 Conclusion 30 Recommendations for Executive
    Action 31 Agency Comments and Our Evaluation 31

    Appendixes

    Appendix I: Objectives, Scope, and Methodology 34

    Appendix II: Private Sector Structured Interview Participants 37

    Appendix III: Assessment of IMF Forecasting 38 GAO Analysis Used Standard
    Econometric Techniques for Forecast

    Evaluation 38

    Appendix IV: Eighty- seven Emerging Market Countries 50

    Appendix V: Standards, Codes, and Principles Assessed under IMF and World
    Bank Reports on the Observance of Standards and Codes 53

    Appendix VI: Update on the International Monetary Fund*s Safeguards
    Assessments 56 The Fund Introduces Safeguards Assessments for Member

    Countries That Currently Borrow 56 Implementation of the Fund*s Assessment
    Recommendations 57

    Appendix VII: Fund/ World Bank FSAP and ROSC Process 59

    Appendix VIII: Country Participation in and Publication of FSAPs and ROSCs
    60

    Appendix IX: Fund Resources Provided under the Supplemental Reserve
    Facility 64

    Appendix X: Comments from the Department of the Treasury 65

    Appendix XI: Comments from the International Monetary Fund 67 GAO Comments
    70

    Contents

    Page ii GAO- 03- 734 International Financial Crises Appendix XII: Comments
    from The World Bank 72

    GAO Comments 74

    Appendix XIII: GAO Contacts and Staff Acknowledgments 75 GAO Contacts 75
    Staff Acknowledgments 75

    Tables Table 1: Anticipated and Actual GDP Growth Rate of 14 Financial
    Crises, 1990* 2001 12

    Table 2: Forecast Quality for 87 Emerging Market Countries, 1990* 2001 43
    Table 3: Forecast Quality for G- 7 Countries, 1990* 2001 45 Table 4:
    Comparison of Year- Ahead WEO Forecasts for Program

    and Nonprogram Countries, 1990* 2001 46 Table 5: Comparison of Program and
    WEO Year- Ahead Forecasts,

    for 52 Countries in the Initial Years on Program, 1990* 2001 47 Table 6:
    IMF Supplemental Reserve Facility Loans, 1997* 2002 64

    Figures Figure 1: IMF Initiatives to Anticipate, Prevent, and Resolve
    Financial Crises 4

    Figure 2: IMF Vulnerability Assessment Framework 8 Figure 3: Participation
    of 33 Major Emerging Markets in Key ROSC Assessments, 1999* 2003 20

    Figure 4: FSAP and ROSC Participation and Publication by Major Emerging
    Market Countries 61 Figure 5: FSAP and ROSC Participation and Publication
    by Other

    Countries 62

    Abbreviations

    BCP Basel Core Principles BMA Bond Market Association CAC Collective
    Action Clauses CCFF Compensatory Contingency Finance Facility CPI consumer
    price index CPSS Committee on Payments and Settlements Systems DCSD
    Developing Country Studies Division EFF Extended Fund Facility EMCA
    Emerging Market Creditors Association

    Contents

    Page iii GAO- 03- 734 International Financial Crises

    EMTA Emerging Market Traders Association EWS Early Warning System FATF
    Financial Action Task Force FSA Financial Sector Assessment FSAP Financial
    Sector Assessment Program FSSA Financial System Stability Assessment GDDS
    General Data Dissemination Standard GDP gross domestic product IAIS
    International Association of Insurance Supervisors IASB International
    Accounting Standards Board IFAC International Federation of Accountants

    IMF International Monetary Fund IOSCO International Organization of
    Securities Commissions IPMA International Primary Market Association ISMA
    International Securities Market Association KLR Kaminsky, Lizondo, and
    Reinhart MSE mean square error OECD Organization for Economic Cooperation
    and Development RMSE root mean square error ROSC Reports on the Observance
    of Standards and Codes RSS Residual Sum of Squares SBA Stand- By
    Arrangements SDDS Special Data Dissemination Standard SDRM Sovereign Debt
    Restructuring Mechanism SEC U. S. Securities and Exchange Commission SIA
    Securities Industry Association SRF Supplemental Reserve Facility STF
    Systemic Transformation Facility UNCITRAL United Nations Commission on
    International Trade Law WEO World Economic Outlook

    This is a work of the U. S. Government and is not subject to copyright
    protection in the United States. It may be reproduced and distributed in
    its entirety without further permission from GAO. It may contain
    copyrighted graphics, images or other materials. Permission from the
    copyright holder may be necessary should you wish to reproduce copyrighted
    materials separately from GAO*s product.

    Page 1 GAO- 03- 734 International Financial Crises United States General
    Accounting Office

    Washington, D. C. 20548 Page 1 GAO- 03- 734 International Financial Crises

    A

    June 16, 2003 Let t er

    The Honorable Michael G. Oxley Chairman Committee on Financial Services
    House of Representatives

    The Honorable Jim Saxton Vice Chairman Joint Economic Committee House of
    Representatives

    In May 2000, in the aftermath of the Asian financial crisis, the newly
    appointed Managing Director of the International Monetary Fund (IMF or the
    Fund) announced his plans to strengthen the Fund*s reform initiatives to
    enable the Fund to more effectively safeguard the stability of the
    international financial system. Building on reform initiatives instituted
    in the mid- 1990s, after the Mexican financial crisis of 1994* 95, these
    new initiatives were designed to better anticipate, prevent, and resolve
    sovereign financial crises. A sovereign financial crisis can occur when a

    country is unable or unwilling to honor its debt obligations and investors
    lose confidence in that country*s financial markets. Recent crises have
    occurred primarily in *emerging market* countries, larger and more

    economically advanced developing countries such as Argentina, Brazil, and
    Turkey. According to World Bank (the Bank) estimates, the financial costs
    to countries that experienced crises in the 1980s and 1990s exceeded $1
    trillion* greater than the total amount of all donors* assistance to

    developing countries in the 1980s and 1990s. This letter responds to your
    request that we examine the Fund*s efforts to better safeguard the
    stability of the international financial system. In this report, we assess
    (1) IMF*s framework for anticipating financial crises, (2) the status of
    key IMF initiatives to prevent financial crises, and (3) new IMF proposals
    to resolve future financial crises. In addition, we analyzed the

    quality of IMF*s forecasts produced by the World Economic Outlook (WEO),
    the Fund*s primary forecasting tool (see appendix III).

    As part of our assessment, we reviewed documents from the U. S.
    government, international organizations, and private firms, including
    testimonies, reports, and relevant laws. We used statistical models that
    we developed to assess the Fund*s WEO forecasts and program projections
    for selected emerging market countries. We interviewed key officials at
    the

    Page 2 GAO- 03- 734 International Financial Crises

    Fund, World Bank, Department of the Treasury (Treasury), and emerging
    market bond and equity funds. We also interviewed some representatives of
    private sector firms that specialize in international finance, economics,
    and law. Appendix I provides a more detailed description of our
    objectives, scope, and methodology. Appendix II provides a list of the
    private sector firms we interviewed.

    Background In 1994* 95, Mexico faced a severe financial crisis when a
    shift in market sentiment led to sudden large capital outflows. Investors
    also temporarily removed their funds from other emerging market countries,
    an effect known as contagion. In response to the crisis, Mexico quickly
    adopted a strong and ultimately successful program of adjustment and
    reform. To support the program, the Fund approved a loan of $17.8 billion
    to Mexico*

    one of the largest loan commitments it had ever made to a country. One of
    the major reasons cited for the crisis was the lack of timely, reliable,
    and publicly available economic, financial, and sociodemographic data for
    Mexico. Beginning in 1996, to correct this weakness, the Fund created data
    standards to guide countries in disseminating better data to the public.

    However, as we reported in 1997, the Fund needed to address a number of
    other financial, economic, and political challenges, in addition to data
    limitations, to better anticipate, prevent, and resolve financial crises.
    1

    Before the Fund could fully address these challenges, the Asian financial
    crisis of 1997* 98 occurred. After the Asian financial crisis, the Fund
    assessed the effects of its responses to the crisis and reassessed its
    role in

    safeguarding the stability of the international financial system,
    including rethinking its core mission, operations, and lending activities.
    The Fund also recognized that it needed to improve its ability to
    anticipate financial crises; monitor countries* activities; and increase
    public awareness, particularly that of the investment community.
    Recognizing its inability to anticipate past crises, the Fund instituted a
    quarterly vulnerability assessment framework in 2001 to identify countries
    that may be susceptible to crisis.

    1 U. S. General Accounting Office, International Financial Crises: Efforts
    to Anticipate, Avoid, and Resolve Sovereign Crises, GAO/ GGD/ NSIAD- 97-
    168 (Washington, D. C.: July 7, 1997).

    Page 3 GAO- 03- 734 International Financial Crises

    To improve its ability to prevent future crises, the Fund and the World
    Bank in 1999 began performing joint assessments of member countries*
    financial sectors to help identify and monitor existing and potential
    weaknesses. In addition, the Fund and the World Bank began to work with
    countries to promote adherence to voluntary standards to reassure the
    international community that the countries* policies and practices conform
    to standards and codes of good practice. These include standards to
    improve transparency in government economic data; fiscal, monetary, and
    financial policies; and guidelines on strengthening the financial and
    corporate sectors. The Fund acknowledges that it would be almost
    impossible to anticipate or prevent all crises. According to the Fund,
    past efforts to resolve financial

    crises during the 1990s were lengthy and very costly to debtor countries.
    The Fund is encouraging the adoption of agreements that would allow a
    quicker, more orderly, and predictable restructuring of countries* debts.
    The Fund*s ultimate goal is to maintain investor confidence and stability
    in the international financial system. Figure 1 shows the Fund*s key
    initiatives for better anticipating, preventing, and resolving financial
    crises. Treasury through the U. S. Executive Director to the Fund has the
    lead responsibility for monitoring the IMF*s progress in addressing these
    issues.

    Page 4 GAO- 03- 734 International Financial Crises

    Figure 1: IMF Initiatives to Anticipate, Prevent, and Resolve Financial
    Crises

    Results in Brief While it is too early to assess the effectiveness of the
    Fund*s new vulnerability framework for anticipating crises, this framework
    builds on

    key elements that have been unable to anticipate past financial crises.
    The new framework is more comprehensive than the Fund*s previous efforts,
    bringing together country- specific knowledge and financial expertise
    within the Fund to better identify weaknesses in emerging market economies
    that could lead to a crisis. However, the new framework uses the Fund*s
    major forecasting tools, the WEO and the Early Warning System (EWS), which
    have not performed well in anticipating prior crises. The

    WEO has not successfully anticipated past financial crises, and the Fund*s
    EWS models have had a high false alarm rate, having predicted many crises
    that did not occur. The forecasting of crises has been historically
    difficult for all forecasters due to complex underlying factors, including
    concerns about the reliability of important macroeconomic data on emerging
    market countries.

    Page 5 GAO- 03- 734 International Financial Crises

    The Fund, in collaboration with the World Bank, has made progress in
    implementing initiatives to prevent financial crises, but several
    challenges remain. In the late 1990s, the Fund and the World Bank adopted
    two key initiatives designed to assist four parties: Fund staff, World
    Bank staff, member country governments, and private sector participants.
    First, financial sector assessments were introduced to provide reports on
    aspects

    of member countries* financial sectors such as banking systems and crisis
    management capacity. Second, the standards initiative was adopted to
    assess member countries* adherence to 12 standards in areas such as
    banking supervision and economic data dissemination. Use of the
    information provided by these two initiatives by various parties has been
    mixed, and several significant challenges remain. Fund staff frequently
    incorporate information from the assessments into their policy advice to
    member countries. However, assessments have not been completed in some
    important emerging market countries, such as China and Thailand, primarily
    because participation is voluntary. World Bank staff*s use of the
    assessments to inform country development assistance programs is also
    affected by the lack of participation by member countries and by borrower
    countries* competing development needs. Member country governments
    sometimes use the assessments in prioritizing their reform agendas but
    often find that the reforms are too difficult to implement. Some private
    sector participants find the published reports outdated, untimely, and too
    dense to be useful in making investment decisions. The Fund and the World

    Bank acknowledge that the assessments cannot prevent all crises because
    reforms require many years to be fully implemented, and crises can be
    caused by factors outside the scope of the reforms.

    The Fund is considering two approaches to achieve a more orderly and
    predictable restructuring of unsustainable debt between a country and its
    creditors; however, there are significant challenges to implementing both
    approaches. Fund officials assert that the current process for
    renegotiating the terms of member countries* loans with external private
    sector creditors is lengthy and costly. The Sovereign Debt Restructuring
    Mechanism (SDRM), is a proposed international legal framework that would
    allow a majority of a country*s external creditors to quickly restructure
    most private sector loans. The Fund is also encouraging members to include
    Collective Action Clauses (CACs) in individual bonds, which would allow a
    majority of bondholders to renegotiate the bond*s terms. Although some
    elements of both approaches are acceptable to the private sector and to
    governments, a number of political, legal, and technical challenges stand
    in the way of implementing the SDRM; it seems unlikely that these issues
    will be resolved in the immediate future. While private sector officials
    expect

    Page 6 GAO- 03- 734 International Financial Crises

    that many restructurings need only involve the private sector and the
    debtor country, under some circumstances, voluntary debt restructurings
    will not adequately resolve all financial crises. In those cases, they
    said the Fund should provide loans to eligible countries to help fill
    their external financing gaps. However, some financial experts and
    government representatives have raised concerns that such loans have the
    potential to

    increase the probability of future crises. In response, the Fund has
    clarified and strengthened its policy of lending in crisis situations.

    The Fund*s crisis prevention initiatives are hindered by several factors
    that limit their effectiveness. To strengthen the effectiveness of these
    initiatives, we are recommending that the Secretary of the Treasury
    instruct the U. S. Executive Director of the Fund to work with other
    Executive Board members to encourage the Fund to improve the readability,
    timeliness, coverage, and frequency of updates of assessment reports.
    Additionally, the

    Fund should pursue strategies for increasing participation in the
    assessment process including the possibility of making participation
    mandatory for all members of the IMF. In responding to our draft report,
    Treasury, IMF, and the World Bank

    generally agreed with the report*s recommendations. However, the IMF
    stated that we mischaracterized the role of WEO forecasts and EWS models
    in IMF crisis anticipation efforts by saying that they have a greater

    importance than is warranted. We disagree with this depiction. Our
    assessment examined all six components of the IMF*s vulnerability
    assessment framework, including the WEO and EWS. As the only mature and
    quantifiable elements of the framework, our analysis focused more heavily
    on the track records of the WEO and EWS. The World Bank expressed concern
    with the report*s suggestion that consideration be given in making
    participation in the FSAP and ROSC assessments mandatory. While we are not
    suggesting that the assessments should be made

    mandatory, the voluntary nature of these assessments has posed an obstacle
    to full participation by important emerging market countries.

    Page 7 GAO- 03- 734 International Financial Crises

    Too Early to Determine If Fund*s New Process for Anticipating Financial
    Crises Improves on Past Efforts

    In May 2001, the IMF implemented a new vulnerability assessment framework
    for emerging market countries to strengthen the Fund*s ability to
    anticipate financial crises. This framework brings together
    countryspecific knowledge and financial expertise within the Fund to
    better identify weaknesses in emerging market economies that could lead to
    a crisis. Although the new vulnerability assessment framework is more
    comprehensive than the Fund*s previous efforts, it is new and still
    evolving. It is too early to tell whether this new framework will
    successfully anticipate future crises. The new framework uses the Fund*s
    major forecasting tools, the WEO and the EWS, which have not performed
    well in anticipating prior crises. The WEO has not successfully
    anticipated past financial crises, and the Fund*s EWS models have had a
    high false alarm rate, having predicted many crises that did not occur.
    The forecasting of crises has been historically difficult for all
    forecasters due to complex underlying factors, including concerns about
    the reliability of important macroeconomic data on emerging market
    countries.

    The IMF Has Recently Implemented a New Vulnerability Assessment Framework

    The Fund has attempted for many years to identify countries vulnerable to
    financial crisis; however, their existing tools were insufficient to
    anticipate the financial crises of the 1990s and led the Fund in 2001 to
    develop the new vulnerability assessment framework. This comprehensive
    framework brings together detailed, country- specific knowledge and
    financial expertise of various IMF departments, including those with
    regional, macroeconomic, or forecasting expertise. The new framework
    monitors the vulnerability of key emerging market countries that borrow
    significantly from international capital markets. This information is
    provided in a quarterly report on crisis vulnerability. Fund staff report
    monthly on countries identified as vulnerable and provide more frequent ad
    hoc analyses during volatile periods.

    To conduct the vulnerability assessment, the Fund integrates six
    independent inputs that represent the analyses and perspectives of
    different departments of the Fund (see figure 2).

    Page 8 GAO- 03- 734 International Financial Crises

    Figure 2: IMF Vulnerability Assessment Framework

     World Economic Outlook: The WEO is a twice yearly publication that
    provides analyses of global economic developments. Through the WEO, the
    IMF provides current and following year forecasts for countries and
    regions of key economic variables such as economic growth, inflation, and
    current account. 2 According to Fund staff, WEO forecasts use the best
    available information and represent the most realistic estimate of key
    economic variables, including those that could help anticipate a

    financial crisis. The IMF uses these forecasts as an input in the
    vulnerability assessment to gauge the impact of unanticipated adverse
    changes in the global environment. For example, the WEO forecasts for
    selected countries may be recalculated to examine the impact of sudden

    2 The current account represents a country*s trade balance (exports less
    imports of goods and services) plus the net interest income (or payments)
    on outstanding international investments (or debts) plus net transfers
    (grants, workers* remittances).

    Page 9 GAO- 03- 734 International Financial Crises

    increases in oil prices or an unanticipated recession in the advanced
    economies.  Early Warning System models: The Fund uses internal and
    private

    sector EWS models that compute the probability of a country having a
    crisis over the following 12 to 24 months. The model examines a series of
    vulnerability indicators, including whether a country*s real exchange rate
    is overvalued, or whether the country has significantly depleted its
    foreign exchange reserves. 3 The output of the EWS models helps the Fund
    focus on specific areas of vulnerability. For example, if one variable,
    such as the exchange rate, signals a crisis, the Fund would

    more closely examine related components of the vulnerability assessment,
    such as a country*s external financing requirements.

     Country external financing requirements: On a quarterly basis, the Fund
    produces an internal assessment of a country*s ability to meet its total
    external debt obligations and estimates whether that country has
    sufficient foreign exchange to avoid a crisis. This assessment includes
    estimating a country*s ratio of foreign exchange reserves to short- term
    external debt, estimating the magnitude of its current account deficit,
    and considering whether and how it manages its exchange rate.

     Market information: On an ongoing basis, the Fund analyzes most
    countries* cost of borrowing on the international market and whether the
    country is paying a higher interest rate than similar countries. The Fund
    uses this information to provide an internal analysis of the private
    sector*s expectations of a country*s likelihood of default on its foreign
    debt and to identify possible evidence that financial problems are
    spreading across countries.

     Financial sector vulnerability: The Fund assesses the strengths and
    weaknesses of a country*s financial sector, including the banking system.
    IMF staff evaluate the financial sector*s vulnerability to changes in
    market conditions, such as fluctuations in interest and exchange rates.
    Although the detailed results of these assessments are used internally by
    the Fund, summaries of key findings are frequently published.

    3 Foreign exchange reserves are those external assets that are readily
    available to and controlled by monetary authorities. Reserves can be used
    for direct financing of payment imbalances, for indirectly regulating the
    exchange rate, and/ or for other purposes.

    Page 10 GAO- 03- 734 International Financial Crises

     Country expert perspectives: IMF country experts examine the data
    produced by the above analyses, supplementing those results with country-
    specific details such as the political risks of implementing certain
    government policies or the relevance of certain market information.

    Until 1999, the Fund used the WEO as the primary forecasting tool to help
    identify country risks and vulnerability to crises. The new vulnerability
    framework, which has been in operation for about 2 years, is a more
    comprehensive process. According to the Fund, the quarterly integration of
    detailed information from country experts who continuously monitor
    developments in their countries is a great strength of the new
    vulnerability framework. Effective analysis by Fund staff of the
    framework*s six elements could better allow the Fund to give timely advice
    to authorities in vulnerable countries. It is too early to tell whether
    this framework will be successful in anticipating future crises. We assess
    the performance of the WEO and EWS models, the Fund*s primary tools for
    anticipating crises prior to the implementation of the new framework in
    May 2001. 4 Forecasting Elements of the

    Vulnerability Framework Have Performed Poorly in the Past

    The new vulnerability assessment framework uses the Fund*s two major
    forecasting tools* the WEO and the EWS models* which have not performed
    well in anticipating prior crises. The WEO has not successfully
    anticipated the severe financial crises of the past decade. The Fund*s EWS
    models have had a high false alarm rate, having predicted many crises that

    did not occur. The WEO Is Not a Reliable Tool for Anticipating Crises

    Severe financial crises are characterized by a number of negative economic
    outcomes, including large declines in gross domestic product (GDP), also
    known as recessions. We found that the WEO had a poor record of
    forecasting such declines in GDP, tending instead to follow existing
    positive economic growth trends. In addition, the IMF indicates that the
    current account is a key variable in explaining financial crises. We found
    that for the current account, the accuracy of 75 percent of WEO country
    forecasts was worse than simply assuming that next year*s value is the
    same as this year. The WEO*s difficulty forecasting both GDP and current

    4 Data on the performance of the other four components of the framework
    were not made available to us because these elements are considered part
    of the staff level deliberative process and are not provided to the
    Executive Board.

    Page 11 GAO- 03- 734 International Financial Crises

    account demonstrate poor performance in anticipating the severe financial
    crises of the past decade.

    WEO Has Performed Poorly at Forecasting Recessions

    In most cases, countries experiencing a financial crisis also experience a
    severe recession in which their GDP declines significantly. Although most
    recessions do not involve a major financial crisis, successful
    anticipation of recessions, especially the most severe ones, would greatly
    increase the likelihood of anticipating impending financial crisis.
    However, we found that the WEO has a poor track record of forecasting
    recessions, including

    those directly associated with a financial crisis. The WEO did not
    forecast most of the recessions that occurred in emerging market countries
    in the last 10 years. During the 1991* 2001 forecast period, 134
    recessions occurred in all 87 emerging market countries. We found that the
    WEO correctly forecast only 15, or 11 percent, of those recessions,

    while predicting an increase in GDP in the other 119 actual recessions. 5
    The WEO is considerably more likely to forecast a recession when a
    recession has occurred in the prior year. However, a prior year recession
    did not occur in two- thirds of the recessions that the WEO failed to
    forecast. Thus, WEO forecasts generally follow the existing growth trend
    within a country, making it unlikely that the WEO would correctly forecast
    an unanticipated recession. Furthermore, this tendency to follow the
    current growth trend

    makes it especially difficult for the WEO to anticipate a financial crisis
    because nearly all of the crisis- related recessions of the last decade
    occurred in years following positive economic growth.

    Further illustrating our point, the WEO was unable to anticipate large
    declines in GDP, also known as recessions, that corresponded to 14 major
    financial crises of the past decade, including the Mexican and Asian
    financial crises (see table 1). 6 5 In one case, the WEO forecast zero
    growth.

    6 Several studies examining international financial crises from 1990* 2001
    identified 14 major financial crises in 12 prominent emerging market
    countries that were considered to be fundamentally sound before the
    crisis. Balance of payments problems, large GDP contractions, and
    financial panic commonly characterize these crises.

    Page 12 GAO- 03- 734 International Financial Crises

    Table 1: Anticipated and Actual GDP Growth Rate of 14 Financial Crises,
    1990* 2001

    Sources: Various crisis studies and IMF. a Actual GDP growth rate is from
    the WEO published 1 year later.

    b The Asian crisis began in Thailand in the summer of 1997 and spread to
    the other Asian countries in the latter part of 1997. The full effects of
    the crisis on the GDP of other Asian countries and Russia occurred in
    1998. Thus we are using 1998 actual GDP growth rate data for the Asian
    countries* Indonesia, Malaysia, Philippines, South Korea, and Thailand.
    The crisis in Brazil extended into1999.

    The WEO*s failure to identify these recessions demonstrates that it did
    not anticipate the corresponding financial crises. In 14 cases, we found
    that the WEO forecast strong economic growth, averaging a 4. 4 percent
    increase in GDP, despite large declines in actual GDP in 13 of 14 cases. 7
    In fact, actual GDP declined by an average of 5. 7 percent during the
    first full year of these 14 financial crises. Indonesia presents the most
    startling disparity, in which the WEO forecast a growth of 6.2 percent in
    its GDP, when in fact

    Indonesia*s GDP declined by almost 14 percent in the first full year of
    its financial crisis.

    Country Crisis years WEO forecast

    GDP growth rate (percent)

    Actual GDP growth rate a

    (percent)

    Argentina 1995 4.0 -4.4 Argentina 2001 3.7 -4.4 Brazil 1998 4.0 0. 1
    Ecuador 1999 2.5 -7.3 Indonesia b 1998 6.2 -13.7 Malaysia b 1998 6.5 -6.7
    Mexico 1995 4.0 -6.9 Philippines b 1998 5.0 -0.5 Russia 1998 5. 1 -4.6
    South Korea b 1998 6.0 -5.8 Thailand b 1998 3.5 -9.4 Turkey 1994 3.0 -5.5
    Turkey 2001 5.5 -7.4 Venezuela 1994 2. 5 -3.3 Average growth rates 4.4
    -5.7

    7 Brazil was the sole exception, having experienced a small 0.1 percent
    GDP annual growth rate during the 1998 financial crisis.

    Page 13 GAO- 03- 734 International Financial Crises WEO Does a Poor Job in
    Forecasting the Current Account

    According to the Fund, a country*s current account (primarily exports
    minus imports of goods and services) is a key variable in anticipating
    crises. Crises are associated with problems of external financing that
    result from a country having difficulty obtaining foreign exchange. Since
    exports are an important source of foreign exchange for developing
    countries, projections of a country*s current account balance provide
    information

    about the country*s ability to earn foreign exchange and to service its
    external debt. 8 According to the Fund, an unsustainably large current
    account deficit can contribute to or precipitate a crisis. 9 We found that
    WEO forecasts for current account were inaccurate most of

    the time. Our analysis for the 87 emerging market countries shows that,
    for more than 75 percent of the countries, the WEO current account
    forecasts were less accurate than if the Fund had simply assumed that the
    next year*s current account would be the same as this year*s. The results
    are even more

    dramatic for G- 7 countries; a forecast of no change was a better
    predictor than the WEO forecast for six of the seven countries. This
    demonstrates that, even in stable economies with excellent data, the WEO
    has done a poor job of forecasting this key crisis anticipation variable.
    (See appendix III for more detailed explanation on our methodology and
    findings).

    8 A deficit in the current account can also be financed by capital
    inflows, such as foreign private investment, a drawing down of a country*s
    international reserves, by bilateral or multilateral loans, or by
    provision of exceptional finance, such as debt service relief. 9 Fund
    studies indicate that the current account deficit, as a percentage of GDP,
    is significantly larger before a crisis than during relatively stable
    periods, which suggests that unsustainable current account deficits tended
    to be part of the general overheating of the

    economy before a crisis.

    Page 14 GAO- 03- 734 International Financial Crises

    The Fund*s Early Warning System Models Indicate Many Crises That Do Not
    Occur

    Since 1999, the Fund has analyzed the results from internal and private
    sector EWS models in its crisis anticipation efforts. The Fund*s internal
    efforts focused on two EWS models to systematically identify countries
    vulnerable to crises: the Kaminsky, Lizondo, and Reinhart (KLR) model,
    which monitors a set of 15 monthly variables that signal a crisis whenever

    any cross a certain threshold; and the Developing Country Studies Division
    (DCSD) model, which uses five variables to compute the probability of a
    crisis occurring in the next 24 months. 10 The Fund*s models use a
    variableby- variable approach that allows economists to determine which
    variables are signaling the crisis. Internal assessments of the Fund*s EWS
    models show that they are weak predictors of actual crises. While the
    models worked reasonably well in anticipating Turkey*s recent financial
    crisis, they did not successfully anticipate Argentina*s financial crisis
    in 2002.

    According to the Fund, the models* most significant limitation is that
    they have high false alarm rates; that is, they predict many crises that
    did not occur. In about 80 percent of the cases where a crisis was
    predicted over the next 24 months, no crisis occurred. Furthermore, in
    about 9 percent of the cases where no crisis was predicted, there was a
    crisis. 11 Forecasting Crises Has

    Been Historically Difficult for All Parties

    Financial crises have been historically difficult to anticipate because of
    a number of complex underlying factors. Economic outcomes are often
    influenced by unanticipated events such as conflicts and natural
    disasters. Many factors, in addition to weaknesses in a country*s
    financial structure, can lead to a crisis. These include economic
    disturbances, such as an unanticipated drop in export prices, political
    events, and changes in investor sentiment leading to sudden withdrawals of
    foreign capital. Furthermore, data may be inadequate, particularly in
    developing countries

    where data are often not timely and are of poor quality. 10 More
    specifically, the DCSD model is a multivariate probit regression. The five
    variables used are real exchange rate overvaluation, current account,
    foreign exchange reserve losses, export growth, and the ratio of short-
    term debt to foreign exchange reserves. Also, the model defines a currency
    crisis as the weighted average of 1- month changes in exchange rate and
    reserves more than 3 (country- specific) standard deviations above country
    average.

    11 The DCSD model had a cutoff probability of 23 percent and the KLR model
    had a cutoff probability of 15 percent. A forecast probability above these
    cutoff points is deemed to signal a crisis.

    Page 15 GAO- 03- 734 International Financial Crises

    Forecasters consistently fail to foresee crises and recessions. Forecasts
    produced by private sector economic forecasters, governments, and
    multinational agencies including the IMF and the Organization for Economic
    Cooperation and Development, routinely fail to foresee the coming of
    crises and recessions, and often fail to outperform the naive model, which
    simply assumes that next year*s outcome will be the same as this year*s.
    This is true even for evaluations of recent U. S. forecasts of GDP and
    inflation. Our review of a number of forecast evaluation studies confirms
    that the inability to predict recessions is a common feature of growth
    forecasts for both industrialized and developing countries. The studies
    also showed that forecast accuracy improves as the time horizon gets
    shorter, and that there is little difference in forecast accuracy between
    private sector and WEO forecasts.

    Some Progress in Implementing Crisis Prevention Initiatives, but
    Challenges Remain

    In the late 1990s, the Fund and the World Bank began implementing two
    crisis prevention initiatives designed to assist four parties: IMF staff,
    World Bank staff, member country governments, and private sector
    participants. The first initiative, the financial sector assessments,
    provides reports on aspects of member countries* financial sectors such as
    banking systems and crisis management capacity. The second, the standards
    initiative, assesses member countries* adherence to 12 standards in areas
    such as banking supervision and economic data dissemination. Parties* use
    of the information provided by these two initiatives has been mixed, and
    several significant challenges remain. Fund staff frequently incorporate
    information from the assessments into their policy advice when assessments
    are available; however, assessments have not been completed

    in some important emerging market countries primarily because
    participation is voluntary. Bank staff*s use of the assessments to inform
    country development assistance programs is also affected by gaps in the
    completion of some assessments and by borrower countries* competing
    development demands. Member country governments sometimes use the
    assessments in prioritizing their reform agendas but often find the
    reforms too difficult to implement. Some private sector participants find
    the published reports untimely, outdated, and too dense to be useful in
    making investment decisions. The IMF and the Bank acknowledge that the

    initiatives cannot prevent all crises because recommended reforms require
    many years to be fully implemented, and crises can be caused by factors
    outside the scope of the reforms.

    Page 16 GAO- 03- 734 International Financial Crises

    Two Key Crisis Prevention Initiatives Target Four Parties

    In the wake of the Mexican and Asian financial crises of the 1990s, the
    Fund and the World Bank became increasingly aware of the importance of
    transparent financial data and policies, stronger financial systems, and
    better- functioning markets as a complement to member country

    governments* sound macroeconomic policies. Fund evaluations acknowledge
    that the institution failed to collect information that could have enabled
    it to detect financial and corporate sector vulnerabilities and

    to provide appropriate policy advice to the affected countries before the
    crises occurred.

    In response to this, in the late 1990s, the Fund and the Bank jointly
    launched two initiatives to prevent the long- term likelihood of financial
    crises. The first initiative, the Financial Sector Assessment Program
    (FSAP), consists of in- depth assessments of key elements of member
    countries* financial sectors. These elements include the structure of
    financial markets, financial systems* response to changes in key variables

    such as exchange rates, legal arrangements for crisis management, and the
    quality of financial sector supervision. The second initiative, the
    Reports on the Observance of Standards and Codes (ROSC), consists of
    assessments of member countries* adherence to 12 standards 12 related to
    transparency in government policy making and operations, 13 financial
    sector regulation, and corporate sector practices (see appendix V). 14
    Building on earlier efforts to assess transparency, in 1999, the IMF and
    the Bank began conducting joint assessments of observance of standards
    related to

    12 According to the Fund, 11 of these 12 standards are internationally
    accepted. The World Bank*s corporate insolvency and creditor rights
    standard is still in draft form. 13 In addition to the ROSC, the Fund
    launched an experimental program in 2000 to help prevent possible misuse
    of Fund resources. This program assesses weaknesses in central banks*
    ability to safeguard Fund resources through internal controls, accounting,
    reporting, auditing systems, and legal structure. In 2002, the Executive
    Board made assessments of central banks* safeguards a permanent policy.
    See appendix VI for an update on progress in

    this area. 14 In February and March 2003, the IMF and Bank published
    reviews of the FSAP and ROSC initiatives. See International Monetary Fund
    and World Bank, Financial Sector Assessment Program* Review, Lessons, and
    Issues Going Forward (Washington, D. C.: 2003);

    International Monetary Fund and World Bank, International Standards:
    Strengthening Surveillance, Domestic Institutions, and International
    Markets (Washington, D. C.: 2003); International Monetary Fund,
    International Standards: Background Paper on Strengthening Surveillance,
    Domestic Institutions, and International Markets (Washington, D. C.:
    2003); World Bank, Reports on the Observance of Standards and Codes

    Background Paper on Standards Review: Assessing Progress and Lessons
    Learned from Bank- Led ROSC Modules (Washington, D. C.: 2003).

    Page 17 GAO- 03- 734 International Financial Crises

    financial sector regulation, covering areas such as banking supervision
    and securities regulation. The Bank began performing assessments of
    standards related to corporate sector practices, including private sector
    accounting rules and corporate governance principles, in 2000. Some
    transparency, financial sector, and corporate sector standards may be
    assessed under the FSAP. Country participation in both initiatives is

    voluntary. The Fund and the Bank initially considered making participation
    in the ROSC assessments mandatory for member countries after determining
    that the Fund*s Articles of Agreement could allow such a requirement. IMF
    or World Bank staff lead FSAP and ROSC assessment teams, with

    participation by experts from national central banks and supervisory
    agencies and standard- setting bodies such as the Basel Committee and the
    International Organization of Securities Commissions (IOSCO). Before
    undertaking an FSAP or ROSC, the Fund and the Bank work with country
    governments to choose areas on which to focus. During the assessment
    process, FSAP and ROSC teams conduct at least one in- country visit,
    allowing team members to work with government officials from the finance

    ministry, the central bank, and regulatory bodies to collect information
    for the assessment. 15 For example, an FSAP team in Russia analyzed
    financial information for the largest banks and single largest corporation
    to determine how changes in economic variables such as oil prices might
    affect the banking system. The South Korean ROSC team interviewed
    government officials at financial sector regulatory entities and private
    sector representatives to determine how closely regulatory practices
    conform to standards and to identify weaknesses that could put the
    financial sector at risk.

    The two initiatives provide information to assist four parties that play a
    role in crisis prevention: the Fund, the World Bank, member country
    governments, and private sector investors. In- depth information on member
    countries* financial sectors and adherence to standards of good practice
    is intended to help the IMF and the Bank fulfill their missions. Fund
    staff identify countries* vulnerabilities and develop appropriate advice
    to redress them; Bank staff identify long- term financial sector
    development needs and formulate relevant lending and nonlending responses.
    Member countries can use these assessments to help prioritize reform
    agendas and win domestic support for difficult policy decisions that

    15 Appendix VII contains a diagram of the FSAP and ROSC assessment
    process.

    Page 18 GAO- 03- 734 International Financial Crises

    may make their financial sectors and institutions more resistant to
    crisis. The Fund and the Bank often provide technical assistance to help
    governments build capacity to implement reforms. The financial crises of

    the 1990s also raised awareness of the private sector*s role in crisis
    prevention. Thus, the Fund and the Bank expect the assessments to help
    private sector participants make sounder investment decisions, thereby

    reducing volatility in capital markets. Parties* Use of Assessments Is
    Mixed, and Significant Challenges Remain

    The use of FSAP and ROSC assessments in crisis prevention efforts is
    mixed, and significant limitations remain. Fund staff use the assessments,
    when available, as inputs for the policy advice they provide to member
    country governments. However, the Fund lacks crucial information about
    vulnerabilities to financial crisis because some major emerging market
    countries have not participated in the assessments. World Bank staff*s use
    of the assessments to inform development assistance priorities is also
    affected by these gaps in participation and by borrower countries*
    competing development needs. Many member country governments face
    limitations in using assessments to make policy decisions because the
    reforms recommended in the assessments are often difficult to implement.
    Finally, some private sector participants find assessments of limited use
    because they are untimely, outdated, and dense. The Fund Uses Assessments,

    When Available, to Inform Policy Advice

    The IMF uses FSAP and ROSC assessments, when available, as inputs for the
    policy advice it provides member country governments. According to Fund
    officials, these assessments highlight issues such as weak banking
    supervision or high levels of debt held in foreign currency that could
    make

    countries vulnerable to crisis. The assessments also provide
    recommendations to address these issues. The findings and recommendations
    inform the discussions of policy issues that Fund officials have with
    member country authorities during Article IV

    consultations. 16 For example, when an FSAP was performed in Mexico,
    Mexican authorities had begun replacing a system where the government
    fully insured all bank deposits with one that covers deposits up to a
    certain 16 In accordance with Article IV of the IMF*s Articles of
    Agreement, the IMF holds regular

    consultations, normally every year, with each of its members. These
    consultations focus on the member*s exchange rate, fiscal, and monetary
    policies; its balance of payments and external debt developments; the
    influence of its policies on the country*s external accounts; the
    international and regional implications of those policies; and on the
    identification of potential vulnerabilities. (* IMF Surveillance: A
    Factsheet,* http:// www. imf. org, downloaded 4/ 23/ 03).

    Page 19 GAO- 03- 734 International Financial Crises

    limit. The FSAP team was concerned because this reform was undertaken
    before Mexico had developed a well- defined framework for closing
    unprofitable banks. Without a clear framework for bank closures, the
    introduction of limited deposit insurance could damage depositor
    confidence in the banking system and precipitate a banking crisis.
    According to Fund officials, the FSAP team and Mexican authorities
    discussed the need to create such a framework, and a subsequent Article IV
    mission reviewed the government*s progress in this area. In Poland, an
    FSAP team discovered that Polish households and small businesses had high
    levels of debt held in foreign currency. The team was concerned that a

    depreciation of Polish currency could raise the cost of these loans and
    cause widespread repayment difficulties, which could in turn lead to a
    banking crisis. FSAP team members raised this issue with Polish central
    bank authorities and followed up again during the next Article IV
    consultation. In both countries, government officials followed the Fund*s
    advice and implemented reforms. The Mexican government began

    developing a framework for closing banks, and Poland*s central bank
    established a team to monitor household and small business debt. Since
    1999, FSAP assessments have been conducted in more than 40 member
    countries and ROSC assessments in about 90 member countries. However, we
    found that assessments have not been completed for some major emerging
    market countries, limiting the Fund*s awareness of crisis vulnerabilities
    in certain countries. Appendix VIII contains a record of country
    participation in and publication of FSAP and ROSC assessments. Fund and
    Bank staff encourage participation in FSAP and ROSC

    assessments by countries whose economies have worldwide or regional
    implications or have known vulnerabilities to a financial crisis, but
    officials acknowledge that some governments have persistently resisted
    their efforts. According to our analysis, between 1999 and 2003, 17 45
    percent of 33 major emerging market countries participated in an FSAP. 18
    However, the Fund has not performed FSAPs in important countries such as
    China

    and Thailand because their authorities have not agreed to participate.
    These gaps in participation limit the Fund*s ability to develop policy
    advice based on in- depth knowledge of their financial sectors.

    17 Our analysis is based on Fund and Bank data through March 2003. 18
    These 33 countries, a subset of the 87 we analyzed in the previous
    section, represent the most significant emerging market participants in
    the international capital markets, as identified by J. P. Morgan.

    Page 20 GAO- 03- 734 International Financial Crises

    According to the Fund, the Mexican and Asian financial crises were caused,
    in part, by vulnerabilities in areas covered by the ROSCs. Our analysis
    found gaps in participation in assessments of several key standards that
    the Fund identifies as contributing factors to past crises (see figure 3).
    For example, only one- third of major emerging market countries have
    participated in assessments of their adherence to standards for
    dissemination of economic and financial data. About half have participated
    in the fiscal, monetary and financial policy formulation assessments and
    the banking supervision assessment. In addition, Fund documents point to
    limited progress in assessing adherence to the four World Bank- led
    corporate sector standards 19 (accounting, auditing, corporate governance,
    and insolvency regimes), which play a key role in the effective operation
    of domestic and international financial systems. Less than one- third of
    the 33 major emerging market countries have participated in one or more
    assessments related to accounting and auditing.

    Figure 3: Participation of 33 Major Emerging Markets in Key ROSC
    Assessments, 1999* 2003

    19 The Bank performs accounting and auditing assessments simultaneously
    and publishes them as a single report. An agreed- upon standard for
    insolvency regimes has not yet been finalized. The Bank*s draft standard
    for insolvency regimes is under review by the Bank and the Fund.

    Page 21 GAO- 03- 734 International Financial Crises

    The Fund asserts that its delayed response in preventing or mitigating the
    Mexican and Asian crises was partially caused by insufficient information
    on these vulnerabilities. For example, according to the IMF, the Mexican
    government*s publicly available data was outdated and incomplete in 1993*
    94, which contributed to significant delays in responding to the country*s
    excessive indebtedness. The Fund also was unaware of some Asian

    countries* unsound corporate accounting practices, which contributed to
    the Asian financial crisis. Continued participation gaps in these
    assessments suggest that the Fund still lacks crucial information about
    some countries* potential vulnerability to crisis.

    The World Bank*s Use of Assessments in Formulating Country Development
    Assistance Programs Is Mixed

    The World Bank acknowledges the importance of FSAP and ROSC assessments in
    formulating its financial sector development programs, but limited
    participation in corporate sector assessments (described earlier) affects
    the Bank*s ability to respond to weaknesses in borrower countries*

    financial sectors. According to the Bank, country participation in
    corporate sector assessments has been lower than in areas related to
    transparency and financial sector regulation because the Bank has
    experienced delays in finalizing standards and methodologies for
    evaluating the corporate sector. For example, the methodologies for
    performing assessments of the accounting and auditing standards were not
    finalized until October 2000.

    Bank officials acknowledge that even when assessments are available, Bank
    staff do not always incorporate the issues raised as a key priority in
    formulating its country development assistance programs. In justifying

    their limited prioritization of FSAPs and ROSCs, Bank officials cited
    competing development demands and timing issues. First, Bank officials
    stated that they must balance borrower countries* financial sector reform
    needs with other demands for development assistance. Most borrower country
    governments have multiple concerns, and Bank staff may determine that its
    resources will have more impact in areas other than financial sector
    development. Second, Bank officials cited the scheduling of the FSAP and
    ROSC assessments as a reason for their limited use since the timing of
    many assessments does not coincide with the Bank*s preparation of Country
    Assistance Strategies. 20 20 The Country Assistance Strategy describes the
    Bank*s assessment of development

    priorities in each borrower country and identifies the level and
    composition of assistance to be provided based on this assessment. These
    strategies are currently prepared every 3 years.

    Page 22 GAO- 03- 734 International Financial Crises

    Member Country Governments* Use of Assessments Is Limited

    Although member country authorities sometimes use FSAP and ROSC
    assessments to inform policy decisions, reforms recommended in the
    assessments are often difficult to implement. Some member country
    governments have faced obstacles to implementing reforms, including
    political opposition, legal constraints, and lack of technical capacity.
    For

    example, IMF officials stated that political opposition has limited the
    South Korean government*s progress in eliminating extrabudgetary funds, a
    key recommendation of the fiscal transparency ROSC. Extrabudgetary funds
    diminish transparency because they are exempt from rules that require
    scrutiny and prioritization of expenditures for most of South Korea*s
    budget. Fund officials cited Peru as a case where legal constraints
    delayed reform efforts. The FSAP and banking supervision ROSC found that
    protecting bank supervisors from the political pressures of the powerful
    bankers* lobby would strengthen Peru*s banking supervision. According to
    Fund officials, existing legislation precluded awarding supervisors
    greater independence, and passage of new legislation was delayed. In
    Russia, limited technical capacity interfered with the government*s
    ability to

    implement reform recommendations. For example, the FSAP team reviewed the
    government*s proposal to stimulate competition in the banking sector by
    introducing a deposit insurance system for household deposits. However,
    Fund staff noted that Russia*s bank supervisory agency lacks the capacity
    to implement a deposit insurance system for a large number of banks.

    Private Sector Participants Find Assessments Difficult to Use

    The Fund claims that private sector participants increasingly use the
    results of FSAPs and ROSCs to inform investment decisions and risk
    management. However, representatives of major international investment
    firms and ratings agencies we interviewed stated that the reports were
    untimely, outdated, and too dense to be useful. For example, several
    respondents indicated that delays in publishing ROSC assessments reduced
    their usefulness. Some private sector participants stated that ROSC
    reports and FSAP summaries, known as Financial System Stability
    Assessments (FSSAs), should be published within 6 months of performing the
    assessments. However, our analysis of the 58 ROSC reports published for
    major emerging market countries found that in one- third of the cases, 9
    months or more elapsed between assessment and publication. Several private
    sector participants we interviewed stated that outdated ROSC reports are
    unreliable for decision making. The Fund acknowledges that assessments
    must be current for private sector participants to use them. According to
    Fund data, 13 countries have published an update of at least one ROSC
    module. However, IMF officials estimate that, of the more than 40 FSAPs
    performed to date, only 4 have been fully updated. Some private

    Page 23 GAO- 03- 734 International Financial Crises

    sector participants also stated that FSSA and ROSC reports are not clearly
    written. Representatives of one multinational investment bank stated that
    the assessments are written in a way that is difficult to understand,
    which limits the reports* usefulness for investment decisions. While these

    interviews were limited in number and may not be representative of all
    private sector participants, they do provide an indication of the problems
    these individuals may currently have in using FSAPs and ROSCs.

    Fund and Bank outreach sessions and a 2002 Fund survey corroborated our
    findings on private sector participants* difficulties in using ROSC
    assessments. The Fund reports that private sector participants place high
    priority on timely publication and frequent updates of ROSC assessments.

    For example, several participants observed that ROSC reports for Argentina
    had not been updated since their publication in 1999. Moreover,
    respondents to the Fund*s survey commented that ROSC assessments should
    state more clearly the deficiencies in a country*s adherence to a
    standard. In a March 2003 review of the standards initiative, the Fund and
    the Bank concluded that ROSC reports would be more useful if they stated
    the main findings and their significance clearly and prioritized
    recommendations more explicitly. Crisis Prevention Initiatives

    Cannot Prevent all Crises The Fund and the World Bank acknowledge that
    FSAP and ROSC assessments cannot prevent all crises because recommended
    reforms may

    require many years to be fully implemented and because crises can be
    caused by factors outside the reforms* scope. For example, Argentina
    participated in four ROSC assessments in 1999 to improve economic data
    dissemination; banking supervision; and transparency in the formulation of
    fiscal, monetary, and financial policies. According to senior IMF
    officials, the Argentine government followed many of the recommendations
    generated by these assessments, but their actions did not address
    vulnerabilities related to weak fiscal policy and a fixed exchange rate
    regime that contributed to Argentina*s 2001 crisis. Fund officials cite
    Turkey as another example of a country that made considerable progress in
    improving transparency and data provision based on reforms

    recommended by the fiscal transparency and economic data dissemination
    ROSC assessments of 2000* 2001. However, according to the Fund, these
    reforms could not have prevented Turkey*s 2001 crisis, which originated
    with declines in its exchange rate.

    Page 24 GAO- 03- 734 International Financial Crises

    The Fund*s Efforts to Better Resolve Future Financial Crises Face
    Significant Challenges

    Fund officials assert that the current process for renegotiating the terms
    of member countries* loans with external private sector creditors is
    lengthy and costly. In 2001, the Fund began considering the SDRM, an
    international legal framework that would allow a majority of a country*s
    external creditors to approve a restructuring of most private sector
    loans. The Fund is also encouraging members to include CACs in bonds,
    which would allow a majority of bondholders to renegotiate the terms of
    that bond. Although some elements of both approaches are acceptable to the
    private sector and governments, a number of political, legal, and
    technical challenges stand in the way of implementing the SDRM; it seems
    unlikely that these issues will

    be resolved in the immediate future. While private sector officials expect
    that many restructurings need only involve the private sector and the
    debtor country, under some circumstances, voluntary debt restructurings
    will not adequately resolve all financial crises. These officials stated
    that, in those cases, the Fund should provide short- term loans to
    eligible countries to help fill their external financing gaps. However,
    concerns have been raised by some financial experts and government
    representatives that such Fund loans have the potential to increase the
    probability of future crises. In response to these concerns, the Fund has
    clarified and strengthened its

    policy of lending into crisis situations. Efforts to Resolve Future
    Sovereign Debt Crises Face Significant Challenges

    According to the Fund, countries facing severe liquidity problems often go
    to extraordinary lengths to avoid renegotiating or restructuring the terms
    of their loans. They do so because, in the past, restructuring damaged the
    economy and the banking system of participating countries. In some cases,
    even when a voluntary restructuring process is initiated, individual
    creditors may hold out for the best possible terms or sue in an attempt
    for better terms. Additionally, countries believe that creditors also may
    be unwilling to make future loans if they default on their existing debt.

    The SDRM approach is an attempt to create a more orderly, predictable, and
    comprehensive restructuring process and to lower the costs of
    restructuring for both the debtor and creditors. The approach sought to
    reduce the duration of the restructuring process from years to months and
    to provide incentives to restructure debt before default to better protect
    debtor and creditor interests. In the case of debtors, the Fund maintains
    that an orderly restructuring process could reduce the likelihood of a
    reduction in future capital flows. For creditors, it could provide more
    favorable repayment terms from the restructured debt.

    Page 25 GAO- 03- 734 International Financial Crises

    The SDRM is a proposed international legal framework that would allow a
    member country to declare its debt unsustainable and invoke a process to
    restructure most of its external private sector loans. 21 A specified
    majority

    of the country*s external creditors would vote to approve the terms for
    restructuring, which would bind all eligible private sector creditors. The
    framework is designed to increase the incentives for the Fund*s member
    countries and their creditors to reach a rapid and collaborative agreement
    on restructuring unsustainable debt.

    A number of political, legal, and technical challenges stand in the way of
    implementing the SDRM, and it seems unlikely that these issues will be
    resolved in the immediate future. According to the Fund, successful
    implementation of SDRM will require overcoming certain political
    constraints. The SDRM could be put into practice either by countries
    adopting a new international treaty or by amending the Fund*s Articles of
    Agreement. Both options would be difficult to implement since a number of
    countries have indicated opposition to the SDRM. The draft framework
    recommended that the SDRM be created through an amendment to the Fund*s
    Articles of Agreement because the SDRM is closely related to the

    role already assigned to the Fund under the Articles in the resolution of
    its members* external financial obligations. However, the Fund
    acknowledged that, given the opposition of some countries, changing the
    Articles could be

    difficult to achieve since it requires acceptance by three- fifths of the
    members, having 85 percent of the total voting representation. The United
    States, for example, could unilaterally veto any proposed amendment to the
    Fund*s Articles given its 17 percent voting representation.

    21 The SDRM proposal has undergone several revisions. Our report discusses
    the proposal presented to the International Monetary and Financial
    Committee in April 2003. See http:// www. imf. org/ external/ np/ omd/
    2003/ 040803. htm.

    Page 26 GAO- 03- 734 International Financial Crises

    A key legal challenge to the implementation of SDRM is the need for most
    countries to change their domestic laws to conform to the requirements of
    any new Fund articles. Before a member country can vote to accept an
    amendment, it must take all the necessary steps needed under its own
    domestic law to ensure that the amendment will be given full force and

    effect under its domestic law. However, some Fund members have raised
    concerns over whether the domestic legal systems of some member countries
    could accommodate a new legal framework that applied to preexisting
    claims. The proposed SDRM approach also faces technical challenges. For
    example, the proposed framework does not specify how

    the claims of official bilateral creditors and some guaranteed domestic
    debts would be treated. The Fund is consulting with the Paris Club on how
    the Club*s practices may be modified to better facilitate coordination
    between official bilateral and the private creditors in a debt
    restructuring process. 22 The Fund Has Encouraged

    the Adoption of Collective Action Clauses

    CACs are terms in individual bonds that permit a specified majority of
    sovereign bondholders to agree on a debt restructuring that would bind all
    holders of that bond. In June 2002, the Fund*s Executive Board endorsed
    the use of certain CAC provisions in new bonds and agreed to encourage
    member countries to incorporate CACs into their sovereign bonds in future
    restructurings. Inclusion of these clauses into new bonds would be
    voluntary.

    22 The Paris Club is an informal group of official creditors whose role is
    to find solutions to the payment difficulties experienced by debtor
    countries. Paris Club creditors agree to provide a country with debt
    relief through a postponement and, in the case of concessional
    rescheduling, a reduction in debt service obligations.

    Page 27 GAO- 03- 734 International Financial Crises

    The Fund views CACs and SDRM as complementary instruments in resolving
    future financial crises. According to the Fund, the existence of CACs in
    certain bond agreements inspired the development of the SDRM framework.
    Although the Fund has not created its own CAC framework, it has endorsed
    the use of two key features from a G- 10 Working Group Report 23 and an
    Industry Associations draft proposal. 24 These features include the
    following:

     Majority restructuring provision 25 enables a qualified majority of
    bondholders to bind all holders of a particular bond to the financial
    terms of a restructuring, both before and after a default. Although
    majority restructuring provisions have generally been included in bonds
    governed by the laws of the United Kingdom or Japan, they have not been
    included in bonds governed by the laws of the United States or Germany.
    Majority enforcement provisions prevent a minority of creditors from

    pursuing disruptive legal action after a default and before reaching a
    restructuring agreement. Many international sovereign bonds governed by
    both U. S. and English law contain these provisions. Specifically, the
    Fund supports the requirement that (1) an affirmative vote of a minimum
    percentage of bondholders is necessary to approve claims following a
    default and (2) a specified majority of bondholders can reverse an
    approval of a claim that has already occurred.

    In early 2003, Mexico became the first emerging market country to issue a
    public, SEC- registered global bond with CACs under New York law. Previous
    issues under New York law by Lebanon, Qatar, and Egypt had

    been placed privately to institutional investors and included only a
    limited 23 The G- 10 Working Group on Contractual Clauses (the Working
    Group) was formed in June 2002 at the behest of ministers and governors.
    The mandate of the Working Group was to consider how sovereign debt
    contracts could be modified to resolve debt crises in a more orderly
    fashion.

    24 The proposal was put forward to Fund members for consideration by the
    Institute of International Finance and six other financial industry trade
    associations. The six financial industry associations are the Emerging
    Market Traders Association (EMTA), the

    International Primary Market Association (IPMA), the Bond Market
    Association (BMA), the Securities Industry Association (SIA), the
    International Securities Market Association (ISMA), and the Emerging
    Markets Creditors Association (EMCA).

    25 These provisions are referred to in the G- 10 Working Group Report as
    *majority amendment provisions.*

    Page 28 GAO- 03- 734 International Financial Crises

    range of CACs. For example, the bonds issued by Egypt and Qatar included a
    very limited form of majority enforcement provisions, while Lebanese bonds
    did not contain them at all. Since Mexico*s successful

    issue, Brazil, South Africa, and Korea have issued bonds with CACs.
    Uruguay included CACs in the bonds resulting from its debt exchange. The
    details of the Brazil, South Africa, Korea, and Uruguay bond provisions
    were not available at the time we conducted our review.

    Some countries criticize CACs because they would only apply to new bond
    offerings and not existing bonds. Accordingly, during a restructuring of a
    country*s bond obligations, not all creditors would be bound by the CAC
    provisions. Borrowing countries also contend that inclusion of CACs in
    bond offerings could suggest to creditors that countries anticipate having

    difficulty repaying their loans. In response, creditors may charge a
    higher interest rate. However, a May 2000 academic study 26 compared
    interest rates on bonds issued in the United States (where CACs are not
    used) with the United Kingdom (where CACs are used) and found that CACs do
    not contribute to higher rates in United Kingdom bonds.

    While Private Sector Officials Prefer Voluntary Debt Restructuring, They
    Expect Continued Fund Loans in Exceptional Circumstances

    To date, officials from the private sector, including lenders, have
    expressed preference for continuing the current voluntary process, which
    only involves the private sector and borrowing countries, in the efforts
    to restructure sovereign debts. Many private sector officials we
    interviewed oppose the proposed SDRM approach and the Fund*s attempts to
    integrate CACs into new bond issues, partly because they would interfere
    with the normal bargaining process. They maintain that a voluntary
    approach to the restructurings that took place from 1998 to 2001 in
    Ecuador, Pakistan, Russia, and Ukraine were successful. 27 Private sector
    officials assert that these and other experiences have worked well enough,
    and that a

    substantive change in current market practice is unnecessary. In contrast
    to the Fund*s assertion that new approaches are needed to make
    restructurings shorter and less expensive, private sector officials note
    that

    26 See Eichengreen, Barry and Ashoka Mody. *Would Collective Action
    Clauses Raise Borrowing Costs?* Working paper, Center for International
    and Development Economics Research, Berkeley, CA, 2000.

    27 The restructurings in Russia involved domestic debt and Soviet- era
    foreign debt owed to commercial banks. In 1998, Pakistan froze withdrawals
    in foreign currency from all nonresident foreign currency deposits and
    subsequently reached restructuring agreements with these nonresident
    investors.

    Page 29 GAO- 03- 734 International Financial Crises

    most recent voluntary restructurings successfully concluded in 1 year or
    less and that creditor holdouts or litigation did not significantly delay
    the restructurings. 28 While private sector officials expect that many
    restructurings need only

    involve the private sector and the debtor country, under some
    circumstances, voluntary debt restructurings will not adequately resolve
    all financial crises. In those cases, they said the Fund should provide
    loans to

    eligible countries to help fill their external financing gaps. Such loans
    would assist the restructuring process and facilitate efforts at
    implementing necessary reforms. However, large Fund loans, such as those
    given during the Asian financial crises, have received substantial
    criticism

    from financial experts and government representatives, including U. S.
    government officials. One concern is that the possibility of receiving
    substantial financial assistance provides an incentive for debtor
    countries to adopt unsustainable economic policies to forestall needed
    reform. Another concern is that these large loans may encourage private
    sector creditors to continue providing large capital flows to countries
    with unsustainable economic policies because these otherwise risky
    investments have the potential of being *bailed out* by future Fund loans.
    This condition is referred to as *moral hazard.* According to these
    critics, efforts to help resolve existing financial crises through large
    Fund loans may increase the probability of future crises due to these two
    concerns. The Fund has advocated the SDRM framework and CACs to replace
    the current voluntary approach, partially in response to concerns over the
    potential adverse affects of its lending.

    The Fund Has Clarified and Strengthened its Lending Practices to Address
    Concerns Over Exceptional Lending

    To reduce the risk that Fund loans would increase the probability of
    future financial crises, the Fund clarified and strengthened its policy of
    lending in crisis situations. The Fund has clarified elements of its
    Lending into Arrears Policy and strengthened its criteria for requesting
    large short- term loans under the Supplemental Reserve Facility (SRF).
    Since 1997, nine

    countries have received loans under the two mechanisms. The Fund*s Lending
    into Arrears Policy permits the IMF to provide resources to countries that
    are unable to repay their external creditors and are thus considered in
    default. Conceived in the late 1980s and amended in 28 For some countries,
    the negotiating process entailed several years and more than one
    restructuring agreement.

    Page 30 GAO- 03- 734 International Financial Crises

    the late 1990s, the policy is designed to protect the value of creditor
    assets while providing creditors with incentives to enter rapidly into
    restructuring negotiations with countries. The Lending into Arrears Policy
    increases the likelihood that a country*s private sector lenders would
    agree to reduce the value of their loans because Fund resources reduce
    short- term fiscal pressures experienced by the country while in default.
    A country is eligible for Fund resources while in default if the Fund
    determines that the debt burden is unsustainable, and the country is
    making satisfactory progress in implementing reforms. Additionally, the
    country must have demonstrated a

    good faith effort to reach a restructuring agreement with creditors to
    restore its ability to repay its debt. In 2002, the Fund clarified the
    criteria to be used to determine whether the debtor country is making a
    good faith effort. For example, the Fund would consider how quickly the
    debtor engaged in negotiations with its creditors after it defaulted. To
    date, the

    Fund has lent into arrears on international bonds on four occasions*
    Ukraine, Ecuador, Moldova, and Argentina.

    Introduced in 1997, the SRF provides large short- term loans to members
    experiencing exceptional balance of payments difficulties prior to a
    default. The interest rates on these loans are much higher than standard
    Fund loans. The increased cost of these loans is expected to reduce the
    probability that countries consider Fund resources a viable means for

    underwriting unsustainable economic policies. Higher loan terms also
    increase incentives for early repayment and compensate for additional
    repayment risks to the Fund. Countries are expected to repay SRF loans

    within 2 to 2 1/2 years, but they may request extensions of up to 6
    months. All SRF loans carry a substantial surcharge of 3* 5 percentage
    points. In 2003, the Fund strengthened its criteria for providing large
    short- term loans under the SRF. For example, countries requesting SRF
    loans must provide a more extensive justification for their repayment
    difficulties. Additionally, the member has to demonstrate good prospects
    of regaining access to private capital markets within the time period that
    Fund resources are

    outstanding to minimize long- term reliance on Fund resources. To date,
    the Fund has provided SRF loans on nine occasions to six countries,
    including Korea, Russia, Brazil, Turkey, Argentina, and Uruguay (see
    appendix IX).

    Conclusion In accordance with its goal of strengthening the international
    financial system, the Fund has undertaken a number of reforms to better
    anticipate,

    prevent, and resolve sovereign financial crises. The Fund*s new
    vulnerability assessment process is more comprehensive than its previous
    crisis anticipation efforts, but it is too soon to judge its
    effectiveness. The

    Page 31 GAO- 03- 734 International Financial Crises

    Fund*s proposed approaches to better resolve financial crises have met
    considerable resistance, and it is unclear whether they will ultimately be
    adopted. The Fund and the Bank have made progress in their crisis
    prevention efforts by performing assessments of member countries*
    financial sectors and adherence to standards. However, the effectiveness
    of these crisis prevention efforts is hindered by (1) private sector
    participants* limited use of published assessments, which they find
    untimely, outdated, and too dense to be useful and (2) gaps in crucial
    information about crisis

    vulnerabilities in some important emerging market countries due to
    voluntary participation in the assessments. These limitations prevent
    multilateral institutions, national policy makers, and private sector
    participants from making sound decisions, thus reducing the likelihood
    that these reforms will help prevent crises.

    Recommendations for Executive Action

    To help strengthen the Fund*s crisis prevention initiatives we recommend
    that the Secretary of the Treasury instruct the U. S. Executive Director
    of the Fund to work with other Executive Board members to encourage the
    Fund to

     improve the timeliness of publication of Financial System Stability
    Assessments and Reports on the Observance of Standards and Codes;  expand
    the coverage, frequency, and publication of reports on member

    countries* progress on implementing assessment recommendations;  improve
    the assessment reports* readability, for example, by creating a

    standardized format in which to present executive summaries and key
    findings; and

     pursue strategies for increasing participation in the Financial Sector
    Assessment Program and all modules of the Reports on the Observance of
    Standards and Codes, including the possibility of making participation
    mandatory for all members of the IMF. Agency Comments and

    Our Evaluation We received written comments on this report from the
    Department of the

    Treasury, the International Monetary Fund, and the World Bank. These
    comments and GAO*s evaluation of them are reprinted in appendixes X, XI,
    and XII. The organizations also separately provided technical comments

    Page 32 GAO- 03- 734 International Financial Crises

    that GAO discussed with relevant officials and included in the text of the
    report where appropriate.

    Treasury agreed with the report*s recommendations. Treasury recognized
    that some important countries have not volunteered to participate in the
    FSAP and ROSC and that there should be a shorter turnaround between the
    completion of an assessment and its public release. Treasury also pointed
    out that the acceptance of collective action clauses in some recent bond
    offerings serves as an important signal to investors that official
    financing is limited and that they cannot expect to be protected from
    risks.

    The IMF broadly agreed with the report*s recommendations. However, the IMF
    stated that we mischaracterized the role of the WEO forecasts and EWS
    models in IMF crisis anticipation efforts by saying that they have a
    greater importance than is warranted. We disagree with this depiction. Our
    assessment examined all six components of the IMF*s vulnerability
    assessment framework, including the WEO and the EWS. As the only mature
    and quantifiable elements of the framework, our analysis focused more
    heavily on the track records of the WEO and EWS. The IMF also stated that
    its responsibility to maintain financial stability could make its
    predictions less accurate so as not to contribute to a crisis. The IMF*s
    comment not only validates our finding on the WEO*s weakness but also
    raises questions regarding the purpose and credibility of the WEO
    forecasts.

    The World Bank generally agreed with the report*s recommendations.
    However, the Bank expressed concern with the report*s suggestion that
    consideration be given to making participation in the FSAP and ROSC
    assessments mandatory. While we are not suggesting that the assessments

    should be made mandatory, the voluntary nature of the FSAP and ROSC has
    posed an obstacle to full participation by important emerging market
    countries.

    We are sending copies of this report to the Secretary of the Treasury, the
    International Monetary Fund, the World Bank, and interested congressional
    committees. We also will make copies available to other interested parties
    upon request. In addition, the report will be available at no charge on
    the GAO Web site at http:// www. gao. gov.

    Page 33 GAO- 03- 734 International Financial Crises

    If you or your staff have any questions regarding this report, please call
    me at (202) 512- 8979. Other GAO contacts and staff are acknowledged in
    appendix XIII.

    Joseph A. Christoff, Director International Affairs and Trade

    Page 34 GAO- 03- 734 International Financial Crises

    Appendix I

    Appendi xes Objectives, Scope, and Methodology Appendi x I

    The Chairman of the House Financial Services and the Vice Chairman of the
    Joint Economic Committee requested that we assess (1) the International
    Monetary Fund*s (IMF*s or the Fund*s) framework for anticipating financial
    crises, (2) the status of key IMF initiatives to prevent financial crises,
    and (3) new IMF proposals to resolve future financial crises. They
    requested that our review cover the period after the Mexican financial
    crisis of 1994* 95.

    To assess IMF*s framework for anticipating financial crises, we examined
    prior and new IMF mechanisms for anticipating crises. Our analysis focused
    on the World Economic Outlook (WEO) forecasts and the IMF*s Early Warning
    System (EWS) models that were the IMF*s primary forecasting tools prior to
    the implementation of the new vulnerability assessment framework in May
    2001. Data on the performance of the other four components of the
    framework were not made available to us because these elements are
    considered part of the staff level deliberative process, and are not
    provided to the Executive Board. We obtained near- term data

    from the WEO forecasts, including real gross domestic product (GDP) growth
    rate and current account balance for 87 emerging market countries for the
    period 1990* 2001 (see appendix III). We focused on the 81 middleincome
    countries and an additional 6 low- income countries listed by J. P. Morgan
    as emerging markets. To evaluate the WEO and program forecasts,

    we used standard econometric techniques based on methods commonly found in
    the forecasting literature. The formal methodology of our forecast
    evaluations was based on several expert publications, the replication of
    our summary statistics with another author*s results, and discussions with
    a forecasting expert. To describe the performance of the IMF*s EWS models
    in anticipating crises, we reviewed and summarized the results of IMF
    evaluations. We interviewed IMF staff, including country desk economists
    and staff from several departments, to discuss the IMF*s new framework for
    vulnerability assessment, the EWS models, and the WEO methodology. We also
    interviewed 23 major private sector emerging market participants

    to discuss whether and how they use IMF forecasts in their investment
    decisions.

    To assess the status of key IMF initiatives to prevent financial crises,
    we reviewed Fund and World Bank documents published between 1999 and 2003
    on the creation of the Financial Sector Assessment Program (FSAP) and
    Reports on the Observance of Standards and Codes (ROSC) and evaluations of
    progress in implementing these reforms. We interviewed senior Fund
    officials, including staff from the Monetary Affairs and Exchange
    Department, the Policy Development and Review Department,

    Appendix I Objectives, Scope, and Methodology

    Page 35 GAO- 03- 734 International Financial Crises

    and the Fiscal Affairs Department. We also met with senior advisers at the
    World Bank (the Bank) who oversee the Bank*s participation in the FSAP and
    ROSC initiatives. To gain a better understanding of how the Fund uses FSAP
    and ROSC assessments to inform the policy advice it provides to member
    country governments and challenges it faces in using these assessments, we
    interviewed Fund officials in nine area departments (Argentina, Brazil,
    Korea, Mexico, Peru, Poland, Russia, Turkey, and Uruguay). We also spoke
    with Fund officials in these area departments about member country
    governments* use of the assessments in shaping their reform agendas and
    the obstacles that member country authorities encounter in implementing
    the reforms recommended in the assessments. To assess the extent to which
    emerging market countries have participated in and authorized publication
    of FSAP and ROSC assessments, we examined Fund and World Bank data on
    country participation in the FSAP and 12 ROSC modules and publication of
    the resulting reports between May 1999 and March 2003. To obtain views on
    the private sector*s use of Fund and World Bank assessments, we conducted
    structured interviews with 13 representatives of private sector firms,
    including ratings agencies, investment banks, and pension funds. We
    focused on 33 countries (a subset of the 87 we analyzed in the previous
    section) identified as the major emerging market countries by J. P.
    Morgan. 1 To describe new proposals to resolve future financial crises and
    their

    potential challenges, we obtained the most current Fund documentation for
    the two key proposals, the Sovereign Debt Restructuring Mechanism (SDRM)
    and Collective Action Clauses (CACs). We examined the purpose, goals,
    requirements, and status of implementation. To obtain views on the private
    sector*s understanding of the components of the new proposals and
    potential implementation challenges, we conducted structured interviews
    with 22 representatives of private sector firms, including ratings
    agencies, investment banks, and pension funds. We also met with
    government, private sector emerging market participants, and
    nongovernmental officials at several conferences. We also interviewed
    Department of the Treasury officials and experts in international finance
    and law. The IMF did not meet with us on these proposals because they were
    still under negotiation at the time of our review.

    1 These 33 countries represent the most significant emerging market
    participants in the international capital markets, as identified by J. P.
    Morgan.

    Appendix I Objectives, Scope, and Methodology

    Page 36 GAO- 03- 734 International Financial Crises

    We conducted fieldwork in Washington, D. C., and in New York. We performed
    our work from May 2002 to May 2003 in accordance with generally accepted
    government auditing standards.

    Page 37 GAO- 03- 734 International Financial Crises

    Appendix II Private Sector Structured Interview Participants Appendi x I I

    Source: GAO.

    Perspectives obtained on IMF efforts in crisis Company Anticipation
    Prevention Resolution

    Citicorp x xx Cleary Gottlieb & Hamilton x x Credit Suisse x xx Darby*s
    Overseas Investments x x x Elliot Associates x x Emerging Market Traders
    Association x x eStandards Forum x x x Eurasia Group x x x Fitch Ratings x
    xx HBK x x HSBC x xx International Primary Markets Association x x
    Institute for International Finance x x Japan Center for International
    Finance x x J. P. Morgan x x Lehman Brothers x x x Mass Mutual x x Moody*s
    Investor Service x x x Securities Industry Association x x x Societe
    Generale x x x Standard & Poor*s x x x Straus & Boies x x Wilshire
    Associates x x

    Page 38 GAO- 03- 734 International Financial Crises

    Appendix III Assessment of IMF Forecasting Appendi x I I I

    Congress expressed concerns regarding the accuracy of the International
    Monetary Fund*s (IMF*s) growth rate projections and asked us to examine
    them. In response, we analyzed the quality of the forecasts produced by
    the World Economic Outlook (WEO), the Fund*s primary forecasting tool.
    Using econometric techniques common to forecast evaluation studies, we

    examined the basic measures of forecast accuracy, bias, and efficiency.
    This assessment supplements our finding on WEO*s efforts to anticipate
    crises reported earlier. We found that WEO forecasts of gross domestic
    product (GDP) growth and inflation perform somewhat better than an
    assumption that next year*s rate will be the same as this year*s (called a
    *naive* forecast). However, there is evidence of an optimistic bias in the
    forecasts of GDP growth and inflation. In addition, we found that the
    naive forecast of the current account generally performed better than the
    WEO forecast. Moreover, WEO forecasts for the major industrialized
    countries were superior to emerging market forecasts, and forecasts for
    emerging market countries that had been on an IMF program were better than
    for those countries that were not. The shortcomings we observed in WEO
    forecasting are similar to those encountered by other private sector and
    official forecasters.

    GAO Analysis Used Standard Econometric Techniques for Forecast Evaluation

    We evaluated IMF forecasts for 87 emerging market countries. Our analysis
    focused on WEO forecasts of GDP, inflation, and the current account. Our
    measures of forecast quality relied on generally accepted econometric
    measures of accuracy, bias, and efficiency.

    Overall Approach To evaluate the quality of IMF forecasts, we analyzed the
    near- term and year- ahead WEO forecasts for 87 middle- income emerging
    market

    countries. 1 Appendix IV lists the 87 emerging market countries used in
    the analysis. Our analysis focused on three WEO forecast variables: (1)
    the growth rate of real GDP, (2) consumer price index (CPI) inflation
    (average

    over period), and (3) current account balance in billions of U. S.
    dollars. Our evaluation methodology is based on standard econometric
    techniques commonly found in the forecast literature, including the work
    of

    1 Our analysis focused on 81 middle- income countries and an additional 6
    low- income countries listed by J. P. Morgan as emerging markets.

    Appendix III Assessment of IMF Forecasting

    Page 39 GAO- 03- 734 International Financial Crises

    forecasting experts such as Stekler (1991), Artis (1996), and Loungani
    (2001). 2 We also compared the quality of WEO forecasts of emerging market
    countries with WEO forecasts of the G- 7 countries and we compared

    borrowers of IMF resources with those that were not. 3 Our comparison with
    the G- 7 countries 4 allowed us to informally assess whether income level
    or data quality mattered in forecast quality. Our analysis of forecasts
    for program countries permitted us to assess whether WEO forecasts differ
    from forecasts contained within program documents, which are produced
    under conditions of greater staff scrutiny. 5 We also reviewed a number of
    forecast evaluation studies to see how our results compared to previous
    reviews and to contrast IMF forecast quality with other forecasting
    efforts.

    Basic Definitions and Methods of Evaluation

    Our analysis focused on the WEO*s near- term and year- ahead forecasts.
    Near- term forecasts originate from the May WEO for each year, and they
    project for the remainder of the existing year (approximately 6 months
    ahead). The year- ahead forecasts come from the October WEO of the
    preceding year. Thus, a near- term forecast for 2000 would come from the
    May 2000 WEO, and a year- ahead forecast for 2000 would come from the
    October 1999 WEO. We compared these WEO forecasts to the *first settled
    estimate,* which comes from the October WEO of the following year for

    which the forecast is made. Thus, we compared both the near- term and 2
    Herman Stekler, *Macroeconomic Forecast Evaluation Techniques,*
    International Journal of Forecasting 7 (1991): 375- 384; Michael J. Artis,
    *How Accurate are the IMF*s Short- Term Forecasts? Another Examination of
    the World Economic Outlook,* WP/ 96/ 89 (Washington, D. C.: International
    Monetary Fund [IMF], August 1996); Prakash Loungani, *How Accurate are
    Private Sector Forecasts? Cross- Country Evidence from Consensus Forecasts
    of Output Growth,* International Journal of Forecasting 17 (2001): 419-
    432.

    3 We include the following General Resources Account- funded IMF programs:
    Stand- By Arrangements (SBA), Extended Fund Facility (EFF), Systemic
    Transformation Facility (STF), Compensatory Contingency Finance Facility
    (CCFF). Under the CCFF, countries can borrow resources on a stand- alone
    basis, i. e., outside of an IMF program. Macedonia and South Africa
    borrowed funds under the CCFF.

    4 The G- 7 include Canada, France, Germany, Italy, Japan, United Kingdom,
    and United States. 5 Of the 87 emerging market countries analyzed, 57 were
    on an IMF program for some part of the 1990* 2001 period.

    Appendix III Assessment of IMF Forecasting

    Page 40 GAO- 03- 734 International Financial Crises

    year- ahead forecasts for 2000 with the *first settled estimate* from the
    October 2001 WEO. 6 Most of the econometric tools we used to assess the
    quality of WEO

    forecasts analyze the errors deriving from the forecasts. Our econometric
    tools examined these errors for certain qualities and patterns. We defined
    the forecast error, as the difference between the forecasted, and
    realized, value of an indicator. Hence, we have

    Our examination of the errors in the Fund*s forecasts focused on three
    measures of *goodness*: accuracy, bias, and efficiency. We performed these
    tests separately for the 87 countries over the 11- year forecasting
    period.

    Accuracy The credibility of a forecast is established by its accuracy. The
    ultimate test of any forecast is whether it can predict future events
    accurately. Accuracy assesses whether forecasts tend, by some standard, to
    be close to actual

    outcomes. Although there is no objective standard of accuracy, comparisons
    to alternative forecasting methods, such as a naive model that uses
    historical trend data, is one way to judge relative accuracy. The accuracy
    measure we used is Theil*s U- statistic (U 1 ), based on the naive model
    that assumes this year*s growth rate will be the same as last year*s. 7

    The Theil U- statistic is based on an examination of the forecast*s root
    mean square error (RMSE). To compute RMSE, the forecast errors are squared
    and averaged over the sample to get the mean square error (MSE). RMSE is
    the square root of MSE.

    6 A persistent issue in the forecasting literature concerns which *actual*
    value to use to evaluate the accuracy of the forecasts: the first
    available estimate (available in May of the following year), the first
    settled estimate (available in October of the following year), or a later
    revision. We have taken the middle ground, as suggested by Loungani
    (2001), on the basis that forecasters are not attempting to predict later
    revisions, which incorporate information such as revisions of weights and
    changes in methods of construction that forecasters would not have been
    aware of at the time of the forecast.

    , t e , t f

    , t a

    7 That is, the naive model assumes there is no change in the growth rate
    between t- 1 and t.

    Appendix III Assessment of IMF Forecasting

    Page 41 GAO- 03- 734 International Financial Crises

    The Theil U- statistic allows us to scale the RMSE by the variability of
    the underlying data. The standard Theil statistic, commonly denoted as ,
    compares the RMSE of forecast series to the RMSE of the actual series,

    where = - . A statistic greater than one denotes that the naive model
    performed better than the model being tested.

    Bias Bias determines whether forecast errors in one direction tend to be
    larger and/ or more numerous than errors in the opposite direction.
    Forecast errors can be divided into two parts. One part is the *random
    error,* which varies unsystematically, or randomly, from one forecast to
    the next. The other part is the *bias error,* which remains constant for
    any particular forecasting procedure. Bias happens when factors other than
    random events influence the forecast results, resulting in an upward or
    downward tendency. An unbiased forecast means that forecast errors are

    approximately zero on average over time. However, an unbiased forecast
    does not guarantee that a forecast will be accurate enough to be useful if
    the errors are large.

    To assess the extent of bias in the forecasts, we regress the forecast
    error on a constant term and then carry out a t- test for the coefficient.
    For a time series of forecasts (1990* 2001 in our analysis), we have the
    set of forecast errors . The regression test for bias involves estimating
    the intercept coefficient, , for the simple regression

    1 U t a t a 1 t a 1 U

    { } N e e e , , , 2 1 K

    0

    Appendix III Assessment of IMF Forecasting

    Page 42 GAO- 03- 734 International Financial Crises

    This reflects a partition of the forecast error into an estimate of the
    systematic component and a random component ( ). A t- test with degrees of
    freedom is then performed to test versus the twosided

    alternative We perform a t- test to determine whether the average bias is
    significantly different from zero. If the p- value is less than .05, we
    reject the hypothesis that there is no bias at the 95 percent level of
    significance. This means that there is less than a 5 percent chance that
    we are making a false rejection, that is saying the forecast is biased
    when it is not. A determination that a forecast is unbiased is a
    necessary,

    although not sufficient, condition for concluding that a forecast is
    efficient. Efficiency Efficiency examines whether a forecast has taken
    into account all available

    information. Establishing that a forecast is efficient means that no other
    model or readily available information would be able to improve the
    forecast, and there is no way to predict the direction or size of the
    errors. A test of efficiency makes use of the simple linear model where we
    regress the actual outcome on the forecast

    If the forecast is efficient in predicting the actual outcome, then the
    intercept, , should be equal to 0 and the slope, , should be equal to 1. 8
    Using the regression model defined above, we perform a joint hypothesis

    test to check whether these conditions hold simultaneously.

    t 1 N 0 : 0 0 = H

    . 0 : 0 A H

    8 A zero intercept implies that the errors are randomly distributed; they
    vary unsystematically (unbiased). However, a forecast can be unbiased and
    not efficient. A slope of 1 implies a straight line through the zero
    intercept, denoting efficiency. A slope of 1 indicates that the forecast
    and actual value essentially coincide.

    0 1

    Appendix III Assessment of IMF Forecasting

    Page 43 GAO- 03- 734 International Financial Crises

    The algebraically simplified version of the test statistic for this
    hypothesis is shown below. 9

    The reference distribution for this statistic is an F- distribution with 2
    and degrees of freedom. If the p- value for this statistic is less than
    .05, then we reject the hypothesis at the 95 percent level of significance
    that both the intercept is zero and the slope is one. This means that
    there is only a 5 percent chance that we are making a false rejection,
    that is saying the forecast is not efficient when it is.

    WEO Forecasts Demonstrate Some Accuracy, but Also Optimistic Bias

    Our analysis of the WEO forecasts for 87 emerging market countries shows
    that WEO forecasts perform somewhat better than a naive model for GDP
    growth and inflation, but not for current account (see table 2).

    Table 2: Forecast Quality for 87 Emerging Market Countries, 1990* 2001

    Source: GAO analysis of IMF data. a The percentage of countries in which
    the WEO forecast does a better job than the naive forecast

    (based on the Theil statistic). b The percentage of countries in which
    there is statistically significant bias (at the 5 percent level).

    c When bias occurs in 15 percent or more countries and the forecast errors
    tend to vary predominantly in one direction (more than 70 percent), then
    we indicate the direction of this bias.

    9 RSS is the residual sum of squares. 2 N

    Accuracy a (percent) Bias b

    (percent) Direction of bias c Efficiency d (percent) Forecast variable
    Yearahead Nearterm

    Yearahead Nearterm Yearahead Nearterm Yearahead Nearterm

    GDP growth rate 62 74 21 15 Upward Upward 76 85 Inflation 55 70 21 13
    Downward Downward e 72 67 Current account (billions of U. S. $) 24 56 8 8
    Upward e Indeterminate 67 79

    Appendix III Assessment of IMF Forecasting

    Page 44 GAO- 03- 734 International Financial Crises

    d Percentage of countries where we fail to reject the joint hypothesis
    that the intercept is zero and the slope is one; i. e., the evidence is
    not strong enough to reject the assumption of efficiency. e Italics
    indicate that there are too few countries to consider bias a systematic
    trend.

    We found the year- ahead WEO forecast does a better job than the naive
    forecast in more than 60 percent of the countries with GDP, in more than
    half the countries with inflation, and in about one- quarter of the
    countries with the current account. However, even for GDP, nearly 40
    percent of the country forecasts were no better than an assumption that
    next year*s value is the same as this year*s. For all three variables, the
    shorter the forecast period, the more accurate the forecast. When the
    forecast time horizon shortens from 1 year to 6 months, the percentage of
    cases in which the WEO does a better job forecasting than the naive model
    increases for all three variables, exceeding 50 percent for the current
    account.

    WEO forecasts for GDP and inflation demonstrated bias in about 20 percent
    of the country cases. The direction of the bias was upward for GDP and
    downward for inflation, indicating an optimistic tendency within the WEO
    forecasting process. 10 Although the bias was upward for the current
    account, also consistent with optimism, it occurred in only 8 percent of
    country forecasts. 11 For all three variables, we could not reject the
    hypothesis that WEO forecasts were efficient for at least two- thirds of
    the country*s forecasts. However, in about one- fourth of the country
    cases, the WEO forecast could have been improved through the use of a
    different model or the addition of new information.

    Forecasts for Major Industrialized Countries Are Better than Emerging
    Market Forecasts and Program Forecasts Are Better than Nonprogram
    Forecasts

    WEO forecasts of the most developed countries are superior to its
    forecasts of emerging market countries when compared to the naive model
    forecasts. The improved forecast quality is likely due to better available
    data and greater stability of the wealthiest economies. Similarly, WEO
    forecasts for countries that borrow from the IMF are superior to those
    that do not. The increased scrutiny of borrowing countries by IMF staff
    likely

    contributes to the improved forecasts. 10 Our finding of an optimistic
    (upward) bias for GDP forecasts is consistent with, and helps explain, our
    finding in the main report of the WEO*s difficulty in forecasting
    recessions. 11 An optimistic bias in the current account means that the
    forecast was for a smaller current deficit or a larger surplus than
    occurred.

    Appendix III Assessment of IMF Forecasting

    Page 45 GAO- 03- 734 International Financial Crises

    WEO Forecasts for Major Industrialized Countries Are Better than Those for
    Emerging Markets

    WEO forecasts of GDP and inflation for the G- 7 countries are considerably
    better than its forecasts for the emerging market countries when compared
    to the naive model forecasts. (see table 3).

    Table 3: Forecast Quality for G- 7 Countries, 1990- 2001

    Legend N/ A = not applicable Source: GAO analysis of IMF data.

    Note: Number of countries, and not percentages, are reported given the
    small number of cases. a The number of countries for which the WEO
    forecast does a better job than the naive forecast (based

    on the Theil statistic). b The number of countries in which there is
    statistically significant bias (at the 5 percent level).

    c The number of countries where we fail to reject the joint hypothesis
    that the intercept is zero and the slope is one, i. e., the evidence is
    not strong enough to reject the assumption of efficiency. d Italics
    indicate that there are too few countries to consider bias a systematic
    trend.

    This improvement is evident across the full range of analyses. For
    example, in six out of seven countries, the year- ahead WEO forecasts of
    GDP and inflation for the G- 7 countries were found to be accurate, and
    the near- term forecasts for GDP and inflation were accurate for all of
    the G- 7 countries. These results are considerably better than WEO
    forecasts for emerging markets. In the year- ahead forecasts, bias and
    efficiency were a concern in

    two forecasts of GDP, and one current account forecast, but they were not
    a concern in the inflation forecasts. Although the year- ahead forecast
    for current account was inaccurate for six of seven countries, the near-
    term forecast was accurate for five of the G- 7 country cases. The
    improved quality of WEO forecasts for the G- 7 countries is likely due to
    better available data and greater stability of these economies, compared
    to emerging market countries.

    Accuracy a (number of countries)

    Bias b (number of

    countries) Direction of bias (number of countries) Efficiency c

    (number of countries) Forecast variable Yearahead Nearterm

    Yearahead Nearterm Yearahead Nearterm Yearahead Nearterm

    GDP growth rate 6 7 2 0 Upward d N/ A 5 7 Inflation 6 7 0 0 N/ A N/ A 7 7
    Current account (billions of U. S. $) 1 510Downward d N/ A 5 7

    Appendix III Assessment of IMF Forecasting

    Page 46 GAO- 03- 734 International Financial Crises

    WEO Forecasts of Countries on an IMF Program Are Superior to Nonprogram
    Country Forecasts

    We found that WEO forecasts for 57 countries that were on an IMF program
    12 (or that borrowed Fund resources under the CCFF) for any part of the
    forecast period tend to be more accurate than WEO forecasts for the 30
    countries that were never on an IMF program during this period (see table
    4).

    Table 4: Comparison of Year- Ahead WEO Forecasts for Program and
    Nonprogram Countries, 1990* 2001

    Legend N/ A = not applicable Source: GAO analysis of IMF data.

    a For pooled countries, if the U statistic is greater than 1, the naive
    forecast does a better job than the forecast under evaluation. b Bias at
    the 5 percent level of significance means that there is less than a 5
    percent chance that we are

    making a false rejection* i. e., saying the forecast is biased when it is
    not.

    Countries that borrow from the IMF are likely to be under greater scrutiny
    from Fund staff than those that do not borrow, which could contribute to
    an improved forecast. For this analysis, we compared the Theil statistics
    for GDP growth, inflation, and current account for the two pooled
    forecasts. We found that the WEO forecasts of GDP and inflation for
    program countries are more accurate than those for nonprogram countries.
    That is, when compared to the naive model, the program countries have a
    lower Theil statistic than nonprogram countries. For both groups, the
    forecast of current account is inferior to the naive forecast (a Theil
    statistic greater than 1). The WEO program countries forecasts for GDP and
    inflation are biased, whereas the forecasts for the nonprogram countries
    were not. This indicates that by assuming implementation of the policies

    12 We include countries that were on the following General Resources
    Account- funded IMF programs and facilities: SBA, EFF, STF, CCFF. Some
    countries were on more than one program during the forecast period.

    Accuracy Theil U- statistic a Bias

    5- percent significance level b Direction of bias Forecast variable 57
    program

    countries 30 Nonprogram countries 57 program

    countries 30 Nonprogram countries 57 program

    countries 30 Nonprogram countries

    GDP growth rate .827 .941 Ye s No Up wa r d N/ A Inflation .776 .918 Yes
    No Downward N/ A Current account (billions of U. S. $) 2.399 1. 129 No Yes
    N/ A Downward

    Appendix III Assessment of IMF Forecasting

    Page 47 GAO- 03- 734 International Financial Crises

    contained within the program, the Fund expects that GDP and inflation will
    perform better than they actually do. In addition to the publicly
    available WEO forecasts for all countries, the

    IMF also produces a set of program forecasts for countries in the years
    they borrow from the Fund. 13 According to the Fund, these two forecasts
    should be very similar since they are prepared by the same staff in the
    same manner. Our comparison of program 14 and WEO forecasts for the
    initial year that each country was on program confirmed that the accuracy
    of the two forecasts for GDP and inflation were nearly identical (see
    table 5).

    Table 5: Comparison of Program and WEO Year- Ahead Forecasts, for 52
    Countries in the Initial Years on Program, 1990* 2001 a

    Legend N/ A = not applicable Source: GAO analysis of IMF data.

    Note: This analysis is based on 96 country/ year observations, which is an
    average across three variables. a We were able to obtain program
    projections for only 52 of the 57 program countries.

    b If the U statistic is greater than 1, the naive forecast does a better
    job than the forecast under evaluation. c Bias at the 5 percent level of
    significance means that there is less than a 5 percent chance that we are

    making a false rejection* i. e., saying the forecast is biased when it is
    not. d The forecast is biased at the 10 percent level of significance
    means that there is less than a 10

    percent chance that we are making a false rejection* i. e., saying the
    forecast is biased when it is not.

    13 For most of the years studied, these forecasts were not made public. In
    recent years, when countries agree, these forecasts are made public. 14 We
    used the original program numbers, that is, the projections established at
    the outset when the IMF*s Executive Board first approves an arrangement.

    Accuracy Theil U- statistic b Bias

    5 percent significance level c Direction of bias Forecast variable Program

    forecasts WEO forecasts Program

    forecasts WEO forecasts Program

    forecasts WEO forecasts

    GDP growth rate .689 .676 Yes Yes Upward Upward Inflation .683 .694 No d
    No d N/ A N/ A Current account (billions of U. S. $) .524 2.292 No No N/ A
    N/ A

    Appendix III Assessment of IMF Forecasting

    Page 48 GAO- 03- 734 International Financial Crises

    However, program forecasts of current account are substantially better
    than those reported in the WEO. In addition, in all cases the program
    forecasts were substantially more accurate than the naive model. This is
    further evidence that the greater scrutiny experienced by these countries
    while under a program probably contributes to an improved forecast. Other
    Forecasting Efforts

    Have Difficulties Similar to the Fund

    Our review of other forecast evaluations found that the shortcomings we
    observed in WEO forecasting are similar to difficulties encountered by
    other forecasters. 15 These studies examined forecasts 16 produced by the

    private sector (for example, consensus forecasts), governments, and
    multinational agencies including the IMF and Organization for Economic
    Cooperation and Development. These studies, similar to our observation in
    this report, found a general inability to predict recessions. In addition,
    consistent with our results, these studies found that (1) the shorter the
    time horizon, the more accurate the forecasts; (2) current account
    forecasts are

    15 Harjit K. Arora & David J. Smyth, *Forecasting the Developing World: An
    Accuracy Analysis of the IMF*s Forecasts,* International Journal of
    Forecasting 6 (1990): 393- 400; Michael J. Artis, *How Accurate are the
    IMF*s Short- Term Forecasts? Another Examination of the World Economic
    Outlook,* WP/ 96/ 89 (Washington, D. C.: International Monetary Fund
    [IMF], August 1996); Jose M. Barrionuevo, *How Accurate Are the World
    Economic Outlook

    Projections?* in World Economic Outlook, ch. 2, Staff Studies for World
    Economic Outlook (Washington, D. C.: IMF, 1993); Roy Batchelor, *The IMF
    and OECD versus Consensus Forecasts* (London, England: City University
    Business School, August 2000); William W. Beach, Aaron B. Schavey, &
    Isabel M. Isidro, *How Reliable Are IMF Economic Forecasts?* CDA 99- 05
    (Washington, D. C.: The Heritage Foundation, The Heritage Center for Data
    Analysis, August 27, 1999); David Fintzen & H. O. Stekler, *Why Did
    Forecasters Fail to Predict the 1990 Recession?* International Journal of
    Forecasting 15 (1999): 309- 323; IMF, *The Accuracy of World Economic
    Outlook Growth Forecasts: 1991* 2000,* in World Economic Outlook, Box 3.1
    (Washington, D. C.: IMF, December 2001): 37- 39, and IMF, *How Well Do
    Forecasters Predict Turning Points?* in World Economic Outlook, Box 1.1

    (Washington, D. C.: IMF, May 2002): 6- 8; Grace Juhn & Prakash Loungani,
    *Further CrossCountry Evidence on the Accuracy of the Private Sector*s
    Output Forecasts,* IMF Staff Papers 49, no. 1 (2002); Prakash Loungani,
    *How Accurate are Private Sector Forecasts? Cross- Country Evidence from
    Consensus Forecasts of Output Growth,* International Journal of
    Forecasting 17 (2001): 419- 432; Albert Musso & Steven Phillips,
    *Comparing Projections and Outcomes of IMF- Supported Programs,* IMF Staff
    Papers 49, no. 1 (2002); Scott Shuh, *An Evaluation of Recent
    Macroeconomic Forecast Errors,* New England Economic Review (January/
    February 2001); Marten Blix et al., *How Good is the Forecasting
    Performance of Major Institutions?* Economic Review (Stockholm, Sweden:
    Sveriges Riksbank Monetary Policy Department, March 2001): 38- 68; and
    Victor Zarnowitz,

    *The Record and Improvability of Economic Forecasting,* NBER Working Paper
    No. 2099 (December 1986). 16 Forecasters evaluated the accuracy of many
    variables, including output (GDP), inflation, current account, exports,
    and imports, among others.

    Appendix III Assessment of IMF Forecasting

    Page 49 GAO- 03- 734 International Financial Crises

    markedly weaker than the forecasts for GDP and inflation; (3) when bias is
    found, forecasts tend to overestimate GDP and underestimate inflation; and
    (4) GDP and inflation forecasts for the industrial countries tend to be

    more accurate and less biased than forecasts for developing countries.
    While several studies found that WEO forecasts for developing countries
    were inferior to those generated by the naive model, one study found that
    WEO forecasts for developing countries did notably better than a naive
    forecast. A number of studies compared the quality of WEO forecasts with
    those produced by the private sector. Although some studies found the

    relative quality of the forecasts to be generally the same, a few studies
    found WEO forecasts to be less accurate than those of the private sector.

    Page 50 GAO- 03- 734 International Financial Crises

    Appendix IV Eighty- seven Emerging Market Countries Appendi x I V

    Country Region a Income group b

    1 Albania Countries in transition Lower middle income 2 Algeria Africa
    Lower middle income 3 Antigua and Barbuda Western Hemisphere Upper middle
    income 4 Argentina Western Hemisphere Upper middle income 5 Bahrain Middle
    East and Turkey Upper middle income 6 Belarus Countries in transition
    Lower middle income 7 Belize Western Hemisphere Lower middle income 8
    Bolivia Western Hemisphere Lower middle income 9 Botswana Africa Upper
    middle income 10 Brazil Western Hemisphere Upper middle income 11 Bulgaria
    Countries in transition Lower middle income 12 Cape Verde Africa Lower
    middle income 13 Chile Western Hemisphere Lower middle income 14 China
    Developing Asia Lower middle income 15 Colombia Western Hemisphere Lower
    middle income 16 Costa Rica Western Hemisphere Upper middle income 17 Cote
    d*Ivoire Africa Low income 18 Croatia Countries in transition Upper middle
    income 19 Czech Republic Countries in transition Upper middle income 20
    Djibouti Africa Lower middle income 21 Dominica Western Hemisphere Upper
    middle income 22 Dominican Republic Western Hemisphere Lower middle income
    23 Ecuador Western Hemisphere Lower middle income 24 Egypt Middle East and
    Turkey Lower middle income 25 El Salvador Western Hemisphere Lower middle
    income 26 Equatorial Guinea Africa Lower middle income 27 Estonia
    Countries in transition Upper middle income 28 Fiji Developing Asia Lower
    middle income 29 Gabon Africa Upper middle income 30 Grenada Western
    Hemisphere Upper middle income 31 Guatemala Western Hemisphere Lower
    middle income 32 Guyana Western Hemisphere Lower middle income 33 Honduras
    Western Hemisphere Lower middle income 34 Hungary Countries in transition
    Upper middle income 35 India Developing Asia Low income

    Appendix IV Eighty- seven Emerging Market Countries

    Page 51 GAO- 03- 734 International Financial Crises

    36 Indonesia Developing Asia Low income 37 Iran, Islamic Rep. of Middle
    East and Turkey Lower middle income 38 Iraq Middle East and Turkey Lower
    middle income 39 Jamaica Western Hemisphere Lower middle income 40 Jordan
    Middle East and Turkey Lower middle income 41 Kazakhstan Countries in
    transition Lower middle income 42 Kiribati Developing Asia Lower middle
    income 43 Korea Developing Asia Upper middle income 44 Latvia Countries in
    transition Lower middle income 45 Lebanon Middle East and Turkey Upper
    middle income 46 Libya Middle East and Turkey Upper middle income 47
    Lithuania Countries in transition Lower middle income 48 Macedonia, former
    Yugoslav Rep. of Countries in transition Lower middle income 49 Malaysia
    Developing Asia Upper middle income 50 Maldives Developing Asia Lower
    middle income 51 Mauritius Africa Upper middle income 52 Mexico Western
    Hemisphere Upper middle income 53 Morocco Africa Lower middle income 54
    Namibia Africa Lower middle income 55 Nigeria Africa Low income 56 Oman
    Middle East and Turkey Upper middle income 57 Pakistan Developing Asia Low
    income 58 Panama Western Hemisphere Upper middle income 59 Papua New
    Guinea Developing Asia Lower middle income 60 Paraguay Western Hemisphere
    Lower middle income 61 Peru Western Hemisphere Lower middle income 62
    Philippines Developing Asia Lower middle income 63 Poland Countries in
    transition Upper middle income 64 Romania Countries in transition Lower
    middle income 65 Russia Countries in transition Lower middle income 66
    Samoa Developing Asia Lower middle income 67 Saudi Arabia Middle East and
    Turkey Upper middle income 68 Seychelles Africa Upper middle income 69
    Slovak Republic Countries in transition Upper middle income 70 South
    Africa Africa Upper middle income 71 Sri Lanka Developing Asia Lower
    middle income 72 St. Kitts and Nevis Western Hemisphere Upper middle
    income

    (Continued From Previous Page)

    Country Region a Income group b

    Appendix IV Eighty- seven Emerging Market Countries

    Page 52 GAO- 03- 734 International Financial Crises

    Sources: IMF, World Bank, and J. P. Morgan. a Based on WEO regional
    groupings, IMF 2002.

    b World Bank analytical classification by income, July 2001. 73 St. Lucia
    Western Hemisphere Upper middle income

    74 St. Vincent and the Grenadines Western Hemisphere Lower middle income
    75 Suriname Western Hemisphere Lower middle income 76 Swaziland Africa
    Lower middle income 77 Syrian Arab Rep. Middle East and Turkey Lower
    middle income 78 Thailand Developing Asia Lower middle income 79 Tonga
    Developing Asia Lower middle income 80 Trinidad and Tobago Western
    Hemisphere Upper middle income 81 Tunisia Africa Lower middle income 82
    Turkey Middle East and Turkey Upper middle income 83 Turkmenistan
    Countries in transition Lower middle income 84 Ukraine Countries in
    transition Low income 85 Uruguay Western Hemisphere Upper middle income 86
    Vanuatu Developing Asia Lower middle income 87 Venezuela Western
    Hemisphere Upper middle income

    (Continued From Previous Page)

    Country Region a Income group b

    Page 53 GAO- 03- 734 International Financial Crises

    Appendix V Standards, Codes, and Principles Assessed under IMF and World
    Bank Reports on the Observance of Standards and Codes Appendi x V

    Assessment start date Responsible institution Standard or code and
    rationale for adoption

    Transparency standards: The standards on transparency in government
    operations and policy making are considered within the Fund*s direct
    operational focus.

    1996* 97 IMF IMF Special Data Dissemination Standard (SDDS) and General
    Data Dissemination Standard (GDDS). The purpose of the IMF*s SDDS is to
    guide member country governments that have, or might seek, access to
    international capital markets in publishing comprehensive, timely,
    accessible, and reliable economic and financial statistics. The purpose of
    the

    GDDS is to help any member country government provide more reliable
    economic data. The SDDS and GDDS were created in 1996 and 1997,
    respectively.

    1998 (modified 2001) IMF IMF Code of Good Practices on Fiscal
    Transparency.

    This Code is intended to help member country governments improve the
    disclosure of information about the design and results of fiscal policy,
    making governments more accountable for policy implementation and
    strengthening credibility and public understanding of macroeconomic
    policies and choices. 1999 IMF IMF Code of Good Practices on Transparency
    in Monetary and Financial Policies.

    This Code is designed to increase the effectiveness of monetary and
    financial policies by raising public awareness of the government*s policy
    goals and instruments and making governments (especially independent
    central banks and financial agencies) more accountable. Financial sector
    standards: The financial sector standards are considered within the direct
    operational focus of both the Fund and the World Bank and are generally
    assessed under the joint Fund- Bank FSAP. 1999 Joint IMF- Bank Basel
    Committee*s Core Principles for Effective Banking

    Supervision (BCP). The BCP is intended to guide the development of an
    effective system for supervising banks, a large sector of most economies.
    The IMF and World Bank began assessments of countries*

    compliance with the BCP standard in conjunction with the Financial Sector
    Assessment Program (FSAP) launched in 1999.

    1999 Joint IMF- Bank International Organization of Securities Commissions*
    (IOSCO)

    Objectives and Principles for Securities Regulation.

    The IOSCO Objectives and Principles are designed to help governments
    establish effective systems to regulate securities which contribute
    strongly to investor confidence. The IMF and

    World Bank began using them to assess securities regulation in conjunction
    with the FSAP, launched in 1999.

    Appendix V Standards, Codes, and Principles Assessed under IMF and World
    Bank Reports on the Observance of Standards and Codes

    Page 54 GAO- 03- 734 International Financial Crises

    1999 (modified 2000) Joint IMF- Bank International Association of
    Insurance Supervisors* (IAIS)

    Insurance Core Principles.

    The IAIS Core Principles are designed to contribute to effective insurance
    supervision that supports financial stability. The IMF and World Bank
    began assessing member countries* regulatory practices in this area in
    conjunction with the FSAP, launched in 1999.

    1999 Joint IMF- Bank Committee on Payments and Settlements Systems* (CPSS)
    Core Principles for Systemically Important Payment Systems.

    The CPSS Core Principles are intended to strengthen payments systems,
    which provide the channels through which funds are transferred among banks
    and other institutions. The IMF and the World Bank began assessing member
    countries* observance of this standard in conjunction with the FSAP,
    launched in 1999.

    2002 Joint IMF- Bank Financial Action Task Force (FATF) 40 Recommendations
    on AntiMoney Laundering and 8 Special Recommendations on Terrorism
    Financing.

    The FATF*s 40 Recommendations and 8 Special Recommendations are intended
    to promote policies that combat money laundering and terrorist financing,
    which threaten financial system integrity and may undermine the sound
    functioning of financial systems. In 2002, the IMF and World Bank agreed
    to perform anti- money laundering and terrorist financing assessments as a
    12- month pilot program, generally in conjunction with the FSAP.

    Corporate sector standards: The corporate sector standards are considered
    important for the effective operation of domestic and international
    financial systems and are assessed by the World Bank.

    2000 World Bank Organization for Economic Cooperation and Development*s
    (OECD) Principles of Corporate Governance.

    The OECD developed its corporate governance principles to help governments
    evaluate and improve their legal, institutional, and regulatory frameworks
    for corporate governance. The World Bank developed a template for
    assessing adherence to corporate governance principles based on the OECD*s
    Principles established in 1999.

    (Continued From Previous Page)

    Assessment start date Responsible institution Standard or code and
    rationale for adoption

    Appendix V Standards, Codes, and Principles Assessed under IMF and World
    Bank Reports on the Observance of Standards and Codes

    Page 55 GAO- 03- 734 International Financial Crises

    Sources: IMF and World Bank. 2000 World Bank International Accounting
    Standards Board*s (IASB) International Accounting Standards.

    The ROSC*s accounting module is intended to compare member countries*
    corporate accounting practices with international accounting standards and
    to analyze actual accounting practice to determine the extent of
    compliance with applicable standards. There is special focus on the
    strengths and weaknesses of the institutional framework for supporting
    high quality accounting and financial reporting. In 2000, the World Bank
    developed a template for assessing adherence to accounting standards based
    on the IASB Standards. 2000 World Bank International Federation of
    Accountants* (IFAC) International

    Standards on Auditing.

    The ROSC*s auditing module compares member countries* auditing standards
    and auditors* professional code of ethics with the standards and codes
    issued by IFAC. Also, the quality of actual auditing practices is
    evaluated. There is special focus on the

    strengths and weaknesses of the institutional framework for supporting
    high quality audit. In 2000, the World Bank developed a template for
    assessing adherence to auditing standards based on the IFAC*s Standards.
    2001 (draft) World Bank World Bank Principles and Guidelines for Effective
    Insolvency and Creditor Rights Systems.

    In 2001, the World Bank developed draft Principles and Guidelines intended
    to help countries develop effective insolvency and creditor rights*
    systems, two important components of financial system stability. The World
    Bank has conducted several assessments based on its draft Principles and
    Guidelines. The United Nations Commission on International Trade Law
    (UNCITRAL) is completing a draft Legislative Guide on Insolvency Law.
    UNCITRAL, Bank,

    and IMF staff are working toward a single standard.

    (Continued From Previous Page)

    Assessment start date Responsible institution Standard or code and
    rationale for adoption

    Page 56 GAO- 03- 734 International Financial Crises

    Appendix VI Update on the International Monetary Fund*s Safeguards
    Assessments Appendi x VI

    In response to allegations of misreporting and misuse of International
    Monetary Fund (IMF or the Fund) disbursements in the late 1990s, the Fund
    increased its efforts to protect its resources by introducing safeguards
    assessments, a process for evaluating the controls employed by the central
    banks of borrowing member countries and for recommending measures to
    address inadequacies. Safeguards assessments have detected numerous
    inadequacies that could lead to misuse of Fund resources and have
    recommended measures to remedy them.

    The Fund Introduces Safeguards Assessments for Member Countries That
    Currently Borrow

    In 2000, the Fund introduced safeguards assessments, a process for
    identifying inadequacies in central banks* ability to ensure the integrity
    of their operations, especially the use of Fund resources. Safeguards
    assessments evaluate central banks* internal and external audit
    mechanisms, legal structure and independence, financial reporting
    procedures, and systems of internal controls.

    In April 2002, the Fund*s Executive Board made safeguards assessments a
    permanent policy. Safeguards assessments apply to all member countries
    with current or anticipated borrowing arrangements with the Fund.
    Countries with borrowing arrangements approved after June 30, 2000, are
    subject to a full safeguards assessment covering the five areas listed
    above. Countries with arrangements in effect before June 30, 2000, that
    have not

    yet repaid all Fund resources, were subject to a partial assessment
    covering only the external audit mechanism. Countries that do not have
    borrowing arrangements or have already repaid all Fund resources are not
    subject to safeguards assessments. According to Fund officials, since 2000
    the IMF has not provided financial resources to countries that did not
    meet its safeguards requirements. As of December 2002, the Fund had
    performed 37 full safeguards assessments and 27 partial assessments, with
    23 assessments under way. 1 The completed assessments detected a number of
    serious vulnerabilities

    that could lead to misuse of central bank resources, including those
    borrowed from the Fund. Of the full safeguards assessments, the Fund found
    the following:

    1 International Monetary Fund, Safeguards Assessments * Semi- Annual
    Update, SM/ 03/ 88 (Washington, D. C.: Mar. 6, 2003).

    Appendix VI Update on the International Monetary Fund*s Safeguards
    Assessments

    Page 57 GAO- 03- 734 International Financial Crises

     Inadequate accounting standards in 82 percent of the central banks,
    which interfere with the accurate recording of central bank operations.
    For example, some central banks did not adhere to a financial reporting
    framework such as the International Accounting Standards (IAS), which
    would help prevent misreporting of transactions.

     Deficient internal audit in 79 percent of central banks, which reduces
    their ability to address risks of misuse and misreporting of Fund
    resources. For example, some internal audit departments did not audit
    high- risk areas such as foreign reserves management.

     Poor controls over foreign reserves and data reporting to the Fund in 49
    percent of the central banks, increasing the possibility of misreporting
    and misuse of Fund resources. For example, safeguards assessments
    identified improper techniques for valuing foreign reserves and failure to
    reconcile data reported to the Fund for program monitoring purposes with
    underlying accounting records.

    According to Fund officials, when IMF staff detect significant weaknesses
    in the controls of assessed central banks, they recommend that the
    government take corrective actions. For actions that IMF staff consider
    essential, they may incorporate the recommendations into the list of
    preconditions that the Fund requires borrower countries meet before
    receiving IMF resources, or they may suggest that the government include
    the recommended actions in its official economic program. The Fund reports
    that of the 275 recommendations that were expected to be implemented on or
    before December 31, 2002, 23 percent were

    incorporated as conditions for IMF resources or included in official
    economic program statements.

    Implementation of the Fund*s Assessment Recommendations

    Fund staff monitor central banks* implementation of recommendations by
    performing in- depth reviews of their audited annual financial statements
    and other documents every 12 to 18 months until the borrower country
    government has repaid all Fund resources. The Fund monitors on a
    continuous basis, central banks* implementation of all other safeguards
    measures and of developments within the central banks that may lead to

    new vulnerabilities.

    Appendix VI Update on the International Monetary Fund*s Safeguards
    Assessments

    Page 58 GAO- 03- 734 International Financial Crises

    Recently, the Fund reported that central banks have implemented 90 percent
    of recommendations that IMF staff included as a precondition for receiving
    IMF resources. According to Fund officials, the IMF stopped disbursing
    resources in the few cases where governments failed to implement these
    essential recommendations. Similarly, the Fund reported that central banks
    have implemented 84 percent of measures included in governments* official
    economic program statements.

    On the other hand, the Fund reported that some recommendations made by the
    safeguards assessments have not been implemented as intended, although
    Fund officials state that these delays typically occurred in nonpriority
    areas. When central bank authorities fail to implement the
    recommendations, Fund staff increase pressure to comply, often proposing
    the measures* inclusion as a precondition for the next disbursement.
    However, the Fund reports that staff can only adopt this approach in
    countries where the Fund is actively disbursing funds. For countries that
    are not currently receiving Fund disbursements, implementation of
    recommendations from the safeguards assessments tends to be more
    problematic because the Fund cannot exert pressure through a borrowing
    arrangement.

    Page 59 GAO- 03- 734 International Financial Crises

    Appendix VII Fund/ World Bank FSAP and ROSC Process Appendi x VI I

    Page 60 GAO- 03- 734 International Financial Crises

    Appendix VIII Country Participation in and Publication of FSAPs and ROSCs
    Appendi x VI I I

    Figure 4 lists all countries that have participated in Financial Sector
    Assessment Program (FSAP) or Reports on the Observance of Standards and
    Codes (ROSC) assessments and whether or not these assessments were
    published. The figure describes participation and publication by the 33
    major emerging market countries. Countries highlighted in bold have not
    participated in any assessments. Figure 5 describes participation and
    publication by other countries (industrial, developing, and smaller
    emerging markets).

    Appendix VIII Country Participation in and Publication of FSAPs and ROSCs

    Page 61 GAO- 03- 734 International Financial Crises

    Figure 4: FSAP and ROSC Participation and Publication by Major Emerging
    Market Countries

    a These countries participated in the FSAP under a pilot program. The
    reports were not intended for publication.

    Appendix VIII Country Participation in and Publication of FSAPs and ROSCs

    Page 62 GAO- 03- 734 International Financial Crises

    Figure 5: FSAP and ROSC Participation and Publication by Other Countries

    Appendix VIII Country Participation in and Publication of FSAPs and ROSCs

    Page 63 GAO- 03- 734 International Financial Crises

    a These countries participated in the FSAP under a pilot program. The
    reports were not intended for publication.

    Page 64 GAO- 03- 734 International Financial Crises

    Appendix IX Fund Resources Provided under the Supplemental Reserve
    Facility Appendi x I X

    In recent financial crises, the International Monetary Fund (IMF or the
    Fund) provided large short- term loans under the Supplemental Reserve
    Facility (SRF) with high interest rates to member countries experiencing
    exceptional balance of payments problems. These problems resulted from a
    sharp decline of investor confidence and significant outflows of capital.
    These loans generally were provided when the countries had exceeded their
    financing limit under other loan mechanisms, including the Stand- By
    Arrangement (SBA). In some circumstances, such as Argentina and Uruguay,
    the Fund provided a mix of SRF and SBA loans. Table 6 lists Fund members
    receiving SRF loans from 1997 to 2002.

    Table 6: IMF Supplemental Reserve Facility Loans, 1997* 2002

    Source: IMF.

    Country Loan approval date Loan mix SRF repaid as of May 2003

    Korea 12/ 18/ 97 SRF Yes Russia 7/ 20/ 98 SRF Yes Brazil 12/ 2/ 98 SRF Yes
    Turkey 12/ 21/ 00 SRF Yes

    Argentina 1/ 12/ 01 SRF (40%) SBA (60%) No

    Argentina 9/ 7/ 01 SRF (63%) SBA (37%) No

    Brazil 9/ 14/ 01 SRF No Uruguay 6/ 25/ 02 SRF (33%)

    SBA (67%) No Brazil 9/ 6/ 02 SRF (33%)

    SBA (67%) No

    Page 65 GAO- 03- 734 International Financial Crises

    Appendix X Comments from the Department of the Treasury Appendi x X

    Appendix X Comments from the Department of the Treasury

    Page 66 GAO- 03- 734 International Financial Crises

    Page 67 GAO- 03- 734 International Financial Crises

    Appendix XI Comments from the International Monetary Fund Appendi x XI

    Note: GAO comments supplementing those in the report text appear at the
    end of this appendix.

    See comment 1. See comment 2.

    Appendix XI Comments from the International Monetary Fund

    Page 68 GAO- 03- 734 International Financial Crises

    See comment 3. See comment 4. See comment 5.

    See comment 6.

    Appendix XI Comments from the International Monetary Fund

    Page 69 GAO- 03- 734 International Financial Crises

    Appendix XI Comments from the International Monetary Fund

    Page 70 GAO- 03- 734 International Financial Crises

    The following are GAO*s comments on the letter from the International
    Monetary Fund, dated June 2, 2003. GAO Comments 1. Our assessment of IMF*s
    efforts to anticipate financial crises did not

    focus exclusively on the World Economic Outlook (WEO) and the Early
    Warning System (EWS) models. We examined all six components of the IMF*s
    vulnerability assessment framework. However, as the only mature and
    quantifiable elements of the framework, our analysis focused more heavily
    on the track records of the WEO and EWS. As we reported, these elements
    have not performed well at anticipating prior crises. While we acknowledge
    that the new framework is more comprehensive than past efforts at
    anticipating crises, it is too early to

    assess whether this framework will be successful in anticipating future
    crises.

    2. The IMF*s comment supports our finding. The IMF states that its
    responsibility to maintain financial stability could make its predictions
    appear less accurate since an accurate prediction of crises within WEO
    forecasts would be irresponsible and could lead to a crisis. In effect,
    the IMF acknowledges that their forecasts are overly optimistic and
    validates our finding on the weakness of the WEO component of the
    vulnerability assessment framework. This raises questions regarding the
    purpose and credibility of the WEO forecasts.

    3. We state in the report that the IMF*s new vulnerability assessment
    framework, which includes the examination of external reserve adequacy and
    the strengths and weaknesses of banking systems, is more comprehensive
    than its previous efforts to identify countries at risk of crisis.
    However, it is too early to assess whether this framework will be
    successful in anticipating future crises.

    4. We recognize that the EWS models are just one of six components of the
    IMF*s vulnerability assessment framework. However, the IMF*s own internal
    review of the EWS models concluded that the false alarm rate was too high.

    5. The report recognizes that the FSAP and ROSC assessments constitute a
    valuable source of information about vulnerabilities in member countries
    and states that IMF staff use these assessments to formulate policy
    recommendations. The report also recognizes that the IMF often provides
    technical assistance to help member countries build their

    Appendix XI Comments from the International Monetary Fund

    Page 71 GAO- 03- 734 International Financial Crises

    capacity to implement FSAP and ROSC recommendations. However, the report
    points out several factors that limit the usefulness of FSAP and ROSC
    assessments. Our recommendation, with which the IMF agrees, is designed to
    improve the timeliness and coverage of these assessments. 6. We based our
    description of IMF safeguards assessments on the IMF*s

    reviews of this program. We consider this topic to be within the scope of
    this evaluation because the framework for conducting safeguards
    assessments is derived from the IMF*s Code of Good Practices on
    Transparency in Monetary and Financial Policies. Safeguards assessments
    are thus related to the standards initiative, which constitutes a central
    element of this report.

    Page 72 GAO- 03- 734 International Financial Crises

    Appendix XII Comments from The World Bank Appendi x XI I

    Note: GAO comments supplementing those in the report text appear at the
    end of this appendix.

    See comment 1. See comment 2.

    Appendix XII Comments from The World Bank

    Page 73 GAO- 03- 734 International Financial Crises

    Appendix XII Comments from The World Bank

    Page 74 GAO- 03- 734 International Financial Crises

    The following are GAO*s comments on the letter from the World Bank, dated
    June 2, 2003.

    GAO Comments 1. The report states unambiguously that crises can stem from
    a number of factors, some of which are outside the scope of the FSAP and
    ROSC

    assessments. However, there is broad agreement that the roots of the
    Mexican and Asian financial crises lay in weaknesses in financial systems
    and other institutions. The IMF and the World Bank based their decision to
    launch the FSAP and ROSC initiatives on the premise that timely
    identification of financial sector and institutional vulnerabilities can
    contribute to crisis prevention. The IMF and the World Bank have also
    acknowledged that FSAP and ROSC assessments can contribute to crisis
    prevention efforts by helping private sector participants make better
    informed investment decisions. 2. Our recommendation to pursue strategies
    to increase participation in

    the FSAP and ROSC assessments, including the possibility of making these
    assessments mandatory, stems from the IMF*s and the World Bank*s
    recognition of the need to prioritize participation by important emerging
    market countries. Although many of these countries have volunteered to
    participate in these assessments, others have not. While we are not
    suggesting that the assessments should be made mandatory, it is evident
    that the voluntary nature of the FSAP and ROSC has posed

    an obstacle to full participation by important emerging market countries.

    Page 75 GAO- 03- 734 International Financial Crises

    Appendix XIII GAO Contacts and Staff Acknowledgments Appendi x XI I I

    GAO Contacts Tom Melito (202) 512- 9601 Zina Merritt (202) 512- 5257

    Staff Acknowledgments

    In addition to those individuals named above, Eric Clemons, Suzanne Dove,
    Bruce Kutnick, Jonathan Rose, R. G. Steinman, Ian Ferguson, Mary Moutsos,
    Lynn Cothern, Carl Barden, David Dornisch, and Martin De Alteriis made key
    contributions to this report.

    (320114)

    GAO*s Mission The General Accounting Office, the audit, evaluation and
    investigative arm of Congress, exists to support Congress in meeting its
    constitutional responsibilities

    and to help improve the performance and accountability of the federal
    government for the American people. GAO examines the use of public funds;
    evaluates federal programs and policies; and provides analyses,
    recommendations, and other assistance to help Congress make informed
    oversight, policy, and funding decisions. GAO*s commitment to good
    government is reflected in its core values of accountability, integrity,
    and reliability.

    Obtaining Copies of GAO Reports and Testimony

    The fastest and easiest way to obtain copies of GAO documents at no cost
    is through the Internet. GAO*s Web site (www. gao. gov) contains abstracts
    and fulltext files of current reports and testimony and an expanding
    archive of older products. The Web site features a search engine to help
    you locate documents using key words and phrases. You can print these
    documents in their entirety, including charts and other graphics.

    Each day, GAO issues a list of newly released reports, testimony, and
    correspondence. GAO posts this list, known as *Today*s Reports,* on its
    Web site daily. The list contains links to the full- text document files.
    To have GAO e- mail this

    list to you every afternoon, go to www. gao. gov and select *Subscribe to
    GAO Mailing Lists* under *Order GAO Products* heading.

    Order by Mail or Phone The first copy of each printed report is free.
    Additional copies are $2 each. A check or money order should be made out
    to the Superintendent of Documents. GAO

    also accepts VISA and Mastercard. Orders for 100 or more copies mailed to
    a single address are discounted 25 percent. Orders should be sent to:

    U. S. General Accounting Office 441 G Street NW, Room LM Washington, D. C.
    20548

    To order by Phone: Voice: (202) 512- 6000 TDD: (202) 512- 2537 Fax: (202)
    512- 6061

    To Report Fraud, Waste, and Abuse in Federal Programs

    Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm E- mail:
    fraudnet@ gao. gov Automated answering system: (800) 424- 5454 or (202)
    512- 7470

    Public Affairs Jeff Nelligan, Managing Director, NelliganJ@ gao. gov (202)
    512- 4800 U. S. General Accounting Office, 441 G Street NW, Room 7149

    Washington, D. C. 20548

    United States General Accounting Office Washington, D. C. 20548- 0001

    Official Business Penalty for Private Use $300

    Address Service Requested Presorted Standard

    Postage & Fees Paid GAO Permit No. GI00
    *** End of document. ***