Global Health: Global Fund to Fight AIDS, TB and Malaria Has	 
Improved Its Documentation of Funding Decisions but Needs	 
Standardized Oversight Expectations and Assessments (07-MAY-07,  
GAO-07-627).							 
                                                                 
The Global Fund to Fight AIDS, Tuberculosis and Malaria has	 
approved about $7 billion in grants to developing countries; the 
U.S. has contributed $1.9 billion. The State Department's Office 
of the Global AIDS Coordinator (OGAC) coordinates the U.S.	 
government's overseas AIDS programs, with participation from the 
Department of Health and Human Services (HHS) and the U.S. Agency
for International Development (USAID). In 2003, Congress directed
GAO to report on the Global Fund every 2 years. This report	 
assesses the Global Fund's (1) documentation of information used 
to support performance-based funding decisions, (2) progress in  
implementing a risk assessment model and early warning system,	 
and (3) oversight of the performance of "local fund agents"	 
(LFAs), which monitor grant progress in recipient countries. GAO 
reviewed the documentation for funding decisions and interviewed 
key officials.							 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-07-627 					        
    ACCNO:   A69166						        
  TITLE:     Global Health: Global Fund to Fight AIDS, TB and Malaria 
Has Improved Its Documentation of Funding Decisions but Needs	 
Standardized Oversight Expectations and Assessments		 
     DATE:   05/07/2007 
  SUBJECT:   Acquired immunodeficiency syndrome 		 
	     Decision making					 
	     Diseases						 
	     Documentation					 
	     Federal aid to foreign countries			 
	     Funds management					 
	     Grant administration				 
	     Grant monitoring					 
	     Grants						 
	     International relations				 
	     Performance appraisal				 
	     Performance measures				 
	     Program evaluation 				 
	     Reporting requirements				 
	     Risk assessment					 
	     Risk management					 
	     Tuberculosis					 
	     Program implementation				 
	     Global Fund to Fight AIDS, Tuberculosis,		 
	     and Malaria					 
                                                                 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-07-627

   

     * [1]Results in Brief
     * [2]Background

          * [3]Global Fund Portfolio Distribution
          * [4]Key Elements of the Global Fund's Structure
          * [5]Performance-Based Funding Process

     * [6]Global Fund Has Improved Documentation of Its Basis for Fund

          * [7]Global Fund Has Improved Its Processes for Documenting the B
          * [8]Grant Files Demonstrate Improved Documentation Supporting Fu
          * [9]Global Fund Informally Gathers Missing Data on Grant Recipie

     * [10]Global Fund Did Not Implement Earlier Risk Model but Is Deve

          * [11]Global Fund Did Not Implement the Grant Risk Assessment Mode
          * [12]Global Fund Applies Elements of Risk Management but Lack of
          * [13]Global Fund Is Developing a Comprehensive Framework for Risk

     * [14]Global Fund Has Limited Ability to Manage and Oversee Local

          * [15]Global Fund Has Not Standardized Its Expectations for Local
          * [16]Global Fund Does Not Assess LFAs Systematically and Lacks Fo

               * [17]LFA Performance Assessment and Reporting Are Informal
                 and Ir
               * [18]Lack of Assessment Records Limits Available Information
                 on L

          * [19]Global Fund Previously Attempted to Implement Systematic LFA
          * [20]LFA Performance and Capacity Reportedly Vary

     * [21]Conclusions
     * [22]Recommendations for Executive Action
     * [23]Agency Comments
     * [24]Appendix I: Objectives, Scope, and Methodology
     * [25]Appendix II: Other Sources Cited

          * [26]Principles for Contractor Management

               * [27]Analyses of LFA Capacity and Performance

     * [28]Appendix III: Comments from the Global Fund
     * [29]Appendix IV: Comments from Departments of State and Health a
     * [30]Appendix V: GAO Contact and Staff Acknowledgments

          * [31]GAO Contact
          * [32]Staff Acknowledgments

               * [33]Order by Mail or Phone

Report to Congressional Committees

United States Government Accountability Office

GAO

May 2007

GLOBAL HEALTH

Global Fund to Fight AIDS, TB and Malaria Has Improved Its Documentation
of Funding Decisions but Needs Standardized Oversight Expectations and
Assessments

GAO-07-627

Contents

Letter 1

Results in Brief 3
Background 5
Global Fund Has Improved Documentation of Its Basis for Funding Decisions
14
Global Fund Did Not Implement Earlier Risk Model but Is Developing
Comprehensive Risk Framework That Includes Early Alert and Response System
19
Global Fund Has Limited Ability to Manage and Oversee Local Fund Agents'
Performance 22
Conclusions 27
Recommendations for Executive Action 28
Agency Comments 28
Appendix I Objectives, Scope, and Methodology 30
Appendix II Other Sources Cited 36
Appendix III Comments from the Global Fund 37
Appendix IV Comments from Departments of State and Health and Human
Services and U.S. Agency for International Development 40
Appendix V GAO Contact and Staff Acknowledgments 44

Table

Table 1: Key Source Documents for Disbursement and Grant Renewal Decisions
13

Figures

Figure 1: Global Fund Allocations between 2002 and 2006 by Geographical
Region and Disease 7
Figure 2: Global Fund Allocations by Disease and Intervention Category 8
Figure 3: Performance-based Funding Oversight by Global Fund Secretariat,
Grant Recipients, and LFAs 9
Figure 4: Global Fund Disbursement Decision Process 12
Figure 5: Key Elements of the Global Fund's Phase 2 Grant Renewal Process
13

Abbreviations

CCM country coordinating mechanism
HHS Department of Health and Human Services
HIV/AIDS human immunodeficiency virus/acquired immune deficiency
  syndrome
LFA local fund agent
OGAC Office of the U.S. Global AIDS Coordinator
TB tuberculosis
UNAIDS Joint United Nations Programme on HIV/AIDS
USAID United States Agency for International Development
WHO World Health Organization

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

United States Government Accountability Office
Washington, DC 20548

May 7, 2007

The Honorable Joseph R. Biden, Jr.
Chairman
The Honorable Richard G. Lugar
Ranking Minority Member
Committee on Foreign Relations
United States Senate

The Honorable Tom Lantos
Chairman
The Honorable Ileana Ros-Lehtinen
Ranking Minority Member
Committee on Foreign Affairs
House of Representatives

The Global Fund to Fight AIDS, Tuberculosis and Malaria (the Global Fund)
was created to help finance the fight against three infectious diseases
that kill more than 6 million people each year. Established in 2002, the
Global Fund is a private foundation intended as a partnership among
governments, civil society, the private sector, and affected communities.
From 2002 through February 2007, the Global Fund approved grant proposals
totaling about $7 billion to governmental and nongovernmental entities to
carry out disease intervention activities in more than 100 countries.
During this period, the U.S. government contributed $1.9 billion--the
largest amount provided to the Global Fund by a single donor. The State
Department's Office of the Global AIDS Coordinator (OGAC) coordinates the
U.S. government's overseas AIDS programs, with participation from the
Department of Health and Human Services (HHS) and the U.S. Agency for
International Development (USAID).

The Global Fund uses a performance-based funding system designed to
encourage financial accountability and program progress and to link its
continued grant disbursements to achievement of these results. The Global
Fund contracts with "local fund agents" (LFA) to assess recipients'
capacity for grant implementation and verify recipients' financial and
program performance data. In June 2007, it plans to issue applications for
new contracts for LFA oversight services.

In 2003, Congress directed the Comptroller General to monitor and evaluate
projects supported by the Global Fund and to submit a report every 2
years.1 In June 2005, we reported that the Global Fund's grant
documentation did not always show clear grounds for performance-based
funding decisions. 2 These decisions can include periodic disbursements as
well as decisions to renew, suspend, or terminate grants. We further noted
that some of the grant recipients' self-reported performance data were of
poor quality or incomplete. We also reported that the Global Fund was
developing a risk assessment model and early warning system to alert its
fund portfolio managers to the need to respond to poorly performing
grants.3 In addition, we noted that LFAs' assessments of recipient
performance contained inconsistencies and that LFAs often lacked the
knowledge and experience needed for these assessments. In its response to
our 2005 report, the Global Fund generally concurred with our findings and
indicated that it was taking steps to improve its structures and systems.

Following up on our 2005 review, this report assesses the Global Fund's
(1) documentation of the information used to support performance-based
funding decisions, (2) progress in implementing a risk assessment model
and early warning system, and (3) oversight of local fund agents'
performance.

To address these objectives, we met with key officials from the Global
Fund and reviewed Global Fund documents, including key source documents
from grant files. To assess the documentation of the information used to
support performance-based funding decisions, we analyzed two separate
random samples of grant decisions from the Global Fund. The first sample
consisted of 80 grants that were assessed for performance-based funding
disbursements between January 1 and October 1, 2006. The second sample
consisted of 45 grants that were assessed for phase 2 grant renewal.4 We
projected our findings to the files of all grants that received a
disbursement or were assessed for grant renewal during that period. To
assess Global Fund progress in implementing a risk assessment model and
early warning system, we reviewed international standards on risk
management,5 met with Global Fund secretariat officials, and reviewed
Global Fund documents. We also held discussions with the Global Fund's
Inspector General and three of its key technical partners.6 To assess the
Global Fund's oversight and measurement of LFAs' capacity and performance,
we met with each of the six LFA headquarters offices and relevant
secretariat personnel. In addition, we held structured interviews with LFA
teams in 12 countries via telephone. We also obtained perspectives on the
progress of the Global Fund from its partners and outside experts. We
conducted our work between May 2006 and March 2007 in accordance with
generally accepted government auditing standards. (See app. I for a
detailed description of our scope and methodology.)

1United States Leadership Against HIV/AIDS, Tuberculosis, and Malaria Act
of 2003, Pub. L. No. 108-25, S 202 (f), 117 Stat. 711, 727 (codified at 22
U.S.C. S 7622).

2The June 2005 report on the Global Fund was our first review in response
to the mandate. See GAO, Global Health: The Global Fund to Fight AIDS, TB
and Malaria Is Responding to Challenges but Needs Better Information and
Documentation for Performance-based Funding, [34]GAO-05-639 (Washington,
D.C.: June 2005). We also completed a report in 2003 on the Global Fund at
the request of the Chairman, Committee on Appropriations, Subcommittee on
Foreign Operations, Export Financing, and Related Programs. See GAO,
Global Health: Global Fund to Fight AIDS, TB and Malaria Has Advanced in
Key Areas, but Difficult Challenges Remain, [35]GAO-03-601 (Washington,
D.C.: May 2003). This report is our final response to the 2003 mandate.

3Risks faced by Global Fund grants include problems associated with
operating in countries with varying levels of economic development and
capacity, which can affect grant effectiveness.

Results in Brief

The Global Fund has improved its documentation of information that
supports its performance-based funding decisions for disbursements and
grant renewals. Since our 2005 review, the Global Fund has updated its
documentation system to require that portfolio managers more consistently
document factors, such as grant ratings and contextual information, on
which they base their decision to continue funding a grant.7 Our review of
80 grant disbursements and 45 phase 2 grant renewal decisions confirmed
that Global Fund grant files consistently contained explanations of the
information used to make its funding decisions.8 For example, 100 percent9
of the disbursement and phase 2 files in our samples contained narratives
explaining the manager's rating of the grant's performance. We noted that
many reports completed by grant recipients lacked explanations either for
not meeting or for exceeding the grant's targets. However, Global Fund
officials said that fund portfolio managers obtain the needed information
informally from the recipients or other stakeholders.

4The Global Fund's Board approves grant proposals covering up to a 5-year
period and initially commits grants for the first 2-year phase of the
proposal period. Continued grant funding for phase 2 is conditioned on
program performance, based on a review of performance reports and other
information taken 20 months after the grant program's start date.

5See Committee of Sponsoring Organizations of the Treadway Commission,
Enterprise Risk Management --Integrated Framework Executive Summary: 2004.

6As a financing entity, the Global Fund relies on partners to provide
technical support to grant recipients. Its key partners include UNAIDS,
OGAC, the World Bank, and the World Health Organization (WHO).

The Global Fund did not implement the risk assessment model that it was
developing at the time of our 2005 report, because it determined that the
model did not accurately identify grant risk. To identify risks that may
affect grant implementation, the organization currently relies on elements
of its structures and processes, including its initial technical review,
disbursement decision-making form, periodic grant ratings, oversight by
country stakeholders, information from technical partners, and LFA
oversight. Recognizing the need for a more comprehensive approach to risk
management the Global Fund has begun developing a risk assessment
framework for the organization that includes an early alert and response
system to address poorly performing grants.

Because it does not require systematic assessments of LFAs' performance,
the Global Fund has limited access to the information it needs to manage
and oversee LFAs.10 Global Fund portfolio managers set expectations for
LFA performance informally and individually, often resulting in, for
example, variances in scopes of work and definitions of satisfactory
performance. Moreover, the Global Fund does not require systematic
assessment or reporting of LFAs' performance; instead, managers provide
feedback to LFAs informally. Because of the lack of systematic assessment,
the Global Fund has limited ability to determine the quality of grant
services provided by LFAs and to identify situations in which more
oversight of LFAs by the Global Fund may be required. The Global Fund
previously introduced a tool to assess LFA performance more
systematically. However, this effort was unsuccessful because use of the
tool was not mandatory. Numerous sources raise concerns about the quality
of grant oversight provided by LFAs, particularly their ability to assess
and verify recipients' procurement capacity and program implementation.

7According to the Global Fund's disbursement decision-making form,
contextual information critical to disbursement decisions includes
qualitative factors relating to governance, grant management, and external
factors. This form, together with the principal recipient's report on the
achievement of all individual targets and the LFA's verification is the
documented basis for funding decisions during implementation.

8The Global Fund maintains a file for each grant that contains, among
other things, all reports relating to that grant's performance, funding
requests, and disbursement and phase 2 assessments and funding decisions.

9For both samples the estimate was 100 percent. For each of these
estimates we are 95 percent confident that at least 94 percent of all
files in our population contain narratives explaining the manager's rating
of the grant's performance.

10Good practice suggests that organizations should monitor their
contractors' performance during the course of interaction and periodically
evaluate performance based on clear expectations. Assessment results
should include consistent patterns of rewards, penalties, or both. (See
app. II for sources of these criteria.)

To improve the Global Fund's ability to oversee the performance of its
local fund agents, we recommend that the Secretaries of State and HHS work
with the Global Fund's Board Chair and Executive Director to (1) establish
standardized expectations for LFA performance; and (2) require systematic
assessments of LFA performance and the collection and analysis of
performance data.

In responding to our draft report, the Departments of State and Health and
Human Services, the U.S. Agency for International Development, and the
Global Fund agreed with the report's findings, conclusions, and
recommendations. (See apps. III and IV for their comments.)

Background

The Global Fund, a private foundation in Switzerland, was established in
2002 as a funding mechanism to collect public and private contributions to
finance grants for the prevention and treatment of HIV/AIDS, tuberculosis
(TB), and malaria.11 About 96 percent of the contributions to the Global
Fund since its inception have come from governments. Private sector
support increased in 2006, notably owing to a pledge of $500 million by
the Bill and Melinda Gates Foundation and receipt of almost $19 million
through the (RED)(TM) initiative.12

11UNAIDS and WHO estimate that 39.5 million people were living with HIV in
2006 and that 2.9 million people died of AIDS-related illnesses in 2006.
WHO estimates that malaria kills more than 1 million people per year and
that 124 million people in Africa live in areas that are at risk for
seasonal malaria epidemics. WHO also estimates that 1.7 million people die
of TB each year.

By December 2006, the Global Fund had completed six proposal rounds,
approved about $7 billion in grants, signed grant agreements worth over $5
billion and disbursed more than $3 billion to grant recipients in 132
countries. The Global Fund also reports that by December 2006, its grants
enabled recipients to supply antiretroviral drugs to 770,000 people
infected with HIV/AIDS, treat 2 million people infected with tuberculosis,
and distribute 18 million insecticide-treated nets to combat malaria.

Global Fund Portfolio Distribution

The Global Fund's underlying principles require, among other things, that
it maintain a balanced grant portfolio in terms of region, disease, and
type of intervention; operate transparently and accountably; and use a
simple, rapid, and innovative grant-making process.13 As figure 1 shows,
between 2002 and 2006, the Global Fund distributed 55 percent of grant
funds to sub-Saharan Africa, with the remainder of the funding divided
among other regions of the world. Also during this time period, the Global
Fund distributed 57 percent, 15 percent, and 27 percent of grant funds,
respectively, for HIV/AIDS, tuberculosis, and malaria programs. The
remaining 1 percent funded projects to strengthen health systems.

12(RED)(TM) is a private sector initiative to provide financing for Global
Fund HIV/AIDS projects in Africa. Companies participating in the
initiative donate a portion of profits from the sale of (RED) branded
items.

13The Global Fund's key principles are to (1) operate as a financial
instrument, not an implementing entity; (2) make available and leverage
additional resources; (3) support programs that evolve from national plans
and priorities; (4) operate in a balanced manner with respect to
geographic regions, diseases, and healthcare interventions; (5) pursue an
integrated and balanced approach to prevention, treatment, care, and
support; (6) evaluate proposals through an independent review process; and
(7) operate in a transparent and accountable manner and employ a
simplified, rapid, and innovative grant-making process.

Figure 1: Global Fund Allocations between 2002 and 2006 by Geographical
Region and Disease

In 2005, the Global Fund began tracking the allocation of grant funds in
grant proposals according to the disease intervention categories of
prevention, care, and treatment. Figure 2 shows the Global Fund's
estimated distribution by disease and intervention category.

Figure 2: Global Fund Allocations by Disease and Intervention Category

aFor HIV/AIDS, "Other" includes grants for health system strengthening,
monitoring and evaluation, supportive environment, HIV/TB collaborative
activities, and other activities.

bFor tuberculosis, "Other" includes grants for health system
strengthening, monitoring and evaluation, supportive environment, and
other activities.

cFor malaria, "Other" includes grants for health system strengthening,
supportive environment, and other activities.

Key Elements of the Global Fund's Structure

The Global Fund's principles also require it to focus on performance by
linking its resources to the achievement of clear, measurable, and
sustainable results. To achieve this focus on performance-based funding,
the Global Fund relies primarily on oversight by, and interaction among,
Global Fund secretariat personnel,14 grant recipients, and LFAs. (See fig.
3.)

14Between 2003 and 2006 the Global Fund secretariat staff increased from
63 to 240 employees.

Figure 3: Performance-based Funding Oversight by Global Fund Secretariat,
Grant Recipients, and LFAs

aFund portfolio managers also have formal relationships with grant
recipients and they interact with all stakeholders including LFA central
teams

           o Global Fund secretariat personnel. Within the secretariat, the
           fund portfolio managers are primarily responsible for reviewing
           grant progress and making performance-based funding
           recommendations. Other secretariat personnel are consulted during
           key stages in the grant life cycle. Fund portfolio managers
           interact with the LFA country teams providing oversight services
           for their grant, as well as with principal recipients,
           subrecipients, technical partners, and other stakeholders in the
           recipient countries. In addition to the country teams, staff from
           the contracts department and the Global Fund's LFA manager
           interact with LFA headquarters teams, known as the central teams,
           regarding contracting and administrative matters and any problems
		   related to LFA country teams.

           o Grant recipients. At the country level, the principal grant
           recipient is responsible for receiving and implementing the
           grant.15 During implementation, the principal recipient is
           responsible for monitoring and evaluating the grant's
           effectiveness in accordance with grant objectives and making sure
           that funds are properly accounted for. The principal recipient is
           also responsible for overseeing the activities of any
           subrecipients implementing grant activities.
           o Local fund agents. The Global Fund currently contracts with LFA
           headquarters central teams at six organizations16 to oversee
           grants in 120 countries and multicountry programs.17 In the
           majority of cases, the central team establishes interfirm
           agreements with its affiliated offices within each country to
           perform LFA services.18 The LFA assesses recipient capacity prior
           to grant signing and performs verification and oversight duties
           throughout the life of the grant. The LFA central teams typically
           contract with external specialists for work outside the expertise
           of their organization, especially during the initial assessments
           of recipient capacity. For example, during the initial
           assessments, some organizations contract with a procurement expert
           to review recipients' capacity for procurement and supply chain
           management. The Global Fund budgeted approximately $22 million--26
           percent of the secretariat's operating budget--to support the LFA
           structure for 2006 and spent about an additional $320,000 on
           LFA-related functions, including personnel costs related to LFA
           management, studies of the LFA model, and travel to meet with
           LFAs.
		   
		   Performance-Based Funding Process

           The Global Fund's performance-based funding process involves an
           initial assessment of recipients' capacity to manage and implement
           grants, periodic disbursement decisions based largely on reports
           by the grant recipient and the LFA, and a phase 2 grant renewal
           review after about 2 years.

           o Assessment of grant recipients' capacity. Prior to funding a
           grant, the Global Fund considers, among other things, a grant
           recipient's ability to (1) manage, evaluate, and report on
           grant-funded activities; (2) manage and account for funds; and (3)
           procure goods and services. LFAs assess prospective recipients'
           capacity to perform these activities. Funding is conditioned on
           recipients' meeting performance targets mutually agreed with
           Global Fund technical experts.

           o Periodic disbursements. After the initial screening and approval
           process, the grant recipient must produce quarterly or semi-annual
           reports describing financial and program progress as well as a
           funding request for the next disbursement period. The grant
           recipient sends these reports to the LFA, which prepares a report
           verifying the accuracy of the information, identifying potential
           grant problems, rating grant performance, and recommending a
           disbursement amount. The LFA then forwards the grant recipient's
           report and its own report to the designated Global Fund portfolio
           manager. The fund portfolio manager compares the financial
           information and data on program progress with the grant's
           performance targets, taking into account information such as the
           status of procurements and political or economic constraints. The
           fund portfolio manager's team leader makes a funding decision if
           the disbursement is less than $2 million.19 Figure 4 illustrates
           the Global Fund's periodic disbursement decision process.

           Figure 4: Global Fund Disbursement Decision Process

           o Grant renewal process. A key milestone in the performance-based
           funding approach is the phase 2 grant renewal review--a detailed
           analysis of grant progress at about 2 years, required to continue
           funding for up to the next 3 years. Similar to its process for
           disbursement decisions, the Global Fund's grant renewal process
           requires grant recipients to send documentation of financial and
           program progress to the LFA, which verifies the information and
           analyzes the phase 2 proposal. In a report to the Global Fund, the
           LFA identifies potential grant problems, rates the performance of
           the grant, and recommends the amount needed to fund up to the next
           3 years of the grant. Because phase 2 is the decision point for
           continuing the grant to its 5-year completion, the process also
           includes an extensive and detailed review of cumulative
           information by fund portfolio managers, their supervisors, and
           evaluation and finance staff. The phase 2 renewal process
           culminates in a panel discussion and final recommendation to the
           board. Figure 5 illustrates key elements of the Global Fund's
           phase 2 grant renewal process.

15Government agencies are the principal recipients of 64 percent of grant
funds. Nonprofit development organizations and multilateral organizations
also act as principal recipients. In many cases the principal recipient
further distributes the funds to multiple subrecipients to implement grant
activities.

16As of 2007, the Global Fund had contracted with the following
organizations to perform LFA oversight duties: PricewaterhouseCoopers,
KPMG, Emerging Markets Group, Crown Agents, the Swiss Tropical Institute,
and the United Nations Office for Project Services. PriceWaterhouseCoopers
and KPMG together account for LFA services in 97 of 120 countries and
regional programs.

17In addition to 113 country programs, LFAs oversee grants in 1
multicountry program in Africa 5 multicountry programs in Latin America
and the Caribbean, and 1 multicountry program in the Western Pacific.

18The specific responsibilities delegated to the LFA central team, LFA
country team, and outside experts sometimes vary from one LFA organization
to another. For example, of the two firms with the largest portfolios, one
uses a decentralized approach in which the majority of the work occurs
within the country team, and the other uses a centralized approach in
which the country teams perform the oversight tasks, but all LFA products
are vetted through the central team before reaching the secretariat. Using
a different approach, one of the smaller LFA organizations has very
limited presence within the recipient countries; instead, it sends
specialists into the country to perform oversight activities as needed.

Performance-Based Funding Process

19A disbursement greater than $2 million requires a decision by the
secretariat's Director of Operations.

Figure 5: Key Elements of the Global Fund's Phase 2 Grant Renewal Process

Table 1 outlines the key source documents related to disbursement
decisions and phase 2 grant renewals from electronic "grant files" that we
reviewed in conducting this study.

Table 1: Key Source Documents for Disbursement and Grant Renewal Decisions

Funding decision  Reporting form                   Completed by            
Disbursement                                                               
                     Progress Update and Disbursement Grant recipient         
                     Request                                                  
                     LFA Progress Review and          LFA                     
                     Recommendation                                           
                     Disbursement Decision-Making     Global Fund secretariat 
                     Form                                                     
Phase 2 renewal                                                            
                     Request for Continued Funding    Country coordinating    
                                                      mechanism (CCM)20       
                     Grant Performance Report         Global Fund secretariat 
                     Phase 2 Request Assessment       LFA                     
                     Report                                                   
                     Grant Scorecard                  Global Fund secretariat 

Source: GAO analysis based on Global Fund documentation.

In preparation for its retender of LFA contracts worldwide in mid to late
2007, the secretariat will continue discussions on the key principles that
will potentially shape the design of the next LFA model. According to a
Global Fund official, they will take into consideration findings from this
review and two internal Global Fund studies. They expect to issue the
request for applications in late June.

Global Fund Has Improved Documentation of Its Basis for Funding Decisions

The Global Fund has improved its documentation of information supporting
its performance-based funding decisions. For example, the Global Fund's
system now requires that portfolio managers consistently document factors
that may affect the decision to continue funding a grant, especially when
the grant falls short of its performance targets. In addition, the Global
Fund grant files that we reviewed consistently contained explanations of
funding decisions and contextual data informing the decisions. Although we
found many reports submitted by grant recipients to the Global Fund do not
provide explanations for all unmet grant performance targets, according to
Global Fund officials, fund portfolio managers are able to obtain the
needed information directly from the grant recipients or other
stakeholders.

20Country coordinating mechanisms are the entities that formally submit
the Request for Continued Funding. In their capacity as the national
authority that oversees grant implementation, CCMs include representatives
from both the public and private sectors, including governments;
multilateral or bilateral agencies; non-governmental organizations;
academic institutions; private businesses; and people living with the
diseases. For each grant, the CCM nominates one or a few public or private
organizations to serve as grant recipient.

Global Fund Has Improved Its Processes for Documenting the Basis for Funding
Decisions

The Global Fund has developed a more systematic process for documenting
the basis for its funding decisions. In June 2005, we noted that the
Global Fund did not consistently document the reasons for its
determinations that recipients' performance warranted additional funding
for periodic disbursements or its recommendations regarding grant
renewals. We recommended that the Global Fund continue efforts to clearly
document the Global Fund's reasons for periodically disbursing funds and
renewing grant agreements. Subsequently, to increase the transparency of
its performance-based funding system, the organization made several
changes to its documentation process.

In late 2005, the Global Fund developed a disbursement decision-making
form to ensure that fund portfolio managers consistently show the basis
for each disbursement decision. The form requires, among other things,

           o an explanation of the performance rating assigned to the grant,
           addressing any discrepancy between ratings by the fund portfolio
           manager and the LFA and verifying that the manager has considered
           any problems raised by the LFA regarding data quality and
           reporting, program progress, and expenditures;21 and
           o contextual information critical to the disbursement decision,
           such as information demonstrating governance problems, grant
           management weaknesses, currency fluctuations, or natural disasters
           in the country or region where a funded project is being
           implemented.

           The Global Fund also revised other forms to provide more
           systematic reporting of grant program progress. These revisions
           included improvements to the grant progress update and
           disbursement request, the LFA progress review and recommendation
           report, and the phase 2 grant scorecard.22 For example, the
           updated progress update form contains a new requirement that LFAs
           comment on the reliability of grant recipient data on target
           achievement. The updated grant scorecard includes requests for
           data showing the timeliness and effectiveness of grant recipients'
           use of the funds during phase 1 of grant implementation. It also
           includes a new section where the secretariat can record plans to
           materially change the grant in phase 2--for example, adding or
           dropping goals or objectives because of lessons learned during
           phase 1.23

           According to Global Fund officials, the changes to their system of
           recording information about grant performance and problems have
           made the decision process more systematic and transparent. For
           example, Global Fund officials, including some fund portfolio
           managers, said the new disbursement decision-making form provides
           managers a place to systematically explain their ratings.
           Officials from OGAC and the World Bank with whom we met said that
           the Global Fund's more consistent documentation of the basis of
           its decisions has made the decisions easier to understand.
		   
		   Grant Files Demonstrate Improved Documentation Supporting Funding
		   Decisions

           Our review of random samples of disbursement and phase 2 grant
           files indicated that most files systematically explain a number of
           key factors used to support disbursement and grant renewal
           decisions.24 In our 2005 report, we observed that the Global
           Fund's grant files provided little explanation of disbursement
           decisions, including decisions to disburse funds to recipients who
           reported not meeting their targets. In addition, our 2005 report
           noted that, according to Global Fund officials, many grant renewal
           decisions were not supported by the information needed to assess
           whether funding decisions were consistently based on grant
           performance.

           In our current review, we found that Global Fund grant files
           consistently contained explanations of ratings of grant
           performance that were important to funding decisions, even when
           key stakeholders' ratings and recommendations did not concur.
           Based on our samples, 100 percent25 of both disbursement and phase
           2 files contained narratives explaining the manager's rating of
           the grant's performance. Most grant files also contained
           explanations for lack of concurrence in ratings or recommendations
           by the various officials charged with assessing grant
           performance.26 For example:

           o In all 8 cases in which fund portfolio managers' phase 2 grant
           ratings differed from those of the evaluation staff, the
           scorecards in the grant files contained an explanation for the
           lack of concurrence.27 
           o In all 5 cases in which fund portfolio managers' phase 2 grant
           renewal recommendations differed from those of the LFAs', the
           scorecards in the grant files contained an explanation for the
           lack of concurrence.
           o In cases in which fund portfolio managers' assessments of grant
           performance led them to disagree with the disbursement amount
           recommended by the LFA or the amount requested by the grant
           recipient, 31 of 32 of those grant files included a narrative
           explaining the discrepancy.28

           According to Global Fund officials, such explanations provide
           valuable information that helps decision-makers better understand
           and consider grant performance.

           The Global Fund files that we reviewed also consistently explained
           contextual information and other issues that may affect the grant
           recipient's ability to meet agreed-upon targets. We found that an
           estimated 96 percent of disbursement files in which portfolio
           managers had specified contextual issues that negatively affected
           grant performance included narratives explaining each issue.
           According to Global Fund officials, such information is crucial to
           grant disbursement and renewal decisions, especially if recipients
           have not met many of the targets set at the grant's initiation. In
           addition, in reviewing 43 disbursement files for grants whose
           recipients had not yet met remedial requirements imposed by the
           Global Fund as conditions for grant renewal, we found that 42 (98
           percent) contained an explanation of recipients' progress toward
           meeting the requirements.
		   
		   Global Fund Informally Gathers Missing Data on Grant Recipientsï¿½
		   Performance

           Although recipients' formal documentation of grant performance
           remains incomplete, Global Fund officials informally gather the
           missing information that they consider necessary for making and
           documenting the basis for funding decisions. In June 2005, we
           reported that recipient progress reports varied in quality and
           that the limited monitoring and evaluation capabilities of many
           recipients raised questions about the accuracy of their
           reporting.29 Our current analysis found that an estimated 39
           percent of phase 2 grant files contained evidence, identified by
           fund portfolio managers, of weaknesses in the reliability of the
           grant recipient's data. In contrast to the consistent
           documentation we found on forms completed by Global Fund staff,
           many periodic performance reports completed by grant recipients
           lacked explanations either for not meeting or for exceeding the
           grant's targets. In those cases, according to LFAs, fund portfolio
           managers, and other Global Fund officials, fund portfolio managers
           relied on regular phone and e-mail contact with grant recipients
           or other stakeholders to obtain the missing information needed for
           performance-based funding decisions. Global Fund officials said
           that information crucial to funding decisions is documented on the
           disbursement decision-making form or phase 2 grant scorecard, even
           if it was not recorded by grant recipients or LFAs in their
           formally required reports.

           Global Fund portfolio managers said that recipients' explanations
           for not meeting targets may influence the portfolio manager's
           recommendation to withhold funds or, alternately, to continue to
           fund the grant despite performance problems. According to Global
           Fund officials and public health experts, improving grant
           recipients' monitoring and evaluation capacity, as well as their
           ability to report accurately and completely, will require
           long-term investment on the part of the Global Fund and its
           partners.
		   
		   Global Fund Did Not Implement Earlier Risk Model but Is Developing
		   Comprehensive Risk Framework That Includes Early Alert and Response
		   System

           The Global Fund did not implement the risk assessment model that
           it was developing at the time of our 2005 report, because it
           determined that the model did not accurately identify grant risk.
           The organization relies on elements of its structures and
           processes, including its initial technical review, disbursement
           decision-making form, periodic grant ratings, CCM oversight,
           information from technical partners, and LFA oversight, to
           identify potential risks. Recognizing the need to establish a more
           comprehensive approach to risk management, the Global Fund has
           begun developing a risk assessment framework for the organization
           that includes an early warning and response system to address
           poorly performing grants.
		   
		   Global Fund Did Not Implement the Grant Risk Assessment Model
		   Described in the 2005 GAO Report

           The Global Fund did not implement the risk-assessment framework to
           identify poorly performing grants that it was developing at the
           time of our 2005 report.30 We reported that the organization had
           devised a risk assessment model and an early warning system to
           identify poorly performing grants and more systematically alert
           portfolio managers when they needed to intervene.31 According to
           the Global Fund, because it disburses grants in countries with
           varying levels of economic development and health systems
           capacity, the risk model was to incorporate quantitative data,
           such as grant size and performance as well as country development
           and corruption data. Using these quantitative indicators, the
           model would generate reports that indicated potential problems,
           and portfolio managers would evaluate the reports in light of
           contextual information. According to an official who participated
           in developing the framework, it was not implemented because it
           relied too heavily on quantitative indicators and did not
           adequately account for contextual factors, leading to risk ratings
           that were too inaccurate to be useful. For example, according to
           Global Fund officials, in a test of the model, a particular grant
           was rated as low risk based on the host country's development and
           corruption indicators. However, the grant ultimately failed and
           was not renewed at the phase 2 grant renewal review.
		   
		   Global Fund Applies Elements of Risk Management but Lack of Risk
		   Assessment Model Limits its Ability to Respond to Poorly
		   Performing Grants

           Although the Global Fund's grant oversight incorporates several
           elements of risk management, the absence of a risk assessment
           model limits its ability to respond methodically to poorly
           performing grants and use available resources to increase
           oversight as needed. According to Global Fund officials, the
           organization currently relies on existing oversight mechanisms
           such as its initial technical review of grant proposals,
           standardized decision forms, periodic grant ratings, CCM
           oversight, technical partners' assistance, and LFAs' reviews to
           identify and assess potential mismanagement, irregularities, or
           other risks to successful grant implementation.

           o Technical review. The technical review panel raises important
           technical issues that grantees must address when presenting their
           proposals to the panel. Proposals with a very high technical risk
           profile generally are not recommended for funding.

           o Disbursement decision-making form. Grant risk is mitigated by
           the fund portfolio manager's use of the standard disbursement
           decision-making form, which includes financial and program data
           from the recipient that has been verified by the LFA.

           o Rating grant progress for each disbursement request. Since 2006,
           according to Global Fund officials, every progress and
           disbursement request, after the first disbursement, has been
           graded on a scale of A, B1, B2, or C. This information is posted
           on the Global Fund Website's financial data base and included in
           grant performance reports that are updated on the Global Fund
           website at each disbursement. This rating system is a key tool for
           identifying grants at risk and for sharing information with
           technical partners.

           o Country coordinating mechanism (CCM). A key responsibility of
           the CCM is to continue to monitor grant progress during
           implementation and identify problems that may affect progress. In
           many countries, Global Fund technical partners such as UNAIDS,
           WHO, and USAID are members of CCMs.

           o Partner assistance. Global Fund officials rely on technical
           partners with staff based in recipient countries to provide
           contextual information on grant performance, identify risks, and
           help respond to problems that affect grant implementation.

           o LFA reviews. Global Fund officials emphasized that LFAs identify
           potential grant risks in their periodic verification of grant
           progress and during the cumulative review and performance
           assessment for the phase 2 renewal process.

           The absence of a model to assess grant risk has limited the Global
           Fund's ability to methodically identify poorly performing grants
           and use available resources to increase oversight as needed.
           According to a senior manager from an LFA with one of the largest
           grant portfolios, because the Global Fund has not assessed grants
           as high, medium, or low risk, LFA oversight has not differed
           substantially by grant risk level and LFA oversight resources have
           not been used to optimal effect. For example, although some grants
           may require more oversight because of potential misuse of grant
           money or limited recipient capacity, LFA task orders and fees are
           generally no different for potentially problematic grants than for
           grants with few expected problems. Similarly, the frequency of LFA
           visits to project sites for monitoring and data verification has
           not been based on a systematic estimate of risk.
		   
		   Global Fund Is Developing a Comprehensive Framework for Risk
		   Management and Has Established First Components

           Global Fund officials informed us in April 2007 that they are
           developing a top-level framework for risk management that will
           cover all aspects of the organization's strategic direction,
           including addressing resources and skills at the secretariat
           level, developing a model for systematically assessing grant risk,
           and fully implementing an early alert and response system. Global
           Fund officials have acknowledged the need for a quantitative model
           for ranking grants by level of risk, supplemented by additional
           qualitative factors that may change a grant's rating or ranking,
           to identify and mitigate grant risk in a timely manner. The Global
           Fund recently organized a risk seminar for directors and senior
           managers and engaged an international firm to help them achieve
           consensus on a shared approach to comprehensive risk management.
           We reviewed a draft outline of the guiding principles for
           developing a risk management framework that the Global Fund
           developed in February 2007. As of April 2007, the Global Fund is
           in varying stages of progress in developing and implementing
           detailed risk strategies.

           According to Global Fund officials, the secretariat has put in
           place the first components of its early alert and response system
           intended to identify as early as possible grants that are
           performing poorly and facing implementation difficulties. These
           first components include posting the grant disbursement and
           renewal ratings on the Global Fund Website and efforts to engage
           more technical assistance from partners for poorly performing
           grants. Global Fund officials recognize that for the system to
           work effectively principal recipients, CCMs, stakeholders, and
           partners must share responsibility for tracking the progress of
           grant implementation and take appropriate action when progress is
           impeded. Global Fund officials stated that the early alert and
           response system will eventually include documenting and
           disseminating best practices, formalizing the system within the
           secretariat and among country-level stakeholders and partners, and
           increasing the engagement of partners in grant oversight.
		   
		   Global Fund Has Limited Ability to Manage and Oversee Local Fund
		   Agentsï¿½ Performance

           The Global Fund lacks the information it needs to manage and
           oversee the LFAs because of a lack of systematic assessments of
           LFAs' performance. Good practice suggests that organizations
           monitor their contractors' performance during the course of
           interaction and periodically evaluate performance based on clear
           expectations. Assessment results should include consistent
           patterns of rewards, penalties, or both. (See app. II for sources
           of these principles.) Global Fund portfolio managers set
           expectations for LFA performance informally and individually,
           frequently leading to inconsistent expectations and resulting in
           challenges for both the LFAs and their managers. Moreover, the
           Global Fund does not require formal assessments or reports of LFA
           performance; instead, LFAs are assessed informally and
           irregularly, and assessment results are not systematically
           documented. As a result, the Global Fund has limited records of,
           and data on, LFA performance. Although the Global Fund made a
           previous attempt to assess LFAs more systematically, this effort
           was unsuccessful. According to secretariat staff, key partners,
           and outside experts, LFAs' performance and capacity vary.
		   
		   Global Fund Has Not Standardized Its Expectations for Local Fund
		   Agent Performance

           Rather than establish standardized performance expectations for
           all LFAs, the Global Fund portfolio managers establish
           expectations informally for the quality of each LFA's performance.
           The Global Fund's contracts with LFAs, as well as the guidelines
           that it provides, outline the tasks that all LFAs must complete,
           such as considering a principal recipient's programmatic and
           financial implementation progress in relation to its plans. The
           contracts also contain general statements about LFA performance;
           for example, a sample LFA contract specifies that all tasks should
           be carried out to the satisfaction of the Global Fund.32 However,
           Global Fund policy states--and secretariat staff and 11 of the 12
           LFA country teams that participated in our structured interview
           confirmed--that the fund portfolio manager is primarily
           responsible for defining satisfactory performance for his or her
           designated LFAs. As of March 2007, the Global Fund had not
           provided fund portfolio managers any specific training or written
           guidance defining acceptable or good LFA performance, including
           the quality of their reporting.33

           The lack of standardized performance expectations results in
           challenges both for the LFAs and for Global Fund management.

           o LFA challenges. Because fund portfolio managers set expectations
           informally and individually, LFA country teams frequently
           encounter different expectations from successive portfolio
           managers. For example, among the LFA country teams who
           participated in our structured interviews, 6 of 10 teams who had
           interacted with more than one fund portfolio manager said that the
           managers' criteria for a high-quality report had varied. According
           to the respondents, variations in fund portfolio managers'
           expectations often involved differences in the scope of work that
           they expected LFAs to complete. For instance, members of one
           country team said that their previous fund portfolio manager had
           wanted the team to focus primarily on financial data in its
           reports, while the current manager wanted a focus on programmatic
           issues. In another country, one fund portfolio manager wanted the
           team to conduct on-site verification of grant subrecipients, while
           a subsequent manager did not consider site visits a priority. In
           the remaining four cases, country teams did not report substantial
           variation between different fund portfolio managers. In one of
           these cases, the team said that a previous manager established
           expectations that subsequent managers used in turn, therefore
           providing continuity.

           o Management challenges. The lack of standardized LFA performance
           expectations also presents challenges among those responsible for
           managing the LFAs. According to the manager of one LFA central
           team with a large grant portfolio, variations in Global Fund
           expectations made it difficult to establish best practices for its
           LFA country teams. Likewise, several Global Fund officials noted
           that secretariat staff have conflicting ideas about LFA
           performance criteria. In one instance, according to Global Fund
           officials, a group of fund portfolio managers were unable to agree
           on what constituted a high-quality report. Fund portfolio managers
           and other key officials observed that more standardization of
           expectations would be beneficial, although some noted that this
           standardization should be balanced to allow them to maintain
           flexibility to account for country context.
		   
		   Global Fund Does Not Assess LFAs Systematically and Lacks Formal
		   Records of LFA Performance

           The Global Fund does not require formal, regular assessments of
           LFA performance. As a result, it lacks LFA performance records and
           cannot systematically track performance.
		   
		   LFA Performance Assessment and Reporting Are Informal and Irregular

           Although Global Fund officials regularly review the reports they
           receive from the LFAs and provide informal feedback to country
           teams, they assess LFA performance informally and at irregular
           intervals.34 All fund portfolio managers that we interviewed, as
           well as 11 of the 12 LFA country teams that participated in our
           structured interviews, said that fund portfolio managers interact
           informally and frequently with country team members, often orally
           or by e-mail. In addition, all but one country team said that fund
           portfolio managers provide informal feedback on their performance.
           For example, members of one team noted that they received feedback
           from their fund portfolio manager for each report and requested
           more information as necessary. In another case, the manager
           provided feedback to the LFA central team, which passed the
           feedback to the country team. A majority of the country teams
           reported receiving feedback from their fund portfolio manager when
           he or she traveled to the recipient country, at least once a year.
           Fund portfolio managers told us that if they continually encounter
           problems with a country team, they inform the Global Fund's LFA
           manager, who discusses the issue with the central team; if this
           does not resolve the problem, the contract may be terminated. A
           key Global Fund official stated that two to three LFA country team
           subcontracts are terminated each year based on information from
           fund portfolio managers on their poor performance.

           Although LFA central teams provide some performance feedback to
           their country teams, the scope and frequency of that feedback
           varies depending on the LFA's structure. Ten of the LFA country
           teams that participated in our structured interviews told us that
           LFA central teams provide informal feedback about their
           performance. The central team at two of the six LFA organizations
           reviews all products before they reach the secretariat. At two
           other LFA organizations, reports are reviewed only if it is deemed
           necessary, and in those cases, regional coordinators or mentors
           may review the reports.35 At the remaining two LFA organizations,
           the distinctions between the country and central teams are less
           clear, because the reports are at least partially written by
           central team members.
		   
		     Lack of Assessment Records Limits Available Information on LFA
			 Performance

           The lack of formal assessments limits the Global Fund's access to
           the information needed to manage LFAs' performance. The fund
           portfolio managers we interviewed said that their frequent
           interaction with country teams generally allows them to know which
           teams are performing well and which teams need improvement.
           However, Global Fund officials with LFA management responsibility
           stated that formalizing the process would allow the agency to
           establish LFA performance records and identify situations in which
           more oversight of LFAs by the Global Fund may be required. A
           working group of Global Fund experts echoed these concerns,
           reporting that despite extensive debate about the LFA model, the
           Global Fund had conducted little systematic assessment of the
           performance of individual LFA country teams.36
		   
		   Global Fund Previously Attempted to Implement Systematic LFA
		   Assessments

           A prior attempt to introduce a tool to make LFA assessments more
           systematic was not successful, because use of the tool was not
           mandatory and fund portfolio managers did not see it as a
           priority. According to the Global Fund, the LFA manager conducted
           a pilot of an evaluation checklist as a tool for fund portfolio
           managers to systematically assess LFA performance. Secretariat
           staff stated that, because Global Fund management did not require
           fund portfolio managers to complete the checklist, very few
           portfolio managers used it. As of early 2007, the LFA manager was
           developing a revised tool to routinely collect staff input on
           LFAs. However, it is too early to determine whether this tool,
           when finalized and made available to staff, will capture the
           information needed to assess LFA performance. According to the LFA
           manager, integrating this tool into existing tasks and making it
           an explicit component of fund portfolio managers' job descriptions
           would increase its use.
		   
		   LFA Performance and Capacity Reportedly Vary

           Although Global Fund officials cited instances in which LFA
           country teams performed adequately, secretariat staff, key
           partners, and outside experts expressed concerns about some LFAs'
           capacity to oversee grant recipients' programmatic and procurement
           activities. To illustrate adequate LFA capacity and performance,
           secretariat staff identified cases where LFA country teams have
           quickly identified problems with grants. For example, the majority
           of fund portfolio managers we interviewed noted that most LFA
           country teams performed financial oversight competently.37
           Similarly, a procurement officer described a country team that
           focused on supply management issues throughout the life of the
           grant and proved instrumental in resolving a large corruption
           problem. One fund portfolio manager described a country team that
           had enough knowledge of monitoring, evaluation, and procurement to
           understand an LFA subcontractor's initial assessment of recipient
           capacity and, therefore, could use this information in subsequent
           reporting about grantee progress.

           Conversely, secretariat staff also cited multiple instances in
           which LFAs demonstrated lack of capacity that affected the conduct
           of their oversight duties. In a recent Global Fund survey,
           secretariat employees who identified themselves as having
           sufficient exposure to LFAs to form an opinion reported mixed
           experiences regarding LFAs' technical program and monitoring
           expertise.38 About half of the respondents disagreed with the
           statement that LFAs with whom they had worked provided "adequate"
           expertise on health and program issues. The remainder was about
           equally divided between those who agreed that health and program
           expertise was adequate and those who neither agreed nor disagreed.
           Global Fund employees' assessments of the adequacy of LFAs'
           expertise for monitoring and evaluation were similarly mixed. The
           survey also provided evidence that the service quality varies
           greatly among countries: about three-quarters of the staff agreed
           with the statement that "within LFA firms there are substantial
           differences in service quality across countries."

           In addition, Global Fund partners and outside experts have raised
           concerns about LFAs' oversight capacity. UNAIDS officials said
           that they have serious doubts about the capacity of LFAs to
           monitor program progress, specifically noting that LFAs have
           little understanding of how to weigh and balance various
           performance targets that are not of equal importance.39 For
           example, according to these officials, an LFA that lacks strong
           technical program capacity might not recognize that a recipient
           exceeding a simple performance target, such as training staff, may
           be trying to compensate for poor performance in more difficult
           target areas, such as administering drugs to treat AIDS, TB, or
           malaria. Several other sources--a report commissioned by the
           Global Fund, our 2005 report, and several reports by an
           acknowledged expert on the Global Fund--also note problems with
           LFA capacity. (See app. II for a list of sources.)
		   
		   Conclusions

           Since our report in June 2005, the Global Fund has taken steps to
           improve operations at its secretariat in Geneva while it continues
           to confront complex issues facing grant implementation at the
           country level. The Global Fund will continue to face important
           challenges, including evaluating the effectiveness of its
           investment of the billions of dollars it receives annually from
           the United States and other donors and the impact of its programs
           over time. The Global Fund has improved the transparency of its
           periodic disbursement and grant renewal decisions by providing
           better documentation in its grant files. Delays in developing a
           systematic approach to assessing grant risk have left the Global
           Fund vulnerable to some of the risks associated with implementing
           grants in developing countries. However, the Global Fund board
           recognizes the importance of developing a more comprehensive
           approach to risk management, and the secretariat has recently
           drafted principles to guide its development of a risk framework
           for the organization. In addition, the Global Fund's lack of a
           formal process for setting expectations and assessing and
           analyzing LFA performance constrains its ability to determine the
           quality of the LFAs' services, track LFA performance over time,
           and address ongoing capacity issues. The lack of a formal
           assessment process also limits the information available to the
           Global Fund during its upcoming global retender of LFA contracts.

21Fund portfolio managers, with recommendations from LFAs, give
performance ratings that range from A (expected or exceeding
expectations), B1 (substantial achievements made), B2 (little progress
made), C (unacceptable).

22In preparation for the phase 2 grant renewal decision, Global Fund staff
prepare a summary of information from the grant recipient, LFA, portfolio
manager, performance evaluation team, and secretariat financial staff on
grant performance.

23The Global Fund calls such changes material reprogramming.

24All percentage estimates based on this sample provided in this report
have 95 percent confidence intervals of within plus or minus 13 percentage
points. See appendix I for more information on our sampling methodology.

25For both samples the estimate was 100 percent. For each of these
estimates we are 95 percent confident that at least 94 percent of all
files in our population contain narratives explaining the manager's rating
of the grant's performance.

26LFAs, portfolio managers, and evaluation staff make phase 2 funding
recommendations (Go, Conditional Go, or No Go). The Global Fund Board of
Directors reviews these recommendations and makes the final phase 2
recommendation to renew the grant, not to renew the grant, or to renew the
grant with conditions attached. Global Fund officials said that
differences between ratings and proposed funding levels are common and do
not necessarily indicate mistakes or poor judgment by LFAs, portfolio
managers, or staff from the evaluation team. They said, for example, that
after LFAs have completed their assessments, LFAs and fund portfolio
managers often receive new information about grant performance that leads
them to propose a different phase 2 amount than originally recommended by
the LFA.

27Fund portfolio managers and the evaluation staff gave the majority of
the phase 2 grants matching ratings. They concurred on about 73 percent of
their phase 2 grant renewal recommendations.

28We estimated that in almost half of grant disbursement files, the fund
portfolio manager disagreed with the amount recommended by the LFA or the
amount requested by the grant recipient.

29 [50]GAO-05-639 .

30According to international standards, risk assessment reduces
operational surprises and losses and enhances an organization's ability to
identify potential concerns and establish appropriate responses. See
Committee of Sponsoring Organizations of the Treadway Commission,
Enterprise Risk Management --Integrated Framework Executive Summary, 2004.

31 [51]GAO-05-639 .

32Global Fund officials gave us a sample LFA contract and said that its
contracts with the six LFAs were very similar.

33The Global Fund secretariat began piloting training courses for fund
portfolio managers in 2006 and told us that a module on LFAs would be
added to this training series in the future.

34Good practice for overseeing contractors suggests that contractors'
fulfillment of established expectations should be systematically measured.
See appendix II.

35Mentors are individuals from country teams that the central team has
identified as high performers.

36Challenges and Opportunities for the New Executive Director of the
Global Fund - Seven Essential Tasks" by Steve Radelet et.al. (Washington,
D.C.: Oct. 2006).

37Of the 12 country teams that participated in our structured interviews,
almost all team members had financial expertise but limited training in
procurement or monitoring and evaluation.

38The Global Fund drafted a questionnaire that a contractor tested and
administered anonymously in a Web-based survey from November 9 to 16, 2006
to the approximately 120 Geneva-based Global Fund secretariat staff who
use LFA services. The 75 staff who responded rated their agreement on more
than 50 statements about LFA-related issues using a five-point scale of
"Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly
Agree" while being given the option on most questions of not answering if
they had "Insufficient experience to form an opinion." We report responses
for the 51 operations staff, all of whom had direct contact with LFAs on a
range of issues.

39The Global Fund categorizes targets into three levels of complexity. The
first two levels are considered to be "process" indicators, and include
measures such as "people trained" or "service points supported." The
third, more complex category, denotes "coverage" indicators and measures
the number of individuals receiving services. The Global Fund is also
considering the most appropriate way to measure "impact" indicators.
		   
		   Recommendations for Executive Action

           To enhance Global Fund operations, we recommend that the
           Secretaries of State and HHS work with the Global Fund's Board
           Chair and Executive Director to take the following two actions:

           o establish standardized expectations for LFA performance, and

           o require systematic assessments of LFA performance and the
           collection and analysis of LFA performance data to improve the
           management and oversight of LFAs.
		   
		   Agency Comments

           Written comments on a draft of this report from the Departments of
           State and Health and Human Services and the U.S. Agency for
           International Development are represented in the unified response
           received from the Office of the U.S. Global AIDS Coordinator. The
           Global Fund also provided written comments on the draft. Both
           entities agreed with the report's findings, conclusions, and
           recommendations. The Office of the U.S. Global AIDS Coordinator
           stated that the report accurately reflected areas in which the
           Global Fund has made progress as well as areas in which more work
           remains to be done. The Office's response noted that the United
           States will urge the Board to adopt concrete recommendations from
           several studies to improve the accountability and transparency of
           Global Fund activities and will work with the Global Fund to
           improve its oversight and risk assessment mechanisms. The Global
           Fund stated that the report reinforced the importance of measures
           it has begun taking to better measure local fund agent
           performance. Regarding risk assessment, the Global Fund emphasized
           the role of its country-based and global partners in helping to
           track grant implementation and provide technical support as
           needed. (See apps. III and IV for their comments.)

           We are sending copies of this report to interested congressional
           committees and will make copies available to others on request. In
           addition, this report is available on GAO's Web site at
           http://www.gao.gov .

           If you or your staff have any questions, please contact me at
           (202) 512-3149 or [email protected] . Contact points for
           Offices of Congressional Relations and Public Affairs may be found
           on the last page of this report. Key contributors to this report
           are listed in appendix V.

           David M. Gootnick
		   Director, International Affairs and Trade
		   
		   Appendix I: Objectives, Scope, and Methodology

           To assess the Global Fund's efforts to improve documentation of
           performance-based-funding decisions, we analyzed the forms used to
           collect data on grant performance and discussed with Global Fund
           officials how the forms have changed since GAO's 2005 report. We
           met with officials from all of the entities that have current
           contracts to serve as local fund agents and with officials from
           the Office of the Global AIDS Coordinator, UNAIDS, and the World
           Bank---the Global Fund's key technical partners--to discuss how
           documentation of the Global Fund's decisions has changed and
           whether it has improved.

           To review information in the grant files used to support
           performance-based funding decisions, we analyzed two separate
           simple random probability samples of grant funding decisions using
           a separate data collection instrument for each sample. The first
           sample consisted of 80 of the 348 grants that the Global Fund
           assessed for performance-based funding disbursements between
           January 1 and October 1, 2006. The second sample consisted of 45
           of the 65 grants that the Global Fund assessed for phase 2 grant
           renewal during the same time period.1 We chose January 2006 as the
           starting point for our sample because Global Fund officials told
           us that their new disbursement decision worksheet--created to
           address GAO's recommendation that they better document their
           decisions--was fully implemented by the end of 2005. With our two
           probability samples, each member of the study population had a
           nonzero probability of being included, and that probability could
           be computed for any member. Each sample case file was subsequently
           weighted in the analysis to account statistically for all the case
           files in the population, including the files that were not
           selected.

           Because we followed a probability procedure based on random
           selections, our samples are only one of a large number of samples
           that we might have drawn (for both the grant disbursement and
           phase 2 file reviews). Since each sample could have provided
           different estimates, we express our confidence in the precision of
           our particular sample's results as 95 percent confidence
           intervals. These are intervals that would contain the actual
           population values for 95 percent of the samples we could have
           drawn. All percentage estimates provided in this report based on
           our samples have 95 percent confidence intervals of within plus or
           minus 13 percentage points.

           For each grant in our disbursement sample, we requested the grant
           recipient's Progress Update and Disbursement Request report, the
           Local Fund Agent Recommendation report, and the Global Fund's
           Disbursement Decision-Making Worksheet. For the phase 2 decision
           sample, we requested the following documents for each grant:

           o Phase 2 Grant Scorecard (the in-house version that goes to the
           panel)
           o Phase 2 Grant Scorecard (the panel's version that goes to the
           board)
           o Documentation of the board's decision with any explanation or
           justification of the decision (whether this is in the form of a
           scorecard or some other format)
           o Phase 2 Panel Minutes
           o Board Notes
           o Grant Performance Report
           o Annex A Attachments 2 and 3
           o Local fund agent Phase 2 Assessment Report

           We gathered the information from the documents for our data
           collection instruments and verified the data that we used to
           support our findings.

           Our file review consisted of a number of checks on the presence or
           absence of an array of qualitative and contextual information that
           the Global Fund forms specify as important for grant
           decision-making and that therefore should be provided by those
           responsible for filling them out. We did not evaluate the
           appropriateness or accuracy of the judgments made on the basis of
           the narrative or contextual information provided in the documents;
           we focused on the issue of whether the contextual or narrative
           information provided was relevant to particular judgments or
           decisions.2

           In our analysis of the disbursement files, while we examined
           information contained in other documents, we focused mostly on the
           Disbursement Decision-Making Form, due to the fact that this was
           the key form that the Global Fund instituted for
           disbursement-related decision-making since our last report. In
           this form, we checked for the presence or absence of the following
           pieces of information:

           o Fund portfolio manager/cluster explanation for differences in
           fund portfolio manager/cluster and LFA ratings
           o Fund portfolio manager's explanation for performance rating
           o Fund portfolio manager's explanation for differences in funding
           decision with the principal recipient request and LFA
           recommendation
           o Fund portfolio manager's check-offs on an array of grant
           decision-making issues such as performance evaluation, financial
           considerations, future and outstanding requirements, etc.
           o Fund portfolio manager's explanation for array of contextual
           issues (e.g., governance, general management, and external
           factors)
           o Fund portfolio manager's explanation for required documents and
           conditions precedent items

           In the phase 2 files we focused on the Grant Scorecard, as this is
           the key Global Fund decision-making document for this phase,
           although we performed checks on several other documents as well.
           We checked for the presence or absence of the following pieces of
           information in the following phase 2-related documents; all items
           are for the Grant Scorecard unless otherwise noted:

           o Evaluation unit's explanation for nonconcurrence with fund
           portfolio manager/cluster rating

           o General explanation for difference between fund portfolio
           manager/cluster recommendation and the LFA's recommendation
           o Evaluation unit's explanation for difference between fund
           portfolio manager/cluster recommendation and the LFA's
           recommendation
           o General explanation for difference in the recommended amounts
           between Global Fund financial review and cluster leader phase 2
           o General explanation for difference in the recommended amounts
           between LFA and fund portfolio manager/cluster leader (i.e. the
           Global Fund) phase 2
           o Fund portfolio manager/cluster identification of material issues
           with reliability of principal recipient's data

           o Fund portfolio manager/cluster explanation for phase 2 rating

           o Fund portfolio manager/cluster explanation for phase 2
           recommendation
           o (LFA phase 2 assessment) report-specific conditions identified
           by LFA for `Conditional Go' recommendations
           o Time-bound actions identified by portfolio manager/cluster for
           `Conditional Go' recommendations

           To assess the Global Fund's progress in implementing a risk
           assessment model and early warning systems for its grants, we
           followed up on the risk assessment model and early warning system
           described in our 2005 report with Global Fund officials. In
           addition, we reviewed international standards for risk that
           establish the criteria for assessing risk developed by the
           Committee on Sponsoring Organizations of the Treadway Commission.
           We also discussed the need for a risk assessment model and early
           warning system with key Global Fund staff, key Global Fund
           partners, UNAIDS, World Bank, and OGAC, and with the Global Fund
           Inspector General and LFA officials. In addition, we reviewed
           Global Fund board notes and the Global Fund's draft outline of
           Guiding Principles for a Risk Management Framework. In their
           technical review of our draft, Global Fund officials clarified the
           distinction between the risk assessment model that was not
           implemented and the current early alert and response system, and
           they provided additional information on the current status of the
           early alert and response system. We modified the risk assessment
           section of the draft based on our analysis of this information.

           To assess how the Global Fund oversees and measures the capacity
           and performance of local fund agents, we established criteria for
           good practices in performance assessment, consulted key reports
           and documents, met with relevant personnel, conducted structured
           interviews with LFA country teams, and analyzed the structured
           interview data using a content analysis framework. We consulted
           GAO experts in performance assessment and contracting, good
           practice documents from the Government Performance and Results
           Act,3 performance-based contracting literature, internal control
           literature, and training development literature. (See app. II) To
           understand Global Fund policies and requirements regarding LFA
           performance, we reviewed Global Fund documentation, including
           guidelines for LFA work, the Global Fund's operational policy
           manual, LFA manager terms of work, and LFA framework contract
           forms, as well as relevant evaluative reports published by the
           Global Fund and development experts. We meet with key Global Fund
           officials in Geneva, Switzerland, including the fund portfolio
           managers and members of the Portfolio Services and Projects Team,
           representatives from all LFA central teams, selected LFA
           subcontractors, and recognized experts on the Global Fund to
           discuss the current practices and potential difficulties of the
           LFA system. We also reviewed the methodology and results of a
           survey of Geneva-based Global Fund secretariat staff who are users
           of LFAs conducted by a Global Fund contractor in November of 2006.

           Additionally, to support our review of issues related to LFA
           capacity and performance, we conducted structured telephone
           interviews with a nonprobability sample of 12 LFA country teams
           out of a total of 102 LFA country teams worldwide.4 We selected
           country teams for this sample based on criteria such as the size
           and amount of the grants that the LFA oversaw, whether the LFA was
           located within the country, and whether the LFA acted as a mentor
           to other LFA country teams. The purpose of these 12 interviews was
           to obtain information on the views of these important participants
           in the Global Fund process; we did not obtain a representative
           perspective of the views of all 102 country teams. We pre-tested
           our questions with two of our initial respondents and refined our
           questions based on their input. We followed up via e-mail with our
           respondents to obtain supplementary information. To summarize the
           open-ended responses and developed categories for the analysis, we
           first grouped interview responses by general categories, based on
           our objectives, criteria, and inductive observation. Within each
           of these general categories, we then further grouped the data into
           subcategories based on the emerging themes. To ensure the validity
           and reliability of our analysis, these subcategories were reviewed
           by a methodologist, who proposed modifications. After discussion
           of these suggestions, we determined a final set of subcategories.
           An analyst and a methodologist independently tallied the number of
           respondents providing information in each subcategory, compared
           their tallies, and reconciled any variation.
		   
1We did not consider grants for the disbursement sample that received
their first disbursement of funds during the period so our results were
not skewed by problems that commonly accompany grants when they start.

2A limitation of our analysis was that we did not evaluate the adequacy of
the information provided, as determining adequacy would have required us
to make judgments requiring substantial additional grant-specific
information not contained in the grant files.

3Government Performance and Results Act of 1993, Pub. L. No. 103-62, 107
Stat. 285.

4Results from our nonprobability sample cannot be used to generalize to
the overall population of LFA country teams because, in a nonprobability
sample, some elements of the population being studied have no chance or an
unknown chance of being selected as part of the sample.
		   
		   Appendix II: Other Sources Cited
		   
		   Principles for Contractor Management

           o GAO, Strategic Issues Web pages on the Government Performance
           and Results Act (GPRA)
           o GAO, Internal Control Management and Evaluation Tool.
           [38]GAO-01-1008G . Washington D.C.: Aug. 2001.
           o Office of Federal Procurement Policy, Seven Steps to
           Performance-Based Services Acquisition, see:
           www.acqnet.gov/Library/OFPP/BestPractices/pbsc 
           o GAO, Contract Management: Guidance Needed for Using
           Performance-based Service Contracting. [40]GAO-02-1049 .
           Washington, D.C.: Sept. 23, 2002.
		   
		   Analyses of LFA Capacity and Performance

           o GAO, Contract Management: Guidance Needed for Using
           Performance-Based Service Contracting. [41]GAO-02-1049 .
           Washington, D.C.: 2002.
           o GAO, Internal Control Management and Evaluation Tool.
           [42]GAO-01-1008G . Washington, D.C.: 2001.
           o Kruse, Stein-Erik, and Jens Claussen. Review of the Roles,
           Functions and Performance of Local Fund Agents. Centre for Health
           and Social Development and Nordic Consulting Group; 2004.
           o Office of Federal Procurement Policy, Seven Steps to
           Performance-Based Services Acquisition. See:
           www.acqnet.gov/Library/OFPP/BestPractices/pbsc

           o Radelet, Steven et al. Challenges and Opportunities for the New
           Executive Director of the Global Fund: Seven Essential Tasks.
           Global Fund Working Group, Center for Global Development,
           Washington D.C.: October 2006.
           o Radelet, Steven. The Global Fund to Fight AIDS, TB, and Malaria:
           Progress, Potential, and Challenges for the Future. Center for
           Global Development, Washington D.C.: 2004.
		   
		   Appendix III: Comments from the Global Fund
		   
		   Appendix IV: Comments from Departments of State and Health and
		   Human Services and U.S. Agency for International Development
		   
		   Appendix V: GAO Contact and Staff Acknowledgments
		   
		   GAO Contact

           David Gootnick (202) 512-3140
		   
		   Staff Acknowledgments

           In addition to the person named above, Audrey Solis, Assistant
           Director; Tom Zingale; Jeffrey Baldwin-Bott; Bruce Kutnick;
           Barbara Steel-Lowney; David Dornisch; and Reid Lowe made key
           contributions to this report. Kathryn Bernet, Antoine Clark, Etana
           Finkler, Hynek Kalkus, and Tracy Guerrero provided technical
           support.
		   
		   GAOï¿½s Mission

           The Government Accountability Office, the audit, evaluation and
           investigative arm of Congress, exists to support Congress in
           meeting its constitutional responsibilities and to help improve
           the performance and accountability of the federal government for
           the American people. GAO examines the use of public funds;
           evaluates federal programs and policies; and provides analyses,
           recommendations, and other assistance to help Congress make
           informed oversight, policy, and funding decisions. GAO's
           commitment to good government is reflected in its core values of
           accountability, integrity, and reliability.
		   
		   Obtaining Copies of GAO Reports and Testimony

           The fastest and easiest way to obtain copies of GAO documents at
           no cost is through GAO's Web site ( www.gao.gov ). Each
           weekday, GAO posts newly released reports, testimony, and
           correspondence on its Web site. To have GAO e-mail you a list of
           newly posted products every afternoon, go to www.gao.gov and
           select "Subscribe to Updates."
		   
		   Order by Mail or Phone

           The first copy of each printed report is free. Additional copies
           are $2 each. A check or money order should be made out to the
           Superintendent of Documents. GAO also accepts VISA and Mastercard.
           Orders for 100 or more copies mailed to a single address are
           discounted 25 percent. Orders should be sent to:

           U.S. Government Accountability Office 441 G Street NW, Room LM
           Washington, D.C. 20548

           To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
           (202) 512-6061
		   
		   To Report Fraud, Waste, and Abuse in Federal Programs

           Contact:

           Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
           [email protected] Automated answering system: (800) 424-5454 or
           (202) 512-7470
		   
		   Congressional Relations

           Gloria Jarmon, Managing Director, [email protected] (202)
           512-4400 U.S. Government Accountability Office, 441 G Street NW,
           Room 7125 Washington, D.C. 20548
		   
		   Public Affairs

           Paul Anderson, Managing Director, [email protected] (202)
           512-4800 U.S. Government Accountability Office, 441 G Street NW,
           Room 7149 Washington, D.C. 20548

(320416)

www.gao.gov/cgi-bin/getrpt?GAO-07-627 .

To view the full product, including the scope
and methodology, click on the link above.

For more information, contact David Gootnick at (202) 512-3149
([email protected]).

Highlights of [53]GAO-07-627 , a report to congressional committees

May 2007

GLOBAL HEALTH

Global Fund to Fight AIDS, TB and Malaria Has Improved Its Documentation
of Funding Decisions but Needs Standardized Oversight Expectations and
Assessments

The Global Fund to Fight AIDS, Tuberculosis and Malaria has approved about
$7 billion in grants to developing countries; the U.S. has contributed
$1.9 billion. The State Department's Office of the Global AIDS Coordinator
(OGAC) coordinates the U.S. government's overseas AIDS programs, with
participation from the Department of Health and Human Services (HHS) and
the U.S. Agency for International Development (USAID). In 2003, Congress
directed GAO to report on the Global Fund every 2 years. This report
assesses the Global Fund's (1) documentation of information used to
support performance-based funding decisions, (2) progress in implementing
a risk assessment model and early warning system, and (3) oversight of the
performance of "local fund agents" (LFAs), which monitor grant progress in
recipient countries. GAO reviewed the documentation for funding decisions
and interviewed key officials.

[54]What GAO Recommends

We recommend that the Secretaries of State and Health and Human Services
work with the Global Fund's Board Chair and Executive Director to (1)
establish standardized expectations for LFA performance and (2) require
systematic assessments of LFA performance and the collection and analysis
of performance data. The Departments of State and HHS, USAID, and the
Global Fund agreed with our findings and recommendations.

Since our 2005 review, the Global Fund has improved its documentation for
decisions to disburse funds and renew grants. The Global Fund now requires
that fund portfolio managers more consistently document factors, such as
grant ratings and contextual information that support disbursement and
grant renewal decisions. Our current review of 80 grant disbursements and
45 grant renewal decisions confirmed that Global Fund grant files
consistently contained explanations of the information used in its funding
decisions. For example, all grant disbursement files in our sample
contained a written narrative explaining the ratings that portfolio
managers gave the grants, based in part on reports completed by grant
recipients. Although we noted that many of the grant recipients' reports
lacked some information needed for disbursement and renewal decisions,
Global Fund officials said that portfolio managers obtain this information
informally from the recipients or other stakeholders.

The Global Fund did not implement the risk assessment model that it was
developing at the time of our 2005 report, because it determined that the
model did not accurately identify grant risk. To identify risks that may
affect grant implementation, the organization currently relies on elements
of its structures and processes, including its initial technical review,
disbursement decision-making form, periodic grant ratings, oversight by
country stakeholders, information from technical partners, and LFA
oversight. Recognizing the need for a more comprehensive approach to risk
management, the Global Fund has begun developing a risk assessment
framework for the organization that includes an early alert and response
system to address poorly performing grants.

The Global Fund has limited access to the information it needs to manage
and oversee LFAs because it does not require systematic assessments of
LFAs' performance. As a result, the Global Fund has limited ability to
determine the quality of LFAs' monitoring and reporting and to identify
situations in which more oversight of LFAs' performance may be required.
Previously, the Global Fund introduced a tool to assess LFA performance
more systematically; however, this effort was unsuccessful, because use of
the tool was not mandatory. Numerous sources raise concerns about the
quality of grant monitoring and reporting provided by LFAs, particularly
their ability to assess and verify recipients' procurement capacity and
program implementation.

References

Visible links
  34. http://www.gao.gov/cgi-bin/getrpt?GAO-05-639
  35. http://www.gao.gov/cgi-bin/getrpt?GAO-03-601
  38. http://www.gao.gov/cgi-bin/getrpt?GAO-01-1008G
  40. http://www.gao.gov/cgi-bin/getrpt?GAO-02-1049
  41. http://www.gao.gov/cgi-bin/getrpt?GAO-02-1049
  42. http://www.gao.gov/cgi-bin/getrpt?GAO-01-1008G
  50. http://www.gao.gov/cgi-bin/getrpt?GAO-05-639
  51. http://www.gao.gov/cgi-bin/getrpt?GAO-05-639
  53. http://www.gao.gov/cgi-bin/getrpt?GAO-07-627
*** End of document. ***