Foreign Assistance: USAID's Reengineering at Overseas Missions (Letter
Report, 09/12/97, GAO/NSIAD-97-194).
Pursuant to a congressional request, GAO reviewed the effect of the U.S.
Agency for International Development's (USAID) reforms on its overseas
missions' operations and delivery of assistance, focusing on how: (1)
USAID missions have reengineered their operations; (2) reengineering has
affected the content of USAID's assistance program; (3) USAID monitors
and evaluates the results of its projects; (4) USAID allocates funds for
its projects; and (5) USAID's New Management System supports mission
operations.
GAO noted that: (1) in reengineering the way assistance is delivered,
USAID missions have only recently made the investment and operational
changes necessary to bring about long-term change; (2) to date, USAID's
reengineering efforts to improve its delivery of assistance have shown
some benefits in the areas of planning, implementing, and monitoring
projects; (3) notwithstanding this progress, USAID still has major
obstacles to overcome in deploying its New Management System and
establishing valid and reliable performance measures; (4) overseas
missions have changed the way they manage assistance in support of
USAID's strategic goals; (5) however, because missions had no baseline
data on operations management prior to reengineering their processes,
neither they nor GAO can identify measurable increases in efficiency or
effectiveness of the delivery of assistance; (6) USAID missions have, to
varying degrees, begun to establish more results-oriented indicators and
report the results of their projects annually; (7) however, the missions
still have difficulty linking their activities to the broad indicators
of development--such as a country's rate of economic growth; (8) one way
to provide a more complete picture of program performance could be to
supplement performance measurement data with impact evaluation studies;
(9) although the relative performance of mission programs is clearly a
factor in USAID's resource allocation decisions, these decisions are
still largely driven by other considerations, such as contributions to
foreign policy and agency priorities, country need and commitment, and
funding priorities of the executive branch and Congress; (10) USAID's
New Management System, one of the agency's key tools in reforming its
operations, is not working as intended; (11) this computer system, which
is expected to cost at least $100 million by the end of fiscal year
1998, was designed to enable the agency to manage its resources and
monitor results more effectively by consolidating accounting, budgeting,
personnel, procurement, and program operations into a single, integrated
network that can be accessed worldwide; (12) despite warnings that the
system had not been tested and did not meet basic federal requirements,
USAID activated the system in July 1996 in Washington and deployed it in
October 1996 in its missions; (13) USAID suspended much of the system's
operation in April 1997, after it failed to work properly; and (14)
correcting system deficiencies will be critical to continued progress of
the agency's reform effort.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: NSIAD-97-194
TITLE: Foreign Assistance: USAID's Reengineering at Overseas
Missions
DATE: 09/12/97
SUBJECT: Federal aid to foreign countries
Reengineering (management)
Management information systems
Public administration
Systems conversions
Strategic planning
Human resources utilization
International cooperation
Information resources management
IDENTIFIER: El Salvador
Honduras
Bangladesh
Philippines
AID New Management System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to the Chairman, Committee on International Relations, House
of Representatives
September 1997
FOREIGN ASSISTANCE - USAID'S
REENGINEERING AT OVERSEAS MISSIONS
GAO/NSIAD-97-194
Foreign Assistance
(711231)
Abbreviations
=============================================================== ABBREV
CMM - Capability Maturity Model
GPRA - Government Performance and Results Act
NMS - New Management System
USAID - U.S. Agency for International Development
Letter
=============================================================== LETTER
B-277413
September 12, 1997
The Honorable Benjamin A. Gilman
Chairman, Committee on International Relations
House of Representatives
Dear Mr. Chairman:
In response to your request, we have reviewed the effect of the U.S.
Agency for International Development's (USAID) reforms on its
overseas missions' operations and delivery of assistance.
Specifically, we focused on (1) how USAID missions have reengineered
their operations, (2) how reengineering has affected the content of
USAID's assistance program, (3) how USAID monitors and evaluates the
results of its projects,\1 (4) how USAID allocates funds for its
projects, and (5) how USAID's New Management System supports mission
operations. In doing this review, we visited missions in El
Salvador, Honduras, Bangladesh, and the Philippines.
This is the second of a two-part review of USAID's reform efforts.
In September 1996, we reported that USAID had adopted a development
strategy with priority areas of concentration, reorganized its
headquarters units, closed some overseas missions, cut its total
workforce, developed an information resource management system for
its business and service functions, and had begun to reengineer its
procurement and overseas mission operations.\2
--------------------
\1 For consistency and simplicity, we use the term "project" when
referring to individual USAID development efforts. As part of its
reengineering efforts, USAID is now referring to "projects" as
"activities" that support its strategic objectives. USAID reports to
Congress by project and activity.
\2 See Foreign Assistance: Status of USAID's Reforms
(GAO/NSIAD-96-241BR, Sept. 24, 1996).
BACKGROUND
------------------------------------------------------------ Letter :1
USAID has provided foreign aid since 1961 in an effort to improve
countries' economies, health, environment, and democratic processes.
In times of crisis, the agency has also provided humanitarian
assistance to those in need. USAID currently has 72 overseas
missions and offices that manage projects associated with this
foreign assistance. These projects are generally implemented by
private voluntary organizations, nongovernmental organizations,
international agencies, universities, and contractors.
Since the end of the Cold War, Congress has reduced USAID's
appropriations, citing other funding priorities as well as basic
questions about the effectiveness of foreign aid. USAID's fiscal
year 1996 obligations of $5.7 billion were 13 percent less than those
in the previous year (see app. I for information on USAID
obligations for fiscal years 1995 through 1997). To accommodate
budget reductions, USAID cut its total workforce from 11,150 in
September 1993 to 7,609 in June 1997 and will have closed 24 missions
by the end of fiscal year 1997. Under the executive branch's
decision to consolidate the Department of State, the Arms Control and
Disarmament Agency, and the U.S. Information Agency, the USAID
Administrator will report directly to the Secretary of State rather
than, as previously, to the President.
Congress specifically directs how USAID spends much of its funding,
earmarking substantial amounts of USAID's direct appropriation for
development assistance for specific purposes, such as child survival.
Congress also earmarks funds for economic support and food
assistance, which are administered by USAID. Congressional earmarks
and executive branch directives (e.g., for the population program)
accounted for 59.8 percent of development assistance funds in fiscal
year 1995, 66.5 percent in fiscal year 1996, and 69.5 percent for
fiscal year 1997 (see app. II for more details on directed and
undirected assistance).
In 1993, in response to criticisms about USAID's inefficient
processes and inability to demonstrate a significant impact on
developing countries and some calls for the agency's abolition,
USAID's Administrator volunteered the entire agency as a "reinvention
laboratory." USAID established five strategic goals to meet its
agency mission of pursuing sustainable development in developing
countries: (1) achieving broad-based economic growth, (2) building
democracy, (3) stabilizing world population and protecting human
health, (4) protecting the environment, and (5) providing
humanitarian assistance.
According to USAID, reengineering the agency's overseas operations
has revolved around (1) increasing its customer focus, (2) managing
for results, (3) enhancing staff participation and teamwork, (4)
empowering and increasing the accountability of staff, and (5)
valuing diversity. The reengineering included reorganizing mission
organizational structures, eliminating unnecessary administrative
requirements, and including the participation of development partners
and recipients in program planning.
In 1994, USAID selected 10 missions to test these reengineering
principles, concepts, and approaches prior to deployment worldwide.
Based on observations of the experiments and feedback from the
participating missions, USAID initiated full implementation of its
reengineering approach on October 1, 1995. In fiscal year 1996,
USAID deployed the New Management System, which was intended to
provide an integrated database of financial and program information
to facilitate the management of resources and monitoring of results.
(See app. III for a chronology of USAID reforms.)
USAID's Administrator has noted that the agency's reforms are
consistent with the Government Performance and Results Act of 1993
(GPRA) and have positioned USAID well to meet the act's requirements
for strategic planning and performance monitoring.\3
--------------------
\3 We recently issued a report analyzing USAID's draft strategic plan
and its compliance with GPRA. See The Results Act: Observations on
USAID's November 1996 Draft Strategic Plan (GAO/NSIAD-97-197R, July
11, 1997).
RESULTS IN BRIEF
------------------------------------------------------------ Letter :2
In reengineering the way assistance is delivered, USAID missions have
only recently made the investment and operational changes necessary
to bring about long-term change. As our previous work has shown,
reengineering efforts of the magnitude of USAID's can require years
of steady work to yield intended benefits. Thus, while the agency
has built momentum for change, the efforts have not yet led to a
fundamental change in the substance of its program, and USAID cannot
yet demonstrate that it has made significant progress in achieving
its strategic goals. Additional important factors such as foreign
policy objectives and congressional earmarks will continue to
significantly influence the allocation of USAID's resources and
affect its progress in changing its program. To date, USAID's
reengineering efforts to improve its delivery of assistance have
shown some benefits in the areas of planning, implementing, and
monitoring projects. Notwithstanding this progress, USAID still has
major obstacles to overcome in deploying its New Management System
and establishing valid and reliable performance measures.
Overseas missions have changed the way they manage assistance in
support of USAID's strategic goals. USAID has increased the
authority and flexibility of missions to plan, design, and implement
their activities, and mission directors have delegated much of their
authority to staff teams. These teams have fostered collaboration
among mission staff, development partners, and recipients of
assistance in designing and implementing development activities.
However, because missions had no baseline data on operations
management prior to reengineering their processes, neither they nor
we can identify measurable increases in efficiency or effectiveness
of the delivery of assistance. Nonetheless, mission officials were
optimistic about the potential impact of their changes in operations.
USAID missions have, to varying degrees, begun to establish more
results-oriented indicators and report the results of their projects
annually. However, the missions still have difficulty linking their
activities to the broad indicators of development--such as a
country's rate of economic growth. Since many other factors outside
USAID's control have a significant impact on these indicators, USAID
cannot easily measure how effectively its programs are achieving the
agency's strategic goals. One way to provide a more complete picture
of program performance could be to supplement performance measurement
data with impact evaluation studies.
Although the relative performance of mission programs is clearly a
factor in USAID's resource allocation decisions, these decisions are
still largely driven by other considerations, such as contributions
to foreign policy and agency priorities, country need and commitment,
and funding priorities of the executive branch and Congress. The
decisions to provide funds for Haitian democratization efforts and to
support the Guatemalan peace process, for example, were made
independently of performance considerations.
USAID's New Management System, one of the agency's key tools in
reforming its operations, is not working as intended. This computer
system, which is expected to cost at least $100 million by the end of
fiscal year 1998, was designed to enable the agency to manage its
resources and monitor results more effectively by consolidating
accounting, budgeting, personnel, procurement, and program operations
into a single, integrated network that can be accessed worldwide.
Despite warnings that the system had not been tested and did not meet
basic federal requirements, USAID activated the system in July 1996
in Washington and deployed it in October 1996 in its missions. USAID
suspended much of the system's operation in April 1997, after it
failed to work properly. Correcting system deficiencies will be
critical to continued progress of the agency's reform effort.
MISSIONS ARE REENGINEERING
THEIR OPERATIONS IN LINE WITH
USAID REFORMS
------------------------------------------------------------ Letter :3
Overseas missions have changed their processes to streamline the way
they provide assistance in support of USAID's strategic goals. These
changes include taking a team approach to management, accepting
increased accountability through the authority and flexibility given
them by USAID, and increasing the participation of development
partners in the design and implementation of projects. USAID has
supported the missions' reform efforts with information and advice.
Mission officials believe that these changes will result in more
effective assistance programs. Moreover, our review indicated that
the changes show promise that USAID is making progress in better
managing the process of providing assistance.
We could not fully assess the impact of the missions' operational
changes for the following reasons: (1) missions are at varying
stages of implementing changes; (2) the impact of these changes may
not be seen for several years, especially those that affect the
design of new projects; and (3) the missions we visited had not
established baseline information on the efficiency or effectiveness
of their management operations before reengineering their processes
and therefore could not demonstrate many measurable improvements. In
some missions, officials indicated that some baseline data could be
compiled from historical records but that doing so would demand an
inordinate amount of scarce mission resources.
MISSIONS HAVE ESTABLISHED
TEAM STRUCTURES
---------------------------------------------------------- Letter :3.1
Overseas missions we visited have reengineered their organizational
structure to focus on achieving strategic objectives. Most missions
have replaced their former office structure with a team approach to
management, and each team is responsible for one of the mission's
strategic objectives. Team members at the missions we visited
commented that the team approach has improved the management of their
activities. USAID said that overseas missions have observed that
teams are now more likely to (1) remain focused, (2) rethink tactics
quickly when activities do not go as planned, and (3) terminate
activities that only marginally contribute to results.
Prior to reengineering, missions had several offices: technical
offices, such as the education, health, and agriculture offices,
which had day-to-day responsibility for implementing assistance
projects, and support offices, such as the contracts, controller, and
legal adviser's offices, which assisted the technical offices.
According to mission staff, in the past, each of these offices had
its own management hierarchy and separately reviewed proposed actions
such as expenditure requests or new project starts. This review was
often done sequentially, and as each office raised problems or
issues, action was delayed until they could be resolved; the proposal
was then passed to the next office for review.
The four missions that we visited had reorganized their offices into
teams to manage assistance activities that support each mission's
strategic objectives. Staff members that previously worked for
different technical and support offices now work together on a team
toward a specific common objective. In most missions, the teams
replaced much of the traditional office structure. For example, the
mission in El Salvador combined the staff of its economic, education,
and productive resources offices into a single economic growth
strategic objective team, which also includes members from the
contracts and controller offices.
At the missions we visited, both technical and support staff said
that issues that had previously delayed progress under the old
process can now be resolved within the teams. For example, contract
office staff are now involved in the early stages of project design
and implementation, suggesting appropriate procurement instruments
and helping draft scopes of work for proposed contracts. According
to contract officers, this approach has reduced the number of
sequential reviews and rewrites of proposals and the tensions that
have historically existed between staff and technical offices.
MISSION MANAGEMENT AND STAFF
HAVE MORE AUTHORITY AND
FLEXIBILITY
---------------------------------------------------------- Letter :3.2
Under reengineering, USAID increased the authority and flexibility of
missions in managing their activities by eliminating many
administrative requirements previously imposed on the missions. For
example, USAID eliminated the requirement for missions to prepare
elaborate project papers and obtain bureau approval for new projects.
Once a regional bureau approves a mission's strategic objectives, the
mission is authorized to develop activities to meet those objectives
without the bureau's further review and approval. USAID also
eliminated the 10-year time limit on all projects, allowing the
missions the flexibility to extend projects when it is advantageous
to do so. For example, in Honduras, the mission extended an ongoing
policy project in order to continue to provide high-level policy
advice to the newly elected Honduran administration. Before
reengineering, the mission would have had to design and authorize a
completely new project or apply for a waiver from USAID to extend the
project.
Furthermore, USAID delegated to the missions a host of executive
authorities previously held by officials at USAID headquarters. For
example, missions may now independently issue implementation letters
and negotiate and implement agreements with other U.S. government
agencies.
In accordance with the agency's emphasis on staff empowerment and the
assignment of responsibility and accountability under reengineering,
mission directors delegated new authorities to their missions'
strategic objective teams. For example, the mission director in
Honduras authorized team leaders to approve individual expenditures
of up to $100,000 in program funds without her review.
MISSIONS ARE IMPLEMENTING A
SYSTEM TO REPORT RESULTS OF
ACTIVITIES
---------------------------------------------------------- Letter :3.3
In the past, USAID was more concerned with the required inputs and
expected outputs of a particular project than the results to be
achieved. Projects had individual purposes that often related to an
overall goal or goals in only a very general way, and the goals
themselves were in broad categories rather than specific objectives
against which progress could be measured. Also, missions reported on
their activities in a variety of reports, including semiannual
reviews and project implementation reviews, that did not necessarily
focus on development results but rather focused on the status of
project implementation.
USAID is increasingly holding missions accountable for results. In
fiscal year 1996, it began requiring the missions to report annually
on the results of their programs in a report called the Results
Review and Resource Request (R4). Missions must now report to
high-level management on their progress in meeting agreed-upon
strategic objectives and agency goals and quantify that progress
through performance indicators developed by the missions, often with
assistance from USAID's Center for Development Information and
Evaluation. Missions are to supplement the data in the annual R4
document with analyses of performance data and other evidence that
the mission is making progress in meeting its strategic objectives.
PARTICIPATION OF DEVELOPMENT
PARTNERS HAS INCREASED
---------------------------------------------------------- Letter :3.4
USAID has given its development partners\4 a more active role in
developing country strategies and designing and managing projects.
Many of these partners were enthusiastic about this change. In some
cases, the increased interaction has accelerated the design and
management process. Also, with input from customers,\5 the missions
believe they have been better able to respond to a country's needs
and develop a strategy for meeting those needs.
Missions previously presented completed project designs to host
countries for acceptance. Today, however, the missions invite the
host country, other development partners, and customers to
participate in planning and designing activities. For example,
according to several representatives from nongovernmental
organizations in El Salvador, the mission has invited them to its
planning and designing meetings, and their input was included in the
mission's development strategy. In Bangladesh, the mission brought
together the customers and development partners to redesign a health
and population program that for the first time combines services in
both family planning and maternal child health care.
Missions also include development partners as members of strategic
objective teams. For example, Bangladesh government officials from
the Rural Electrification Board told us that they actively
participated with USAID team members, contractor staff, and
representatives from local rural electric cooperatives in designing a
rural power for the poor project and selecting the indicators by
which results will be measured.
Many nongovernment organizations and donors welcomed the missions'
new collaborative approach to designing and implementing activities
and were very positive about the missions' interaction with them.
The benefit of this increased interaction can be seen in Bangladesh,
for example, where the democracy strategic objective team, working
with nongovernmental organizations and staff offices, designed a new
project in 6 months compared to the 2 years that project designs had
generally required before.
Some donors have also observed increased cooperation with USAID
missions. In Bangladesh, an official of the British Office of
Development Assistance said that the mission was instrumental in
influencing the host government's development of a national health
and population strategy. Increased cooperation with other donors was
not always evident at the missions we visited, however. For example,
in El Salvador, mission officials told us they have tried to
coordinate with the Inter-American Development Bank but have received
little response. Officials also said that some other donors had been
reluctant to share information.
Teams are increasingly relying on surveys of customers to determine
their needs and interests when designing activities and developing
strategies. For example, in Bangladesh, the mission has focused its
democracy and governance activities on local government rather than
the national government because customers indicated that local
government was a more effective agent of change. In El Salvador,
strategic objective teams, with the assistance of local contractors,
held focus groups throughout the country and used the results to
develop customer service plans and as input for the mission's country
strategy.
--------------------
\4 USAID has defined its partners as host-country governments, U.S.
private voluntary agencies, indigenous or international
nongovernmental agencies, universities, professional or business
associations, other U.S. government agencies, other bilateral or
multilateral donors, and international financial institutions.
\5 USAID has defined its customers as the end users or beneficiaries
of assistance.
USAID HEADQUARTERS HAS
SUPPORTED REENGINEERING AT
MISSIONS
---------------------------------------------------------- Letter :3.5
USAID has established new systems and practices to support missions'
implementation of its reengineering efforts by providing advice and
information on reengineering. However, it has not provided training
needed to learn new skills in team dynamics and personnel matters.
USAID provides information, advice, and answers to staff questions on
reengineering through a monthly newsletter, best practices reports
and other publications, and an electronic help desk. The missions
can more easily access up-to-date agency guidance through an on-line
computer system that has replaced 33 manuals on policies, procedures,
and program operations guidance. Also, USAID's Management Bureau
sent a team to visit several missions in 1997 to identify
"significant management, organizational, and personnel-related
constraints being encountered by USAID missions as they embrace
reengineering principles" and to provide assistance to resolve them.
At the time of our visits to the four missions, USAID was not
providing needed training in new job skills and team operations. For
example, USAID has promoted the use of performance-based contracting
without providing the requisite training to the missions. As a
result, the mission contracts officer in Honduras was developing his
own training course. Also, USAID support in personnel matters has
lagged behind mission needs, according to mission managers. Thus,
the missions have, on their own, developed foreign employees'
evaluations and position descriptions and classifications but are
still waiting for USAID's reengineered position descriptions for
American staff.
USAID acknowledged that it had not provided adequate guidance to the
field in training and personnel matters. However, USAID noted that
there is broad agreement that training requirements and Foreign
Service position classification should be the next issues that the
agency examines.
PORTFOLIOS ARE BEGINNING TO
CHANGE BUT ARE SUBJECT TO
CONSTRAINTS
---------------------------------------------------------- Letter :3.6
Missions have begun to change their project portfolios in response to
agency reengineering. However, three factors have constrained the
restructuring of portfolios. First, reduced mission budgets have
limited the availability of funds for new projects. Second, the
missions do not have the authority, without headquarters approval, to
retain deobligated funds and shift them from one project to another
under a restructured portfolio, according to USAID officials. Third,
during the first year of reengineering, the missions concentrated
more on reorganization activities and the development of strategic
frameworks, including defining performance indicators and obtaining
baseline data, which limited the time available for portfolio
restructuring and new project development.
The missions we visited were still restructuring their portfolios to
focus on projects that more directly support their strategic
objectives. They were doing this largely by designing new projects
specifically to help achieve each strategic objective and by shifting
funds to activities that are more effective at achieving the
objective. To date, only a few new projects have been started. In
the Philippines, the mission is reshaping its portfolio by selecting
one project for each strategic objective to serve as a funding
mechanism for all activities under the objective. In Bangladesh, the
mission initiated one new project for each of the mission's strategic
objectives, which were designed using reengineering principles. In
1996, USAID approved a new country strategy that refocused activities
in El Salvador on the rural poor rather than war-to-peace transition
activities and macroeconomic reform at the national level, which were
the past focus. The mission in Honduras largely restructured its
portfolio in 1994 and 1995, prior to the start of reengineering, and
was awaiting approval of its new country strategy before it initiated
any new projects.
Reengineering efforts were undertaken during a period of severe
budget reductions. Between 1994 and 1997, expenditures at the four
missions we visited declined between 43 and 69 percent. A change in
the number of projects can be used as a rough approximation of the
change in mission activities and the restructuring of the portfolio.
In general, the number of active projects declined at these missions,
mostly between 1994 and 1995, before the formal start of
reengineering agencywide. In three missions, a limited number of
projects had been initiated since the start of reengineering.
Because most current mission projects were designed and funded before
the reengineering began and because USAID's budget has been reduced,
a limited amount of funding has been available for new projects.
According to mission officials, the inability to automatically retain
funds that are deobligated from a project limits a mission's
flexibility to use funds most effectively. It may also serve as a
disincentive for terminating projects. Consequently, missions may
allow projects to continue, even though they could identify other
projects that may contribute more to their strategic objectives.
During the first year of reengineering, the missions were busy
reorganizing into strategic objective teams, learning how to function
as teams, selecting measurable indicators, educating their partners
about their new approach, and training staff in the use of the New
Management System. When we visited the four missions in February
1997, the mission in El Salvador had just completed the development
of a country strategy and was still refining its indicators for the
economic growth strategic objective. The mission in Honduras was
developing a new country strategy and had not implemented any new
projects since the start of reengineering.
PERFORMANCE INDICATORS ARE
IMPROVING, BUT USAID'S IMPACT
ON OVERALL DEVELOPMENT IS STILL
UNCLEAR
------------------------------------------------------------ Letter :4
Under reengineering, missions have made varying degrees of progress
in developing performance indicators through which they can measure
the short- term results of individual projects or groups of projects,
but these indicators have not been in use long enough to have made a
significant impact on program management. Missions have had
difficulty in developing indicators that tally up the results of
individual or groups of projects to demonstrate achievement of
strategic objectives and overall, long-term development goals, such
as stimulating broad-based economic growth. USAID officials
acknowledge that in only a few cases have their programs been
directly linked to changes in country-level results. USAID
management indicated that the ability of a program to affect
country-level indicators will depend on the size of the country, the
budget available to support specific strategic objectives, and other
factors such as the context in which programs are implemented. USAID
is now developing common indicators through which it hopes to combine
mission results into overall agency accomplishments worldwide. We
recently reported that program evaluations can be valuable management
tools for demonstrating the programs' effectiveness.\6 Such
evaluations are an option for the agency to consider to help in its
efforts to meet GPRA requirements.
--------------------
\6 See Managing for Results: Analytic Challenges in Measuring
Performance (GAO/HEHS/GGD-97-138, May 30, 1997).
MISSIONS AT VARIOUS STAGES
IN ADOPTING MEASURES OF
INTERMEDIATE PROJECT RESULTS
---------------------------------------------------------- Letter :4.1
USAID missions have been developing performance indicators on two
levels. First, missions are developing project-level indicators
intended to monitor the performance and intermediate results of
individual projects or groups of projects in their portfolio.
Second, missions are developing strategic objective indicators that
are intended to gauge progress in achieving the missions' long-term
or strategic objectives, such as increasing national rural household
incomes or improving national systems of trade and investment.
The four missions we visited were at various stages in developing and
using performance measures for the projects intended to achieve one
of their strategic objectives--economic growth. In most cases, the
project-level indicators were too new for the missions to show impact
on program management. Although one mission had demonstrated a
thorough integration of its measures into its program management
processes, another was still defining the indicators it would use.
Some of the missions had fundamentally revised their project-level
performance measures in 1996 to make them more closely correspond to
their strategic objectives at the country level.
In Honduras, the mission has been using indicators to measure project
results since before reengineering. For example, to measure the
results of its agricultural assistance activities, the mission has
monitored the volume and value of exports for six crops: sweet
onions, ginger, okra, snow peas, asparagus, and plantains. The
mission has several activities aimed at promoting these exports,
including providing technical assistance to farmers for production
and marketing. Indicators the mission uses to measure intermediate
results for other activities include the number of land titles issued
by the government and the number of vocational center graduates
employed. Before adopting these types of results indicators, the
mission focused primarily on measuring the physical outputs supported
by USAID assistance, such as the number of agricultural research
projects conducted or the number of schools built. The mission has
been using results indicators extensively for program management,
including incorporating them in annual work plans for contractors and
other development partners and in its annual performance review.
In El Salvador, the mission recently established performance
indicators that are intended to measure the short-term results of its
assistance projects. In 1996, the missions relied almost exclusively
on macroeconomic statistics to report on the results of its economic
growth-related projects. The mission revised its indicators in 1996
to better reflect the direct outcomes of its program activities. For
example, new indicators include the number of people in rural areas
that are active clients (that have an outstanding loan and/or a
savings account) in participating financial institutions and the
number of people in rural areas receiving assistance in management,
agricultural technology, and marketing. However, because these
indicators are new, they have not been used extensively by the
mission for program management.
In the Philippines, the mission had developed an extensive set of
performance indicators, which it reported in 1996. However, mission
officials acknowledged that these indicators measured development
results that were beyond the scope of their activities in that
country. Since then, the mission has improved many indicators. For
example, indicators for projects in Mindanao have been revised to
more accurately reflect the desired results of the individual
activities. The indicators include the total value of
USAID-facilitated or -assisted private investments and the number of
government policies or practices modified to facilitate rapid and
equitable economic growth. Although many of the mission's indicators
are new, the mission has collected historical data against which to
measure future progress.
In Bangladesh, the mission was restructuring its strategic objectives
and revising its indicators. Many of the mission's indicators for
its economic growth objective had focused on results at the national
level and not specifically on areas targeted in USAID's activities.
The mission is now more closely aligning its indicators with its
activities to reflect what can feasibly be achieved, given its
economic growth resources. Some of the mission's more meaningful
indicators include the amount of fertilizer and improved seed
marketed nationally and the percentage of the population with access
to disaster relief supplies within 72 hours. Some historical data
were available for comparison with the current indicators, but the
mission plans to use a new data collection methodology for one key
indicator being developed, and baseline data are not yet available.
USAID OFTEN UNABLE TO
DEMONSTRATE IMPACT OF
PROJECTS ON MISSION
OBJECTIVES AND AGENCY GOALS
---------------------------------------------------------- Letter :4.2
Projects are expected to contribute to the achievement of missions'
strategic objectives and USAID's overall, long-term goals. USAID is
working to develop better ways to measure the extent of its
contribution to a country's development. According to USAID
officials, internal and external factors, including political
instability and the level of other donors' assistance, affect the
missions' country-level indicators. According to USAID management,
it is not possible to measure, in a quantifiable and precise way, the
impact of USAID activities alone on country-level development
indicators. Therefore, missions' strategic objective indicators are
intended to reflect the results not only of mission projects but also
of the efforts of other development partners. The collective impact
of activities at the four missions we visited was not significant
enough to have a measurable impact on country-level indicators.
USAID does not claim responsibility for the development results
measured by its strategic objective indicators but rather claims a
"plausible association" with the results.
Missions usually measure progress toward their strategic objectives
using country-level development indicators. In the countries we
visited, indicators of progress toward economic growth objectives
included the
-- number of people employed nationally in the agricultural,
industrial, and service sectors in Honduras;
-- percentage of the total population in El Salvador with access to
potable water;
-- national ratios of total exports and imports to gross domestic
product in the Philippines; and
-- per capita gross domestic product in Bangladesh.
At each of the four missions we visited, mission documents showed or
USAID officials acknowledged that the activities did not
significantly affect country-level indicators of economic growth.
For example, for 1997, the Bangladesh mission reported that the per
capita gross domestic product growth and agricultural and industrial
employment had improved but that "given USAID's relatively small
investments in this strategic objective, we cannot claim major
achievements at this national level." In El Salvador, the mission
reported that its projects were not sufficient to achieve its
economic growth strategic objective. The mission's economic growth
portfolio is composed of a limited number of projects that focus
primarily on microfinance, small business, basic education,
small-scale agriculture, and policy reform. The mission in El
Salvador reported that achieving the mission's strategic objective
depends on major contributions from other partners, especially
international banks, which are expected to provide most of the
funding required for activities relating to land, policy, and
infrastructure.
Internal and external factors can have a profound impact on
macroeconomic conditions and thus on country-level indicators. Such
factors include political instability, the commitment of political
leaders to necessary reforms, the magnitude and effectiveness of
assistance from other bilateral and multilateral donors, weather
conditions that affect crop yields, and the stability of
international markets. Recent mission reports for both Bangladesh
and the Philippines cited external factors as explanations for the
failure to achieve country-level development results. The Bangladesh
mission reported that political turmoil and power shortages were the
principal causes of shortfalls in expected economic growth. The
Philippines mission reported that the value of direct exports from
Mindanao was below targeted levels, primarily due to the decline in
pineapple prices and the loss of European markets for bananas.
Other observers have recently noted USAID's difficulty in
demonstrating its impact on broad development indicators. In March
1997, USAID's Inspector General concluded that in most cases, USAID's
goals and objectives exceeded the agency's span of influence and that
it was therefore extremely difficult for USAID to take credit for
improving country-level results. The Office of Management and Budget
similarly noted in 1996, after reviewing USAID's strategic plan and
indicators, that it was very difficult to credit USAID projects with
progress in country-level indicators.
USAID is developing common indicators that missions will use to
consistently measure progress in key areas worldwide. These measures
are intended to help the agency aggregate the results of its various
missions' programs to show progress in achieving overall agency
goals. However, the use of common indicators will not resolve
USAID's difficulty in attributing gains to its programs at the
country level. According to a senior agency official, a key
criterion for using these common indicators, as with the
mission-specific strategic objective indicators currently used, will
be that USAID can show a "plausible association" with the results,
not that the results are attributable to USAID assistance.
The extent of USAID's plausible association with the country-level
results is not reflected in the missions' documents. In some cases,
especially in countries where USAID is the largest donor and projects
are showing some results, this association may be very strong, while
in other cases, the association may be tenuous, based only on token
USAID involvement. The agency has not clearly and consistently
differentiated between levels of association with development results
in its mission performance reports. USAID management acknowledged
that it needs to do a better job of making the plausible associations
clear in its strategic documents.
Citing broad development indicators can result in misleading reports
on USAID's performance. The reports we reviewed showed that when
missions reported that specific strategic objective performance
targets were met, they rarely mentioned other factors outside of
USAID's control that contributed to the results, such as assistance
from other donors, actions taken by the host governments
independently, or favorable international market developments.
Furthermore, the USAID Inspector General recently reported that USAID
seemed to be taking credit in its 1996 Agency Performance Report for
some high-level impact merely because it had projects in that program
area.
PROGRAM EVALUATIONS CAN
DEMONSTRATE EFFECTIVENESS
BUT ARE NOT ROUTINELY DONE
---------------------------------------------------------- Letter :4.3
Although some of the missions we visited had done program evaluations
that could be used to link project results with country-level
development indicators, USAID's current performance measurement
system relies largely on indicators to assess the agency's impact.
However, in our recent report on GPRA implementation, we noted that
demonstrating the impact of a government program on outcome
indicators is difficult and a common problem among federal agencies
that are implementing performance measurement systems.\7 The most
difficult aspect of analyzing and reporting performance data is
separating the impact of a program from the impact of external
factors to measure the program's effect. We noted that in these
cases "it may be important to supplement performance measurement data
with impact evaluation studies to provide an accurate picture of
program effectiveness."
Systematic evaluations of how a program was implemented can provide
managers with important information about a program's success or
failure. For example, in Honduras, a USAID contractor's evaluation
of a basic education project confirmed that USAID activities in that
country, implemented from 1986 to 1995, had a significant impact on
key country-level indicators of improvement of the educational system
in that country relative to other factors.
Before October 1995, missions were required to evaluate each project
upon completion and submit the results to USAID's Center for
Development Information and Evaluation. However, current USAID
guidance allows missions to decide whether they will do performance
evaluations of their activities. In one mission we visited, a senior
USAID official said that the mission would rely on performance
indicators, not on evaluations, to monitor its projects' progress and
results. He said that evaluations would be done primarily when
implementation problems arise. At other missions, USAID officials
also suggested that evaluations would probably be done less often
than in the past.
According to the Director of USAID's Center for Development
Information and Evaluation, the agency's revised policy on evaluation
was not necessarily intended to reduce the frequency of evaluations
at the missions, but to better target them to the management needs of
the mission. He said that missions are expected to do adequate
evaluations to support statements about performance included in
mission reports. However, documents we reviewed rarely cited
evaluations to support their descriptions of performance in economic
growth strategic objectives. Officials from the Center and a mission
we visited told us that they believed the agency needed to reassess
its policies on evaluation and mission practices to ensure that the
agency is sufficiently and appropriately using evaluations for
effective management.
According to USAID management, as missions improve performance
monitoring, there will be less need for routine impact evaluations at
the activity level. Eventually, fully operational performance
monitoring will result in an adequate assessment of the impact of
USAID's activities in a country. In addition, missions could
supplement performance measurement data by obtaining reviews of other
donor programs and host-country efforts. Nevertheless, periodic,
independent evaluations of USAID's funded activities can be a useful
tool to validate program accomplishments.
--------------------
\7 Managing for Results: Analytic Challenges in Measuring
Performance (GAO/HEHS/GGD-97-138, May 30, 1997).
MISSION PERFORMANCE IS BEING
LINKED TO RESOURCE ALLOCATION
------------------------------------------------------------ Letter :5
USAID has begun using program performance in its resource allocation
process in a more systematic manner. However, the process continues
to evolve, and resource allocation decisions are affected primarily
by foreign policy considerations of the executive branch and
congressional priorities. Each of USAID's regional bureaus used
different formulas and assigned different weights for performance
when constructing their position on USAID's fiscal year 1998 budget
request. However, guidance for the fiscal year 1999 budget cycle
attempts to standardize the bureaus' assessment processes, including
the weight accorded to performance in budget deliberations.
REVISED RESOURCE ALLOCATION
PROCESS FOCUSES ON PROGRAM
CONTRIBUTIONS WITHIN EACH
AGENCY GOAL
---------------------------------------------------------- Letter :5.1
A basic principle underlying USAID's reform efforts is to divide its
assistance program funding among its strategic goals in line with
carefully crafted strategic plans and systems for performance
measurement. USAID now includes more explicit and systematic
consideration of program performance in its resource allocation
process.
Mission officials told us USAID's emphasis on performance and results
was good, but expressed mixed views on USAID's allocation process.
Officials generally said political and foreign policy considerations
continue to dominate the budget process. While mission officials
said the process is more transparent than in the past, some said that
USAID should be more consultative with the missions regarding the
allocation of earmarked and/or directed funding. One mission
director said there is little linkage between the mission's
performance, as measured against the mission's "results contract"
with USAID, and the allocation of resources to the mission.
USAID's process for assessing performance and allocating resources
continues to evolve, and modifications continue to be made in
reporting requirements and procedures for weighing program results.
Performance has been a factor in allocating resources. For example,
when we observed headquarters technical sessions for rating
performance, we learned that USAID was eliminating an agricultural
loan program in Guinea due to low rates of loan disbursements. We
also learned that USAID was cutting assistance to El Salvador's
electoral commission due to its poor performance in implementing
needed reforms. Performance assessments may become more meaningful
in the process as missions provide more complete performance
information. In the Asia and Near East Bureau, the number of
activities that reported complete performance information rose from
31 percent in 1995 to 59 percent in 1996.
USAID allowed regional bureaus latitude in how they implemented the
reporting process for the 1998 fiscal year budget request, as well as
for identifying priorities for fiscal year 1997 resource allocations.
The Asia and the Near East Bureau scored and ranked, within each of
the agency's four sustainable development goals, each strategic
objective by performance, contribution to agency and bureau
priorities, contribution to foreign policy goals, and the extent to
which the host country had been a good development partner for that
objective. The ranking became a starting point for further
discussions on what activities should be funded. The Bureau for
Africa also used a similar approach, scoring and ranking strategic
objectives within each sector on performance, pipeline, host-country
performance in the sector, sectoral need and magnitude of the
problem, and contribution to regional initiatives and priorities for
that sector. The Bureau for Latin America and the Caribbean scored
and ranked each strategic objective by performance, then scored and
ranked countries by performance, foreign policy interest (such as
funding Haitian democratization efforts or the Guatemalan peace
process), severity of need, and commitment to free market policies
and democratic governance. These scores were used to place countries
in four funding categories; those in the highest category received
funding priority, after allowing for funding needs based on analysis
of the pipeline and for meeting targets for earmarks and directives.
The USAID Administrator issued guidance in January 1997 to
standardize the bureaus' resource requests, performance assessments,
and resource allocation processes for the fiscal year 1999 budget
request. Among other things, resource requests and allocations are
expected to be based on strategic objective performance and its
significance to the strategic plan. Bureaus are to use three common
factors and common weights to measure performance and allocate
resources: performance (35 percent), contribution to agency goals
(30 percent), and contribution to development initiatives (35
percent). While this guidance provides a more standard ranking and
scoring format, these groupings still allow wide latitude for
including factors that are still considered in the allocation
process. For example, contribution to agency goals includes foreign
policy objectives, and contribution to development initiatives
includes bureau initiatives, country and/or sectoral need, and the
quality of the development partnership in general and within specific
goal areas.
We attended several USAID program review meetings during this year's
deliberations on the fiscal year 1999 budget at the Africa, Asia/Near
East, and Latin America bureaus. Generally, the initial discussions
on strategic objective performance were chaired by technical staff
from the regional bureaus with varying representation from the
Global, Management, and Policy and Program Coordination bureaus. The
Asia/Near East and Latin America bureaus held several levels of
reviews in Washington for each country, often with senior mission
management and mission program staff. Due to the large number of
missions in Africa, the Washington teams conducted the initial
assessment based on the submissions and, in some cases, additional
information sent from the field. The Africa Bureau held subsequent
1-day program reviews with staff from full sustainable development
missions and with missions and operating units with upcoming strategy
reviews. Following the rounds of technical and country meetings,
bureaus conducted regional wrap-up sessions to reach resource
allocation decisions and prepare for budget reviews with agency
management.
At the technical meetings we attended, we noted that the teams had
reviewed the documents, and, in some cases, received additional
information from the missions. Team members that had been given
guidance on the criteria for scoring performance appeared well
prepared, usually supporting their scores with specific examples of
how strategic objectives were or were not meeting targets or making
an impact. In some cases, team members discussed the quality and
relevance of performance indicators and targets and the extent to
which impact could be attributed to USAID assistance. We also heard
some limited discussion on the quality of the data used to measure
performance.
This is only the second year of USAID's revised budget process, and
USAID officials acknowledged that modifications will continue to be
made as best practices are identified. We noted at several meetings
that the team members and mission staff were confused about some of
the mission reporting. For example, the guidance asked missions to
report on the intermediate results that were most central to
achieving the goals of the strategic objectives. However, there was
discussion about whether the reported information was the best
reflection of performance. Some team members cabled to missions for
additional information and used this in their scoring; others did
not. The quality of the reports and the extent to which they provide
an accurate and complete story of each strategic objective are key
factors in performance assessment.
NEW MANAGEMENT SYSTEM IS NOT
WORKING
------------------------------------------------------------ Letter :6
Although USAID will have spent about $100 million by the end of
fiscal year 1998 to develop its New Management System (NMS), the
system does not work as intended and has created serious problems in
mission operations and morale. The system is key to successful
information sharing required under reengineering for accountability
and control. The agency deployed the system worldwide, knowing that
it was not fully operational or adequately tested. Indeed, USAID's
Inspector General reported 6 months after implementation of the
system that it was not complete, had not been demonstrated to work
effectively, and had not been adequately tested.\8 Because of
problems with the system, USAID suspended use of part of the system
in the missions in April 1997 and is now taking corrective steps
recommended by the Inspector General.
--------------------
\8 Audit of the Worldwide Deployment of the New Management System
(NMS) (USAID/OIG, A-000-97-004P, Mar. 31, 1997).
SYSTEM IS KEY TO USAID
REFORMS
---------------------------------------------------------- Letter :6.1
Since 1994, USAID has been developing the NMS to support its
organizational reforms. The system is designed to consolidate
USAID's accounting, budget, personnel, procurement, program
operations, and property management into a single, integrated network
that the agency's missions and offices worldwide can access and to
aid in the effective management and monitoring of its programs.\9 The
purpose of the system is to
-- make financial management more efficient by streamlining
business processes, eliminating paper forms, ensuring its
compliance with federal accounting and financial management
requirements, and providing managers with information needed to
make appropriate decisions and reliably report the status of
USAID projects;
-- facilitate missions' program delivery by providing the means for
missions and offices to share information on-line about program
management; and
-- empower missions and provide USAID management with a means for
maintaining accountability.
The system was activated at USAID headquarters in July 1996 and at
the missions in October 1996. As of October 1996, four of the six
system modules--the accounting, budget, operations, and procurement
modules--were operational, with some limitations, at USAID
headquarters. The personnel and property management modules are
still in development. According to USAID's Office of the Inspector
General, the agency will have invested about $89 million by the end
of fiscal year 1997 and about $100 million by the end of fiscal year
1998 to develop the system.
--------------------
\9 See Foreign Assistance: Status of USAID's Reforms
(GAO/NSIAD-96-241BR, Sept. 24, 1996); Interim Report on the Status
of USAID's New Management System (USAID/OIG, A-000-96-001-S, Sept.
27, 1996); and Interim Report on the Cost of USAID's New Management
System (USAID/OIG, A-000-96-002-S, Sept. 27, 1996).
DYSFUNCTIONAL NMS HAS
HAMPERED OPERATIONS AT
MISSIONS
---------------------------------------------------------- Letter :6.2
The four missions we visited could not routinely use NMS to
successfully execute financial management functions such as
obligating funds and recording procurement actions. According to
mission officials, having a dysfunctional system limits the benefits
of the missions' reengineered operations. The Honduras mission
director told us that "the reform process has now reached a point
where some reforms as contemplated simply cannot go much further
without a functioning NMS."
For several months, the mission in Honduras could not obligate
allocated funds for disaster relief on the country's north coast due
to problems in the system's accounting and procurement modules.
Also, because of the system's long response times, entering
information into the system was often very time-consuming and
frustrating for mission staff. Officials in Honduras attempted to
run some procurement actions while we observed: the actions took
from 2 to 6 hours to process, and none could be successfully
completed. In fact, because of these problems, some mission
officials we visited had two personal computers on their desks: one
for using NMS and one to do other work. In El Salvador, some mission
personnel told us that all other work frequently stopped while NMS
was in use and the system was having a negative impact on
implementation of mission projects.
In addition to citing excessive system response times as a major
problem, the agency's Chief of Staff noted potential problems with
the system development effort. He said that because each module was
independently developed, the database used for NMS had some
inconsistencies. For example, in one module, for some data fields
the system will accept eight characters, while another module will
accept only six characters for the same information. Such
inconsistencies can cause problems when attempting to transfer data
from one module to another.
In March 1997, USAID's Inspector General reported that the system's
premature deployment had increased the risk of fraud and abuse, had
not met users' needs, and did not meet basic federal financial system
requirements. These findings were consistent with our observations.
The Inspector General made a number of recommendations to USAID
management, including suspension of part of the system until many of
its problems were resolved.
In April 1997, following the Inspector General's report and comments
from system users and us, USAID suspended the NMS accounting and
procurement modules and left the budget and operations modules
operational in the missions. These systems are expected to remain
shut down until at least fiscal year 1998. All four modules will
still be used by headquarters offices, which have not had as much
trouble in using NMS as have the missions.
USAID said it is taking the following steps to address problems with
the NMS:
-- A new Director of Information Resources Management, who will
focus exclusively on managing information resource activities,
has been hired.
-- The Administrator has announced that he will designate a senior
official as the agency's Chief Information Officer, who will be
certified by the Office of Management and Budget as qualified
for the position and report directly to the Administrator.
-- A full-time NMS project manager has been selected to supervise
the development efforts until the system is operational.
-- Efforts have been undertaken to analyze the technical and
implementation problems that currently limit NMS from achieving
its full potential.
-- A goal to achieve level 2 of the Software Engineering
Institute's Capability Maturity Model (CMM) for software
development has been established.\10
-- A statement of work has been developed to assess the accounting
and procurement components of NMS and their quality and to
identify the risks and opportunities in the application code
that the components use.
-- A transition to performance-based contracting for NMS and a
reduction in both the number of contract entities and
contractors are underway.
-- Development activities are limited to the portions of NMS that
are still operational and needed to establish the core
functionality in the accounting module.
--------------------
\10 The CMM for Software is a framework of key elements of an
effective software process. The CMM shows an evolutionary
improvement path for software organizations that consists of 5 levels
with
level 1 (termed "Initial") representing an immature process. Level 2
(termed "Repeatable") is defined as basic project management
processes that have been established to track cost, schedule, and
functionality. In addition, the necessary process discipline is in
place to repeat earlier successes on projects with similar
applications. USAID is also requiring in its new software
development contracts that the firms be independently judged as
operating at a minimum of CMM level 2.
CONCLUSIONS
------------------------------------------------------------ Letter :7
It is too early to discern the full extent of the impact of
reengineering efforts at USAID missions, as they have only recently
made the investment and operational changes necessary to bring about
long-term change. However, USAID's reengineering efforts have
produced some tangible benefits in the areas of planning,
implementing, and monitoring assistance projects. Missions are
adapting to the strategic approach to planning, but missions'
progress in changing their portfolios of projects has been
constrained by factors such as funding limitations and obligation
authority. The extent to which USAID can capitalize on operational
changes will depend on continued vigilance by agency management to
ensure that these changes take hold and continue. The ultimate test
will be whether USAID can achieve sustained efficiencies in project
planning and implementation and demonstrate the impact of its
projects. With limited resources available for foreign assistance,
USAID's ability to target its assistance effectively and deliver it
efficiently is especially important.
Since information-sharing is critical to USAID's new operating
system, diligence in properly deploying NMS is key to the reform
process. While NMS promises to aid in more efficient management of
USAID resources, many of the efficiencies expected to result from
this system have been delayed and offset by unsuccessful
implementation. USAID has taken positive steps to address the
concerns discussed in this report and the recommendations in its
Inspector General's report. However, whether these actions will
correct the NMS' problems will depend on the successful
implementation of these efforts. In addressing the problems with
NMS, it is USAID management's responsibility to ensure that NMS
facilitates, rather than impedes, the reform process.
AGENCY COMMENTS
------------------------------------------------------------ Letter :8
In commenting on a draft of this report, USAID stated that the report
was "a fair, balanced, and thoughtful assessment of the state of
overseas missions' reengineering efforts." USAID comments provided
some suggested clarifications to the draft report and additional
information on agency efforts to address problems associated with the
New Management System, which we have incorporated in the text of our
report, as appropriate. USAID's letter, without attachments, is
reprinted in appendix IV.
SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :9
As requested, we examined how USAID missions have reformed their
operations and developed a results-oriented program. In making our
selections for overseas fieldwork, we asked USAID to recommend five
countries in each of its regions where reengineering has progressed
to the point that we could see results and the economic growth
objective was a priority. USAID provided us program budget data on
all countries in which it operates and five country recommendations
per region, along with explanations for their selections. Following
discussions with USAID officials and congressional Committee staff,
we selected El Salvador, Honduras, Bangladesh, and the Philippines
for fieldwork. We based our country selections on the size of the
USAID program, importance of the economic growth strategic objective
in the overall program, and pipeline size. We had done fieldwork in
the Dominican Republic for the first part of the assignment. We
focused on Latin America and Asia because of the greater emphasis on
economic growth in these regions; we excluded Central Europe and the
former Soviet Union region from consideration since these countries'
programs are relatively new, transitional, and expected to end soon.
To assess (1) how USAID missions have reengineered their operations
and (2) how USAID monitors and evaluates the results of its projects,
we interviewed USAID officials in the Management, Global, and
regional bureaus and the Bureau for Policy and Program Coordination,
including the Center for Development Information and Evaluation in
Washington, D.C.; reviewed agency policies, practices, and
procedures; and analyzed agency documentation, including evaluations,
reporting documents, and Inspector General reports. At the missions
visited, we interviewed USAID management and staff, held discussions
with foreign service nationals on the impact of reforms and
reengineering, and collected and analyzed relevant documentation. We
also interviewed representatives of nongovernmental and private
voluntary organizations and USAID contractors that implement USAID
programs and officials of international financial institutions, other
donor countries, and the host governments.
To assess how reengineering has affected the delivery of development
assistance, we interviewed USAID officials, partners, and customers
about reengineered program design and delivery and reviewed
documentation on mission strategies, plans, and projects. In
reviewing portfolio changes at the missions we visited, we assessed
the active, completed, terminated, or initiated projects for each
year between fiscal year 1994 and 1997.
To determine how USAID allocates funds for its projects, we
interviewed USAID officials, attended regional bureau budget
sessions, examined the budget process, and reviewed recent and
current budget documents and agency guidance. We assessed how the
budget process relates the various factors that the agency uses to
measure and rank its performance to provide some insight as to
whether the budget process is meeting expectations.
To assess how USAID's NMS supports mission operations, we met with
officials of USAID's Office of the Inspector General and reviewed
their reports on NMS.\11 We reviewed agency documentation, including
agency cable traffic and e-mails in Washington, D.C., and the field
missions visited; USAID's general notice system and policy
directives, guidance, and statements; and congressional testimony by
the Administrator.\12 We interviewed USAID officials and agency staff
who use NMS in Washington, D.C., and in the field missions and
observed mission officials' and staff members' use and/or tests of
NMS. We reviewed our prior reports: a December 1992 report
identifying weaknesses in federal information resources management
that frequently led to costly projects that show disappointing
results,\13 numerous reports and testimonies since 1993 on
information management overall,\14 and a February 1997 report noting
that failure to follow disciplined management and system development
practices could lead to costly failures.\15
We performed our work from October 1996 to May 1997 in accordance
with generally accepted government auditing standards.
--------------------
\11 See Interim Report on the Status of USAID's New Management System
(USAID/OIG, A-000-96-001-S, Sept. 1996); Interim Report on the Cost
of USAID's New Management System (USAID/OIG, A-000-96-002-S, Sept.
1996); and Audit of the Worldwide Deployment of the New Management
System (NMS) (USAID/OIG, A-000-97-004P, Mar. 1997).
\12 J. Brian Atwood, Administrator, USAID, before the House
Appropriations Committee, Subcommittee on Foreign Operations, March
19, 1997.
\13 See Information Management and Technology Issues (GAO/OCG-93-5TR,
Dec. 1992) and Information Resources: Summary of Federal Agencies'
Information Resources Management Problems (GAO/IMTEC-92-13FS, Feb.
1992).
\14 See Information Technology Investment: Agencies Can Improve
Performance, Reduce Costs, and Minimize Risks (GAO/AIMD-96-64, Sept.
1996); Information Management Reform: Effective Implementation Is
Essential for Improving Federal Performance (GAO/T-AIMD-96-132, July
1996); Executive Guide: Improving Mission Performance Through
Strategic Information Management and Technology (GAO/AIMD-94-115, May
1994); Foreign Assistance: AID Strategic Direction and Continued
Management Improvements Needed (GAO/NSIAD-93/106, June 1993).
\15 See Information Management and Technology (GAO/HR-97-9, Feb.
1997).
---------------------------------------------------------- Letter :9.1
We are providing copies of this report to the Chairmen and Ranking
Minority Members of the House and Senate Committees on
Appropriations, the Senate Committee on Foreign Relations, the House
Committee on Government Reform and Oversight, and the Senate
Committee on Governmental Affairs, and to the Ranking Minority Member
of the House Committee on International Relations. We are also
sending copies to the Administrator, U.S. Agency for International
Development, and to the Director, Office of Management and Budget.
Copies will be made available to others upon request.
Please contact me at (202) 512-4128 if you or your staff have any
questions about this report. Major contributors to this report are
listed in
appendix V.
Sincerely yours,
Benjamin F. Nelson
Director, International Relations and
Trade Issues
USAID OBLIGATIONS, FISCAL YEARS
1995-97
=========================================================== Appendix I
(Dollars in thousands)
Fiscal year
----------------------------------------------------------------------
1995 1996 1997 (est.)
---------------------- ---------------------- ----------------------
Sustaina
ble
developm
ent goal
areas
and
operatin
g
expenses Obligation Percent Obligation Percent Obligation Percent
-------- ---------- ---------- ---------- ---------- ---------- ----------
Economic $2,398,897 40.1 $1,980,349 38.4 $1,996,967 38.0
growth
Democrac 436,506 7.3 388,805 7.5 398,777 7.6
y
PHN\a 1,125,784 18.8 934,700 18.1 968,510 18.4
Environm 495,098 8.3 377,945 7.3 397,499 7.6
ent
Other\b 1,533,031 25.6 1,476,065 28.6 1,495,656 28.4
Total 5,989,316 100.0 5,157,864 100.0 5,257,409 100.0
program
funds
Operatin 612,164 \d 561,732 \d 608,311 \d
g
expense
s\c
================================================================================
Total $6,601,480 \d $5,719,596 \d $5,865,720 \d
--------------------------------------------------------------------------------
\a Population, health, and nutrition.
\b The amounts for "Other" include the $1.2 billion provided to
Israel each year.
\c Includes the U.S. Agency for International Development (USAID)
and USAID Office of Inspector General operating expenses and Foreign
Service Retirement and Disability funds.
\d Not applicable.
Sources: Program funds: USAID, Bureau for Management, Office of the
Budget. Operating expenses: USAID Congressional Presentation,
Summary Tables, Fiscal Year 1998.
DIRECTED AND UNDIRECTED FUNDS FOR
DEVELOPMENT ASSISTANCE AND RELATED
ACCOUNTS, FISCAL YEARS 1995-97
========================================================== Appendix II
(Dollars in thousands)
Fiscal year
-------------------------------------
Account 1995 1996 1997
------------------------------- ----------- ----------- -----------
Child survival $321,092 $352,992 $502,000\a
Basic child's education 104,406 88,000 98,000
Transfers 0 56,500 51,500
U.S. Telecommunications 380 400 500
Training Institute earmark
Other directives\b 302,084 259,173 233,798
Population 514,724 356,000 355,000
Total directed 1,242,686 1,113,065 1,240,798
Undirected funds available for 834,382 561,935 540,702
economic growth, democracy,
and environment
Total development assistance $2,077,068 $1,675,000 $1,781,500
Undirected funds as percentage 40.2 33.5 30.4
of development assistance
----------------------------------------------------------------------
\a Includes $100 million grant to United Nations Childrens Fund.
\b Includes directives reported by USAID as contributing to economic
growth goals.
Source: USAID, Bureau for Management, Office of the Budget.
CHRONOLOGY OF USAID REFORMS
========================================================= Appendix III
(See figure in printed
edition.)
Source: USAID.
(See figure in printed edition.)Appendix IV
COMMENTS FROM THE U.S. AGENCY FOR
INTERNATIONAL DEVELOPMENT
========================================================= Appendix III
MAJOR CONTRIBUTORS TO THIS REPORT
=========================================================== Appendix V
NATIONAL SECURITY AND
INTERNATIONAL AFFAIRS DIVISION,
WASHINGTON, D.C.
Jess T. Ford
Lawrence L. Suda
John D. DeForge
Ann L. Baker
Bruce L. Kutnick
James B. Michels
Nancy L. Ragsdale
Audrey E. Solis
James M. Strus
ACCOUNTING AND INFORMATION
MANAGEMENT DIVISION, WASHINGTON,
D.C.
John C. Martin
GENERAL GOVERNMENT DIVISION,
WASHINGTON, D.C.
Donna J. Byers
OFFICE OF GENERAL COUNSEL,
WASHINGTON, D.C.
Ernie E. Jackson
*** End of document. ***