Grants Management: Enhancing Performance Accountability
Provisions Could Lead to Better Results (29-SEP-06, GAO-06-1046).
Maximizing the extent to which grants achieve their long-term
performance goals is critical to successfully addressing the
challenges of the 21st century. While performance accountability
mechanisms are fairly new to federal grants, they have been used
in contracts for some time and lessons learned have begun to
inform federal grant design. Given this, GAO was asked to examine
(1) challenges to performance accountability in federal grants,
(2) mechanisms being used to improve grant performance, and (3)
strategies the federal government can use to encourage the use of
these mechanisms. GAO performed a content analysis of relevant
literature and interviewed experts. To illustrate the mechanisms
and strategies found in the literature, GAO used examples from
the literature and selected additional case illustrations--two
federal grant programs (vocational education and child support
enforcement) and two nonfederal contracts--for further study.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-1046
ACCNO: A61606
TITLE: Grants Management: Enhancing Performance Accountability
Provisions Could Lead to Better Results
DATE: 09/29/2006
SUBJECT: Accountability
Federal funds
Federal grants
Funds management
Grant administration
Lessons learned
Performance measures
Strategic planning
Information sharing
Program implementation
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-1046
* Results in Brief
* Background
* Child Support Enforcement
* Carl D. Perkins Career and Technical Education
* Massachusetts Division of Medical Assistance Contract with t
* Ontario Realty Corporation's Contract with ProFac
* Trade-offs and Challenges Exist in Ensuring Performance Acco
* Grant Design Features Affect the Balance between Accountabil
* Implementation Issues Present Further Performance Accountabi
* Accountability Mechanisms Can Improve Performance and Perfor
* A Variety of Accountability Mechanisms Exist
* Accountability Provisions Can Be Employed at Different Phase
* Accountability Mechanisms Can Be Tailored to Specific Situat
* Strategies Support Successful Selection, Design, and Impleme
* Ensure Mechanisms Are of Sufficient Value
* Understand the Value of Performance
* Ensure Effective Distribution
* Execute Mechanisms Consistently
* Periodically Renegotiate and Revise Mechanisms and Measures
* Ensure Appropriate Measurement Selection and Usage
* Performance Should Be within Recipient's Ability and Influen
* Measures Should Be Suitable to the Mechanism Cycle
* Measures and Data Should Be Tested
* Ensure Grantor and Grantee Technical Capacity
* Implement System in Stages
* Collaboration and Oversight Also Key to Success
* Various Opportunities Exist at the Federal Level to Enhance
* A Results-Focused Design Encourages Performance Accountabili
* Careful Use of National Program Evaluation Studies and Resea
* Encourage Development and Use of Credible Performance Inform
* Share Good Practices and Lessons Learned
* Conclusions
* Recommendation for Executive Action
* Agency Comments
* Appendix I: Objectives, Scope, and Methodology
* Appendix II: GAO Contact and Staff Acknowledgments
* GAO Contact
* Acknowledgments
* Bibliography
* Order by Mail or Phone
Report to the Chairman, Subcommittee on Government Management, Finance and
Accountability, Committee on Government Reform, House of Representatives
United States Government Accountability Office
GAO
September 2006
GRANTS MANAGEMENT
Enhancing Performance Accountability Provisions Could Lead to Better
Results
GAO-06-1046
Contents
Letter 1
Results in Brief 4
Background 7
Trade-offs and Challenges Exist in Ensuring Performance Accountability in
Federal Grants 11
Accountability Mechanisms Can Improve Performance and Performance
Accountability 15
Strategies Support Successful Selection, Design, and Implementation of
Performance Accountability Mechanisms 22
Various Opportunities Exist at the Federal Level to Enhance Performance
Accountability in Grants 34
Conclusions 39
Recommendation for Executive Action 41
Agency Comments 41
Appendix I Objectives, Scope, and Methodology 42
Appendix II GAO Contact and Staff Acknowledgments 44
Bibliography 45
Tables
Table 1: Examples of Accountability Provisions 16
Table 2: Sources and Types of Authorization and Guidance for Performance
Accountability Mechanisms 20
Table 3: Examples of Ways to Tailor Performance Measures and Mechanisms 21
Figure
Figure 1: Accountability Provisions Can Be Used at Different Points in the
Grant Life Cycle by Various Users 18
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office
Washington, DC 20548
September 29, 2006
The Honorable Todd Platts Chairman, Subcommittee on Government Management,
Finance and Accountability Committee on Government Reform House of
Representatives
Dear Mr. Chairman:
The federal government faces an array of challenges and opportunities to
enhance performance, ensure accountability, and position the nation for
the future. A number of overarching trends-including the nation's
long-term fiscal imbalance-drive the need to reexamine what the federal
government does, how it does it, who does it, and how it gets financed.
Because grants to state and local governments constituted nearly 20
percent of total federal outlays in fiscal year 2005, maximizing the
extent to which grants achieve their long-term performance goals and
objectives is critical to successfully addressing the challenges of the
21st century.
In recent years, interest in federal grant performance accountability has
grown. For the purposes of this report, performance accountability is
defined as the mechanisms by which individuals or organizations are held
accountable for meeting specified performance-related expectations.
Consistent with the decade-long trend toward an increased results
orientation and expectation for performance accountability as evidenced by
the Government Performance and Results Act of 1993 (GPRA),1 performance
accountability mechanisms in federal grants have become more common. For
example, performance assessment mechanisms are present in grants
authorized by both the Job Training Partnership Act and its successor
program, the Workforce Investment Act of 1988 (WIA) and the No Child Left
Behind Act.2 More recently, the Office of Management and Budget (OMB)
developed the Program Assessment Rating Tool (PART), which, among other
things, holds federal programs and their partners accountable for
performance. Simply monitoring and reporting performance can also
encourage performance accountability and performance improvements. In this
report, we have focused on specific mechanisms that are meant to encourage
performance incentives-such as rewards given or penalties imposed-when
performance exceeds or fails to meet specified levels.
1Pub. L. No. 103-62, 107 Stat. 285 (1993).
2Pub. L. No. 105-220, Pub. L. No. 107-110.
While performance accountability mechanisms are fairly new to some federal
grants, they have been used in contracts and loans for some time.
Moreover, lessons learned from performance-based contracting have begun to
inform federal grant design, for example, in the case of WIA grants
requirements. In addition, some states award their federal pass-through
grants to subgrantees as contracts with performance accountability
requirements.3
Given this growing body of experience, you asked us to examine ways to
infuse effective performance accountability mechanisms and practices into
the federal grant process. Specifically, our objectives were to identify
(1) What kinds of challenges to performance accountability exist in
federal grants? (2) What kinds of mechanisms are being used to improve
grant performance, and how? and (3) Given the findings of questions 1 and
2, what strategies can the federal government use to encourage the use of
these mechanisms, as appropriate? For the purposes of this report, we were
interested specifically in the mechanisms by which individuals or
organizations are held accountable for meeting specified
performance-related expectations that are directly tied to a grant.
To address our objectives, we conducted a literature review that included
our prior reports, and interviewed experts in the area of federal grant
and contract performance accountability to identify the: (1) challenges to
performance accountability that exist in federal grants, (2) types of
performance accountability mechanisms-defined as rewards and
penalties-used, and (3) key strategies that appear to encourage the
successful implementation of performance accountability mechanisms. Our
identification of types of mechanisms and strategies was developed by
conducting a content analysis of selected literature from our review that
met our criteria for addressing the issue of accountability. Recognizing
that grants have increasingly assumed features traditionally associated
with contracts, we drew on experiences from both performance-based
contracting and grants to help identify valuable lessons learned that
could inform efforts to improve performance accountability in federal
grants. To illustrate these mechanisms and strategies, and to supplement
our findings and the many case examples identified by our content
analysis, we use relevant case examples found in the literature. We also
selected four additional cases for further in-depth illustrations. These
four cases were selected based on our literature review, interviews with
experts, and reviews of prior GAO work because they are good examples of
where (1) a performance mechanism was present and (2) there is reason to
believe that performance improved. We selected two federal grant programs
and two nonfederal contracts: (1) the federal vocational education grants
authorized by the Carl D. Perkins Vocational and Technical Education Act
of 1998 (Perkins III), which are passed through states to secondary and
postsecondary schools for career and technical education; (2) the federal
Child Support Enforcement (CSE) program, authorized by the Social Security
Act, Title IV, part D, which ensures that children are financially
supported by both parents; (3) the real property management contract
between the Ontario Provincial Government's Ontario Realty Corporation
(ORC) and SNC-Lavalin ProFac, Inc. (ProFac), a private property management
company; and (4) the contract between the Massachusetts Division of
Medical Assistance (DMA) and the Massachusetts Behavioral Health
Partnership (MBHP) for the provision of mental health and substance abuse
services for residents covered by the MassHealth Medicaid program.
3Pass-through grants are federal grants given to state governments that
are subsequently distributed to county, municipal, or township
governments.
To develop the federal grant case illustrations and obtain perspectives on
the strategies we identified, we interviewed federal headquarters and
regional program and finance officials from the federal agencies that
administer the grant programs-the Departments of Education and Health and
Human Services. In addition, we visited selected grantees and subgrantees
from among these programs that federal and state officials identified as
being particularly successful-or as facing particular challenges-with
performance accountability. To develop the contracting case illustrations,
we interviewed, conducted site visits, or both with both contractors and
contracting agencies. For both the grant and contract cases, we reviewed
the authorizing legislation or contract, guidance, documentation, and
prior studies, and interviewed relevant officials to obtain perspectives
on the strategies we identified.
See appendix I for a more detailed discussion of our scope and
methodology.
We conducted our work from December 2005 through August 2006 in offices in
Washington, D.C.; Harrisburg, Lancaster, Norristown, and Philadelphia,
Pennsylvania; Eloy, Glendale, and Phoenix, Arizona; and Boston,
Massachusetts, in accordance with generally accepted government auditing
standards.
Results in Brief
Although there are various ways to design grants to encourage performance
accountability, in general, there are three factors that particularly
affect the degree of performance accountability that can be achieved,
including whether a grant (1) includes performance-oriented objectives in
addition to fiscally oriented objectives, (2) operates as a distinct
program or as a funding stream, and (3) supports a limited or diverse
array of objectives. Because design features that encourage performance
accountability can limit state and local grantee flexibility, achieving
these twin goals can be a delicate balancing act, and has implications for
the accountability relationship between levels of government and the
information needed to support accountability. Even in federal grants with
designs that favor performance accountability, grant implementation
challenges related to developing performance goals and measures as well as
collecting and reporting performance data can influence the extent of
performance accountability achieved.
Accountability mechanisms available for use in grants vary widely and can
be financial or nonfinancial in nature. A financial mechanism could reward
performance with increased funding or a onetime bonus payment;
nonfinancial mechanisms include such things as altered oversight or
flexibility. Financial mechanisms also vary the degree of risk sharing
between the grantor and the grantee. Many mechanisms can be employed by
Congress, agencies, or grant recipients at different points throughout the
grant life cycle. Mechanisms are flexible and need to be tailored to
specific situations since not all mechanisms are appropriate to all
situations, and there is no "one-size-fits-all" or "magic bullet" solution
to performance accountability.
Collectively, five key strategies appear to facilitate the effective
design and implementation of performance accountability mechanisms. They
are as follows:
o Ensure mechanisms are of sufficient value. The value of the
rewards and penalties-whether financial or nonfinancial-and the
cost of improved performance are adequate to motivate desired
behaviors and provide a meaningful return to both the grantor and
the grantee.
o Periodically renegotiate and revise mechanisms and measures.
Provide for and use the flexibility to reevaluate performance
accountability mechanisms and associated performance measures at
regular, scheduled intervals and allow time to learn from each
cycle to improve performance.
o Ensure appropriate measurement selection. Measures should
represent performance that is within the grantee's sphere of
influence, and can reasonably be achieved and evaluated within the
specified time frame, and should be tested over time to minimize
the potential for unintended consequences and perverse incentives.
o Ensure grantor and grantee technical capacity. Grantors and
grantees should have the necessary knowledge about performance
accountability mechanisms and the ability to effectively implement
them.
o Ensure phased implementation. Allow time to design, test, and
revise measurement systems before linking them to accountability
mechanisms.
In addition to these strategies, we noted extensive use of
partnerships and collaborations and regular and effective
oversight and feedback, which appeared critical to the success of
accountability provisions in a third-party environment. We have
previously reported that these practices are often associated with
both high- performing organizations4 and organizations that
effectively used performance information to manage.5
The experiences with and strategies related to federal grant
accountability provisions described in this report suggest a
number of opportunities for Congress and the executive branch to
improve the design and implementation of performance
accountability mechanisms. First, a results-focused design can
help encourage performance accountability in general and
specifically provide for-or at least not prohibit-the use of
accountability mechanisms to encourage desired behavior. In
addition, the use of national program evaluation studies and
research and demonstration grants can provide valuable information
to assist in agency and congressional oversight of and knowledge
about accountability mechanisms. Because credible performance
information and performance measures form the basis for
well-functioning accountability provisions, it remains critical
for Congress and the executive branch to continue to encourage the
development and use of such measures. Finally, OMB and agencies-as
well as grantees-can benefit from sharing good practices and
lessons learned about experiences with performance accountability
provisions in federal grants, as this is an efficient and
effective way to increase grantor and grantee knowledge,
understanding, and use of these provisions.
OMB, as the focal point for overall management in the executive
branch, plays a key role in improving the performance of federal
programs. It uses a number of vehicles, such as Web sites with
information about performance measures and grants targeted to
federal agencies and informal workshops and seminars, to encourage
general performance improvement in federal programs. OMB staff
told us that focusing specifically on performance accountability
provisions in grants is necessary and useful, but that to date,
they have focused their efforts on encouraging and enhancing
agency capacity to develop high-quality, results-based program
performance measures since improving the quality of measures and
data necessarily precedes tying them to accountability provisions.
We are therefore recommending that the Director of OMB encourage
and assist federal agencies in working with the Congress to expand
the effective use of performance accountability mechanisms,
focusing on the practices in this report, when federal grant
programs are being created or reauthorized. We further recommend
that OMB offer opportunities for knowledge transfer among federal
agencies and encourage agencies to share leading practices and
lessons learned in implementing grant accountability mechanisms.
Possible vehicles for the collection and dissemination of this
information include good practices guides and workshops and Web
sites such as results.gov, grants.gov, and expectmore.gov.
On August 22, 2006, we provided a draft of this report to the
Director of OMB and the Secretaries of Education and Health and
Human Services. We also provided relevant sections of a draft of
this report to the grantees and contractors highlighted in this
report. We received technical comments from all three agencies,
which were incorporated as appropriate. In addition, OMB agreed
with our recommendation but suggested we broaden it to address the
role of federal agencies and Congress in the grant redesign and
reauthorization process. We agree, and have amended our
recommendation accordingly.
Background
Grants, along with contracts and cooperative agreements, are tools
used by the federal government to achieve national priorities via
nonfederal parties, including state and local governments,
educational institutions, and nonprofit organizations. Diverse in
structure and purpose, grants can be generally classified as
either categorical or block, with categorical grants allowing less
recipient discretion than block grants. For example, the Community
Services Block Grant provides funds to states and is sometimes
passed to local agencies to support a variety of efforts that
reduce poverty, revitalize low-income communities, and lead to
self-sufficiency among low-income families and individuals, while
giving the agencies broad discretion in how the funds can be
spent. In practice, the "categorical" and "block" grant labels
represent the ends of a continuum and overlap considerably in its
middle range.
Grant funds may also be grouped by their method of allocating
funds. Formula grants allocate funds based on distribution
formulas prescribed by legislation or administrative regulation
and often narrowly define the eligible recipients as state
agencies. On the other hand, categorical grants are generally
awarded on a competitive basis to applicants meeting broader
eligibility requirements.
Despite substantial variation among grants, grants generally
follow a similar life cycle and include announcement, application,
award, postaward, and closeout phases. Once established through
legislation, which may specify particular objectives and
eligibility and other requirements, a grant program may be further
defined by grantor agency requirements. For competitive grant
programs, the public is notified of the grant opportunity
announcement, and potential grantees must submit their
applications for agency review. In the awards stage, the agency
identifies successful applicants or legislatively defined grant
recipients and awards funding. The postaward stage includes
payment processing, agency monitoring, and grantee reporting,
which may include financial and performance information. The
closeout phase includes preparation of final reports, financial
reconciliation, and any required accounting for property.
Traditionally, grant accountability has referred to legal or
financial compliance. The Single Audit Act,6 for example, requires
grantees to conduct an overall financial compliance audit to
promote accountability. As such, at a minimum all grantees are
held accountable for sound financial management and use of federal
funds to support allowable activities. Beyond that, however,
accountability for performance varies from grant to grant. As
discussed earlier, this historical focus on financial
accountability has expanded in response to increasing expectations
of demonstrable performance and performance accountability for all
government programs. For example, the Comptroller General's
Domestic Working Group issued its Guide to Opportunities for
Improving Grant Accountability, highlighting innovative approaches
and promising practices in grants management-focused both on
ensuring grant funds are spent properly as well as achieving their
desired results.7
While performance accountability in grants is a relatively new
pursuit, it has been used in contracts for a number of years. To
illustrate performance accountability mechanisms and the
strategies that contribute to their successful design and
implementation, we examined four cases: (1) the federal CSE
program, (2) the federal Perkins III Career and Technical
Education Program, (3) a performance-based contract between the
Massachusetts DMA and MBHP, and (4) a performance-based contract
between the Canadian Ontario Realty Corporation (ORC) and ProFac.
Child Support Enforcement
The CSE program was established in 1975 by Title IV-D of the
Social Security Act (Pub. L. No. 93-647). CSE functions in all
states and territories through state or local social services
departments, attorneys general offices, or departments of revenue
in order to ensure that children are financially supported by both
of their parents. State programs work toward establishing
paternity, locating parents, establishing and enforcing support
orders, and collecting and distributing child support payments.
The federal Office of Child Support Enforcement (OCSE), an office
of the Department of Health and Human Services' Administration for
Children and Families, oversees the development, management, and
operation of state CSE programs and provides financial support (66
percent of total operating costs) to states. In fiscal year 2005
federal expenditures on CSE were $3.5 billion, with states
spending $1.8 billion. Total collections in fiscal year 2005 were
more than $23 billion. The total legally owed support for fiscal
year 2005 was $29 billion, with $17.4 billion of that collected.
Total arrears (past due payments) for all previous years combined
was $107 billion. Over $7 billion of those past due payments were
collected and distributed in fiscal year 2005.
The Child Support Performance and Incentive Act of 1998 (Pub. L.
No. 105-200) linked incentive payments to performance, and in
fiscal year 2005, OCSE made over $450 million in incentive
payments to states. This act changed the original CSE incentive
program from awarding incentives based solely on
cost-effectiveness to awards based on meeting specific performance
targets in five outcome areas: paternity establishment, order
establishment, current collections, past due collections, and
cost-effectiveness. The performance measures and targets are
defined in the text of the act, which also provides a formula for
determining the amount of each incentive payment. Additionally,
the act established an alternative penalty system for those states
not yet in compliance with the statewide automated data processing
system required by Title IV-D Sec. 454(A) of the Social Security
Act. The new incentive program was phased in from 2000 through
2002.
Carl D. Perkins Career and Technical Education
Effective July 1, 1999, the Carl D. Perkins Vocational and
Technical Education Act of 1998 (Perkins III, Pub. L. No. 105-332)
amends earlier legislation to evaluate and improve vocational and
technical education.8 Each year under Perkins III, Congress has
appropriated more than $1.1 billion in grants to states for career
and technical education. The Office of Vocational and Adult
Education (OVAE), an office of the Department of Education,
administers the grants established in Perkins III, a pass-through
grant to states, which administer the distribution of the funds to
local school districts.
Perkins III defines major roles for OVAE and states in
establishing performance accountability systems for vocational and
technical education. States are given the responsibility for
developing performance measures and data collection systems
related to four required core performance indicators: academic and
technical skill attainment, completion, placement and retention,
and nontraditional participation and completion. OVAE negotiates
these performance measures with states to ensure that they are
sufficiently rigorous. States not meeting their performance levels
for 1 year are required to complete a program improvement plan.
States not meeting their performance levels for 2 years are
subject to financial sanctions, although no state has failed to
meet its overall levels for 2 consecutive years. States have also
been eligible to receive incentive funds if they exceeded
performance goals for the Perkins III grant as well as targets
established by Title I and Title II of WIA. Title I of WIA
supports workforce investment programs. Title II, also known as
the Adult Education and Family Literacy Act, provides adult
education funds to states. Governors have the authority to
allocate the incentive funds for use in any of the three program
areas.
Massachusetts Division of Medical Assistance Contract with the
Massachusetts Behavioral Health Partnership
Beginning in 1996, the Massachusetts DMA entered into a 5-year
contract, which was renewed in 2001, with MBHP to manage mental
health and substance abuse services for roughly 300,000 people
covered by the MassHealth Primary Care Clinician Plan-part of the
Massachusetts Medicaid program. Of the individuals covered by the
plan, more than half are children 18 or younger, including 20,000
children in the custody of the commonwealth's Departments of
Social Services and Youth Services.
The structure of the contract between DMA and MBHP involves a base
contract, which governs requirements related to administrative and
medical operations. These requirements continue through the life
of the contract, or until they are modified through amendments. In
addition to the base contract, there are performance incentive
projects that focus on research and development projects. The
majority of earnings available to MBHP come from the
organization's successful completion of these contractually
defined incentive projects, which are renegotiated annually.
Earnings are achieved only after the successful completion of
specified goals and objectives, as documented and reviewed by the
state.
Ontario Realty Corporation�s Contract with ProFac
ProFac, a facilities management company, was awarded a 5-year
contract in 1999 (since renewed) by ORC to provide facilities
management services for approximately 30 million square feet of
space in 2,100 building sites owned by the Ontario government.
About 280 ProFac managers, engineers, technicians, and support
staff provide these services.
The contract between ORC and ProFac links performance to a 10
percent quarterly management fee holdback: on a monthly basis, ORC
only reimburses ProFac for 90 percent of its administrative costs,
retaining the other 10 percent, a "holdback," which ORC returns to
ProFac on a quarterly basis only if ProFac obtains a sufficient
level of performance. This contract also links performance to an
annual share-in-savings arrangement: ORC sets a budget each year
based on prior years' actual expenditures and budget projections;
if ProFac spends less than the budget, it is able to share the
difference, a share in savings, only if it reaches a sufficient
operational level. ProFac's performance on 30 key performance
indicators (KPI) determines its operational level. These KPIs are
accumulated to determine a score in four performance objectives:
management performance, financial performance, asset integrity,
and customer service. The performance objectives are scored and
then weighted according to ORC's priorities to determine a total
performance rating.
Trade-offs and Challenges Exist in Ensuring Performance
Accountability in Federal Grants
The trade-offs and challenges associated with performance
accountability in federal grants largely depend on several key
aspects of grant design and implementation. As we have previously
reported, performance accountability tends to be greater (and
grantee flexibility lower) in programs with certain types of
design features. Because design features that encourage
performance accountability can limit state and local grantee
flexibility, achieving these twin goals can be a delicate
balancing act and has implications for the accountability
relationship between levels of government and the information
needed to support accountability. Even in federal grants with
designs that favor performance accountability, grant
implementation challenges related to developing performance goals
and measures as well as collecting and reporting performance data
can influence the extent of performance accountability achieved.
Grant Design Features Affect the Balance between Accountability
and Flexibility
Although there are various ways to design grants to encourage
performance accountability, in general, there are three factors
that particularly affect the degree of performance accountability
that can be achieved, including whether a grant (1) includes
performance-oriented objectives in addition to fiscally oriented
objectives, (2) operates as a distinct program or as a funding
stream, and (3) supports a limited or diverse array of objectives.
As we discussed previously, federal grants have traditionally
focused on fiscal or legal accountability, such as holding states
accountable for using federal grant funds to supplement rather
than to supplant their own spending on a particular activity.
However, federal grants that also include performance-oriented
objectives-as well as the provisions that implement them-provide
the basis for performance measurement and accountability for
results, and signal a federal role in managing performance over
the grant. Ideally, both types of objectives would be present in
federal grants. Performance-related objectives focus on service or
production activities and their results. For example, the central
objective of the grants for Special Programs for the
Aging-Nutrition Services, is to provide nutritious meals to needy
older Americans to improve nutrition and reduce social isolation.
In contrast, fiscal or financial assistance objectives focus on
providing dollars to support or expand activities. Typical fiscal
objectives include increasing support for meritorious goods or
underfunded services and targeting grant funding to needy
jurisdictions. For example, the objective of Title VI Innovative
Education grants is to provide funds to support local education
reform efforts. When objectives are purely fiscal, accountability
to the federal agency tends to focus on fiscal matters, such as
holding states accountable for using federal grant funds to
supplement rather than to supplant their own spending on a
particular activity.
Even when performance-oriented objectives are present, whether
federal grants operate as distinct programs or as part of a larger
funding stream directly affects who can be held accountable and
for what.9 A grant that operates as a program has performance
requirements and objectives and carries out specific programwide
functions through a distinct delivery system, such that
grant-funded activities, clients, and products are clearly
identifiable. This type of grant gives the federal agency a role
in managing performance and makes it easier to obtain uniform
information about performance attributable to the grant funds. It
is possible to identify which activities were supported; the
amount of federal funds allocated to each; and to various extents,
the results grantees achieved with federal funds.
In contrast, funds from grants that operate as part of a funding
stream are merged with funds from state or local sources (and
sometimes from other federal sources) to support state or local
activities allowable under the flexible grant. These programs are
managed at the state or local level, with the federal role limited
accordingly. When grants are part of a funding stream, it is
possible to identify which activities federal funds supported and
the amount allocated to each, but once the grant funds are
combined with the overall budget for a state or local activity,
federal dollars lose their identification and their specific
results cannot be separated out. This is particularly the case
when the federal share is small, with most funding coming from
other sources. The program outcome measures available in such
programs are likely to be for outcomes of the state or local
service delivery program, not the federal program from which the
funding originated. Thus, grantees would generally be held
accountable for overall outcomes, regardless of the funding
source. For example, projects such as Oregon Option and the
National Performance Review were designed to promote
accountability for federal and/or national priorities, regardless
of the funding source. They encourage grantors and grantees to
work toward collaboratively developed outcomes. These
intergovernmental partnerships can be particularly useful when
funds come from a combination of federal, state, local, and
private sources, or when the federal funding share is small.
Federal grants vary along a continuum, at one end supporting a
single major activity common to all grantees (such as categorical
grants), and at the other end, allowing unrestricted choice by the
recipient among a wide variety of allowable activities, (such as
block grants). Flexibility is narrowest, but accountability to the
federal level clearest, in programs that focus on a single major
activity. Flexibility is broadest in programs designed to support
diverse state or local activities, but finding a common
performance metric can be extremely challenging since these
activities can vary considerably from state to state. That said,
we have previously reported on options for building accountability
provisions into block grants that help balance states' flexibility
to select a mix of activities and services that will best allow
them to achieve a particular national outcome with accountability
for achieving that outcome. These options include (1) relying on
state processes both to manage block grant funds and to monitor
and assess compliance and (2) emphasizing results-based evaluation
rather than examining specific program or administrative
activities.
Implementation Issues Present Further Performance Accountability
Challenges
In addition to these design features, we have previously reported
on a number of performance accountability challenges encountered
in many grant programs during the grant implementation phase.
Lack of consensus on goals and performance measures: The
priorities of states, tribes, local communities, and the federal
government are not always the same. To ensure that grantees work
toward national priorities, they need to be involved in the
development of performance goals and measures. Lack of agreement
on goals and measures-particularly when the federal funding is a
small portion of the funding stream-could lead to grantees making
choices that do not necessarily support the achievement of
national goals.
Reliance on performance data from state and local partners and
other third parties: Even if grantees collect data on similar
activities, outcomes, and services, absent common data definitions
Congress and program managers will lack comparable information,
limiting the ability to compare state efforts or draw meaningful
conclusions about the relative effectiveness of different
strategies. We have previously reported that agencies relying on
third parties for performance data also have difficulty
ascertaining the accuracy and quality of the data. Further,
programs often rely on state administrative systems for
performance information. For some programs-such as many of the
Administration for Children and Families' programs--since final
reports are not due until 90 to 120 days after the end of the
federal fiscal year, there is a delay in available data.
Onerous and inconsistent grant administration processes and
requirements10: Multiple grants maybe available for the same or
similar purposes, meaning that federal grant recipients must
navigate through a myriad of federal grant programs in order to
find the appropriate source of funds to finance projects that meet
local needs and address local issues. Sometimes programs meant to
address common problems have potentially conflicting requirements.
Variations in performance accountability requirements among these
grants can limit the degree of performance accountability
achieved. We have recently reported that while this situation is
improving because of OMB's efforts to streamline the grants
application process, problems still exist.11
Prohibition of performance information collection12: Because
states are principally responsible for implementing block grants
at the state level, the block grant statutory prohibitions and
requirements, and federal regulations and guidance are generally
kept to a minimum. Sometimes federal agencies are prohibited from
imposing reporting requirements because they are seen as
burdensome. Clearly, this limits the extent to which federal
agencies can oversee grantee performance.
Nevertheless, even with these trade-offs and challenges, agencies
have been able to shift toward increased performance
accountability in federal grants and the use of accountability
provisions to ensure that grantees achieve real results through
the programs, activities, and services financed with federal
funds. The accountability provisions described in this report,
along with strategies for their effective use, can help address
the challenges noted above.
Accountability Mechanisms Can Improve Performance and Performance
Accountability
We found a number of accountability provisions, specific actions
that can be taken-that is, rewards given or penalties imposed when
performance exceeds or fails to meet specified performance
levels-that Congress, granting agencies, and grantees can use at
different points in the grant life cycle to improve both grant
performance and performance accountability. These examples
demonstrate that accountability provisions can result in
significant performance improvement and are flexible enough to
accommodate a variety of situations.
A Variety of Accountability Mechanisms Exist
We found that a wide variety of accountability provisions are
being used in both grant and contracting situations. A selection
of these provisions is shown in table 1. This list is not intended
to be exhaustive; rather, it is meant to illustrate the variety of
mechanisms available. Some mechanisms may be more appropriate in
certain situations than others, but all of these mechanisms can be
used to either encourage improved performance or discourage poor
performance. For example, public recognition and increased funding
are two different mechanisms that can both be used to encourage
and reward good performance. Similarly, mechanisms such as reduced
funding or increased oversight can be used to discourage or
penalize poor performance.
4The other attributes of high-performing organizations are a clear,
well-articulated, and compelling mission; a focus on the needs of the
clients and customers; and the strategic management of people.
5GAO, Comptroller General's Forum: High-Performing Organizations: Metrics,
Means, and Mechanisms for Achieving High Performance in the 21st Century
Public Management Environment, GAO-04-343SP (Washington, D.C.: Feb. 13,
2004), and Managing for Results: Enhancing Agency Use of Performance
Information for Management Decision Making, GAO-05-927 (Washington, D.C.:
Sept. 9, 2005).
6Pub. L. No. 98-502.
7Domestic Working Group, Grant Accountability Project, Guide to
Opportunities for Improving Grant Accountability. October 2005. This guide
states that it is designed to provide government executives at the
federal, state and local levels with ideas for better managing grants. The
guide focuses on specific steps taken by various agencies. The intent is
to share useful and innovative approaches taken, so that others can
consider using them.
8On August 12, 2006, the Carl D. Perkins Career and Technical Education
Improvement Act of 2006 (Perkins IV) became Pub. L. No. 109-270. Perkins
IV includes some revisions to the performance and accountability
provisions.
9GAO, Grant Programs: Design Features Shape Flexibility, Accountability,
and Performance Information, GAO/GGD-98-137 (Washington, D.C.: June 22,
1998).
10GAO, Federal Assistance: Grant System Continues to Be Highly Fragmented,
GAO-03-718T (Washington, D.C.: Apr. 29, 2003).
11Pub. L. No. 106-107, the Federal Financial Assistance Management
Improvement Act of 1999 requires OMB to coordinate agency efforts to
streamline the administrative requirements of federal grants and engage
and involve grantees in developing and implementing their reform goals and
implementation plans. The act also requires GAO to evaluate the reform
efforts. See GAO, Grants Management: Grantees' Concerns with Efforts to
Streamline and simplify Processes. GAO-06-566 (Washington, D.C.: July 28,
2006).
12GAO, Block Grants: Issues in Designing Accountability Provisions.
GAO/AIMD-95-226 (Washington, D.C.: Sept. 1, 1995).
Table 1: Examples of Accountability Provisions
Accountability mechanism Definition
Rewards Praise Public recognition of good
performance, for example, through
the press, Web sites, intranets,
newsletters, hearings, testimony,
and award ceremonies.
Bonus Onetime cash payment.
Rewards or Increase/decrease Increase or decrease in grantee's
penalties flexibility flexibility by issuing
administrative, programmatic, or
financial waivers from
requirements and restrictions or
by adding award conditions.
Increase/decrease Manipulate the workload (e.g., case
workload load).
Increase/decrease award Increase or decrease in the length or
term term of the grant.
Increase/decrease Increase or decrease in the degree of
oversight oversight.
Increase/decrease funding Increase or decrease in the per-unit
rate reimbursement rate (e.g., case rate).
Either partial or full funding can be
based on a unit rate.
Increase/decrease funding Increase or decrease in funding.
level Either the entire award can be tied
to performance or a portion of
funding above an established baseline
(i.e., an incentive portion).
Examples include share in savings
(grantee keeps a portion of dollars
saved) and milestones (payments
linked to a predefined chronological
series of performance levels
typically combining process, output,
and outcome measures).
Use of past performance Use past performance of grantee to
inform selection of future
recipients.
Penalties Reproof Public reprimand for poor
performance, for example, through
the press, Web sites, intranets,
newsletters, hearings, and
testimony.
Reperformance Grantee must reperform the service at
its own cost to meet performance
agreements.
Impose financial penalty Includes a onetime reduction in the
or sanction value of an award. Sanctions may also
be in one area to influence actions
in another area (crossover
sanctions). Also includes suspending
or withholding a payment (temporarily
halting grant payments and/or work),
suspending or terminating the award
(canceling the current grant or
temporarily excluding grantee from
future awards), or finally, debarment
(permanently exclude grantee from
future grant awards).
Source: GAO.
In addition, mechanisms can be either financial or nonfinancial in nature.
A financial mechanism would be an increase in funding or a bonus. For
example, the CSE program employs a financial incentive in the form of a
bonus to encourage states to work toward the program's five performance
goals: states are eligible for a bonus every year based on performance.
Nonfinancial mechanisms would include altered oversight or flexibility.
For example, as part of the National Environmental Performance Partnership
System, the Environmental Protection Agency affords states with high
environmental performance levels greater flexibility in spending their
grant funds.
Financial mechanisms also vary by their degree of risk or risk sharing
between the grantor and the grantee. Grantee risk increases as the amount
of money tied to performance increases. For example, bonuses-money awarded
over and above the base grant amount-represent the least risk, while an
outcome-based milestone payment plan where the entire grant award is based
on performance represents much higher financial risk to the grantee.
Nonfinancial actions, such as altering flexibility or oversight, would be
relatively risk neutral.
Accountability Provisions Can Be Employed at Different Phases of the Grant Life
Cycle
Accountability mechanisms can be used in different phases of the grant
life cycle by different actors, including Congress, granting agencies, and
grantees themselves, and the lessons learned from one grant cycle can be
used to improve a performance accountability mechanism in the next (see
fig. 1). For example, when reauthorizing the CSE program, Congress revised
the original CSE incentive payments, which were solely based on cost
efficiency, to create an incentive program tied to performance measures
that reflect CSE's five key goals: (1) paternity establishment, (2) order
establishment, (3) current collections, (4) collection of payments in
arrears, and (5) cost-effectiveness (design/redesign phase).
Figure 1: Accountability Provisions Can Be Used at Different Points in the
Grant Life Cycle by Various Users
In contrast, performance measures and targets for Perkins III are created
during the implementation phase. Specifically, each state is required by
law to create its own performance measures linked to four core indicators:
(1) student attainment of challenging state-established academic and
vocational technical skill proficiencies; (2) student attainment of
secondary diploma or postsecondary degree or credential; (3) student
placement in employment, pursuit of further education, or both; and (4)
student participation in and completion of vocational technical education
programs that lead to nontraditional training and employment. The
Department of Education periodically negotiates the performance targets
for each state measure (postaward phase).
The use of past performance can inform and improve the recipient selection
process (application phase). Specifically, the Florida Department of
Children and Families has reported considerable success using past
performance in recipient selection-contractors that do not meet their
performance measures and standards are ineligible to be awarded future
contracts.
Other mechanisms, such as altered flexibility or oversight, can be used by
the granting agency-or even the grantee-to encourage improved performance
during the term of the award (postaward phase). For example, according to
the literature we reviewed, Minnesota's Department of Human Services
Refugee Services Section increases its oversight of local agencies if
their performance drops below 80 percent on their key performance
measures, including job placement rates, which nearly doubled over 5
fiscal years.
Importantly, grantees can also use these provisions to extend
accountability to subgrantees and contractors. This is significant because
many federal grants are ultimately passed through states to subgrantees.
Some accountability provisions, such as public award and recognition, can
even be employed by stakeholders or interested parties. For example, the
National Association for State Directors of Career and Technical Education
Consortium annually recognizes high-performing career and technical
administrators and teachers and provides opportunities to share lessons
learned and best practices.
Even when performance accountability provisions are absent from or limited
by a grant's legislation, agencies and grantees may still be able to
include these types of provisions in the terms and conditions of the
grants or subgrants or in contracts as long as the authorizing legislation
does not specifically prohibit their use. OMB Circular A-110 provides that
for grants awarded to nonprofits, a number of accountability mechanisms
may be used-including withholding payments, termination of award, and
"other remedies that may be legally available"-if the grantee materially
fails to comply with the terms and conditions of the award. These terms
and conditions can be specified in federal statute, regulation, assurance,
applications, or the notice of award. For grants to state and local
governments, however, OMB Circular A-102 contains no detailed
accountability provisions and defers to the requirements specified in the
authorizing legislation.
Various authorities govern the use of accountability provisions (see table
2). Provisions set by Congress, such as increased flexibility in the form
of waivers from statutory restrictions, are generally laid out in
authorization or appropriations legislation. As stated earlier, granting
agencies can include accountability provisions in regulations, grant
announcements, the request for proposal, and the notice of award. Grantees
(and subgrantees) can use accountability provisions, such as formal
recognition, to improve their performance internally. For example, the CSE
program in Montgomery County, Pennsylvania, recognizes
performance-improving suggestions from individual employees by publicly
praising and inducting them into the office's "all-star team." These
employees also receive a T-shirt with a picture of a stork-the program's
mascot-with the program's motto Striving Toward Optimizing our Resources
for Kids.
Table 2: Sources and Types of Authorization and Guidance for Performance
Accountability Mechanisms
Authorizing body Authorizing/implementing vehicles
Congress Authorization, appropriations, other legislation
Grantor Regulations, announcements, the request for proposal in
the Federal Register, notice of award
Grantee Internal personnel policies and practices, use with
subgrantees and contractors
Source: GAO.
Accountability Mechanisms Can Be Tailored to Specific Situations
Selecting appropriate performance measures and linking them to performance
accountability mechanisms is not a one-size-fits-all process; rather,
accountability provisions are tailored to reflect the program's
characteristics. In addition to the range of accountability mechanisms
available, we found a number of ways mechanisms were tailored and combined
to reflect a variety of circumstances. Table 3 describes how measures and
mechanisms can be designed and triggered to either reward or penalize
performance.
Table 3: Examples of Ways to Tailor Performance Measures and Mechanisms
Tailored measure and
mechanism Description
Stretch goals An action is taken based on reaching a
secondary, higher goal. For example, a grantor
awards the grantee when performance exceeds one
performance target and reaches a secondary,
higher target.
Hurdles/triggers Specific conditions or performance levels that
must be met before actions can be taken. For
example, once a recipient meets a minimum
performance level or requirement, it becomes
eligible for an award based on performance.
Dead bands Ranges of performance are established for which
the mechanism does not provide an award or
penalty. For example, performance achieved
within a certain range is not eligible for an
award or penalty, but performance above or below
that range is subject to them.
Step up/step down Actions are taken based on a number of preset
performance levels. Step up is similar to a
series of stretch goals. For example, a
recipient receives a reward in which the value
is determined by which performance interval it
reached. Conversely, each time a recipient's
performance drops, it is subject to increasing
penalties or deductions.
Formula Payments are made based on a formula-a
mathematical weighting of a number of factors.
For example, a recipient's entire award or an
incentive portion may be based on a formula
containing a performance component.
Share in savings/share in Payments represent a share of a specific source
revenue of funds. Share in revenue is used to refer to
situations where performance is defined by
revenue generation and payments are based on
some formulation of revenue generated, typically
a percentage. For example, a recipient
responsible for increasing financial collections
is eligible to receive a portion of the
additional funds collected.
Share in savings is used to refer to situations
where performance is defined by cost savings and
payments are based on some formulation of cost
savings, typically a percentage. For example, a
recipient that identifies efficiencies that
result in administrative cost savings is
eligible to receive a portion of the savings.
Milestones Payments are linked to a predefined
chronological series of performance levels
representing processes, outputs, and outcomes.
For example, a recipient is paid 33 percent of
the performance-based portion of award as each
of three milestones is completed successfully.
Floating measures Used to refer to situations where one or sets of
performance measures can be changed during the
term of an agreement. These measures are
selected from a larger set of predefined
performance measures. For example, a grantor
assesses grantee performance on 25 performance
measures, but at any given time provides awards
or penalties based on the performance of a
subset of the measures. The subset the grantor
provides the award or penalty for can change
during the course of the agreement, with
notification.
Indexes Used to refer to a situation where individual
performance measures are weighted to create a
single index. For example, a number of measures
are weighted according to priority, and then the
combined weight is used as a single performance
measure.
Variable target Situation where performance is defined by a
variable or relative measure, such as
performance of a third party in the same issue
area. This method is used, for example, in
situations where performance can be directly
affected by external factors, such as the
economy. For example, grantors assess the
performance of a single grant recipient by
comparing its performance to other recipients of
the same grant.
Source: GAO, based on literature review.
For example, to encourage its contractor, ProFac, to cut costs while
maintaining a high-level of performance, ORC modified a basic financial
incentive to include a share-in-savings feature. ProFac is eligible to
share a portion of the savings if it spends less than its yearly
budget-often referred to as share in savings. To ensure that ProFac does
not cut costs to the detriment of high performance, ORC also requires that
ProFac achieve a performance rating of 80 percent or higher to share in
these cost savings.
The CSE performance measures are an example of a "step up" provision-for
each increasing performance percentage interval there is a corresponding
increase in the incentive percentage paid. Each time a state moves to the
next highest interval, it receives a higher percentage of the incentive
for that measure. Conversely, the alternative penalty procedure for
failure to implement a statewide child support data processing system acts
as a "step down" mechanism. For each year the state fails to implement
such a system, but shows a good faith effort to attempt to do so, the
state will be penalized at increasing intervals-during the first year of
noncompliance, the state will receive a 4 percent penalty, the second year
an 8 percent penalty, the third year a 16 percent penalty, and so on.
Strategies Support Successful Selection, Design, and Implementation of
Performance Accountability Mechanisms
Collectively, five key strategies appear to facilitate the effective
selection, design, and implementation of performance accountability
mechanisms.13 These strategies are
o ensure mechanisms are of sufficient value,
o periodically renegotiate and revise mechanisms,
o ensure appropriate measurement selection and usage,
o ensure grantor and grantee technical capacity, and
o implement system in stages.
In addition to these strategies, we noted extensive use of
partnerships and collaborations and regular and effective
oversight and feedback, which appeared critical to the success of
accountability provisions in a third-party environment. We have
previously reported that these practices are often associated with
high-performing organizations and organizations that effectively
used performance information to manage.14
Ensure Mechanisms Are of Sufficient Value
There are a number of factors to consider when designing
accountability mechanisms that help to ensure the mechanisms are
of sufficient value and motivate performance improvement. Ensuring
sufficient value requires that
o both the grantor and grantee are able to determine the value of
the rewards and penalties and the cost of improved performance-be
they financial or nonfinancial-and provide a meaningful return to
both the grantor and the grantee and
o rewards or penalties should be consistently applied to maintain
the value of the mechanisms to both the grantor and grantee.
Understand the Value of Performance
According to the literature we reviewed, both the grantor and
grantee should understand what a particular level of performance
is worth to them and what it will cost them to achieve that level
of performance. When the value of performance is not properly
identified, funds could be wasted and grantees may not respond to
the mechanism. For example, we found one case where the
contracting agency offered and ultimately paid a $250,000 bonus to
a contractor for completing a pipeline 2-1/2 months earlier than
scheduled. However, because the contracting agency did not
actually need the pipeline to be completed for several years after
the original contractual deadline, the contractor paid $250,000
for a level of performance it did not need. Although the recipient
responded to the incentive, the contracting agency did not
properly calculate the value of the performance improvement to the
agency, resulting in wasted funds.
For a grantor, considering how accountability provisions support
its strategic priorities can assist in determining the value of
performance. The size of the associated rewards and penalties
should be commensurate with the priority. For example, successful
pay-for-performance programs reserve large rewards for achieving
an organization's most important priorities, or those that lead to
large benefits, and provide smaller incentives for achieving goals
that reap smaller benefits or are of lesser importance. For
example, in the health care field, for certain conditions such as
heart attack or stroke, delays in administering appropriate
therapy greatly increase the risk of mortality and disability.
Therefore, the incentives to treat these conditions quickly and
appropriately should be larger than the incentives for other
practices that should be encouraged yet produce fewer direct
effects on mortality and illness, such as avoiding the use of
ineffective antibiotics to treat the common cold.
Based on our literature review, it appears that insufficiently
valued incentives are one of the main reasons that accountability
provisions fail. When an incentive is of sufficient value, the
expected return outweighs the expected risk, and recipients are
motivated to pursue the performance improvement. From 1975 through
1997 the CSE program included an incentive program that focused on
cost-effectiveness. States were guaranteed an "incentive payment"
from 6 to 10 percent of their total collections. In practice, the
4 percentage point difference between the minimum and maximum
payment was reportedly not large enough to motivate states to
increase collections enough to earn the 10 percent bonus. The new
incentive system, established by the Child Support Performance and
Incentive Act of 199815 only provides incentive payments to states
that meet one or more of the act's five outcome-based performance
goals and associated targets,16 and penalizes states that fall
below threshold levels in certain areas. A review of the new
incentive system in a sample of nine states found that the median
score on each of the five performance measures increased from
fiscal years 2000 to 2002, the time period that the incentive
system was implemented.17
Motivating grantees to work toward federal outcomes is
particularly challenging in grants where the federal investment is
relatively small. Officials at Arizona's Department of Adult
Education, Career and Technical Education Division, told us that
state funds in joint technological education districts outweighed
federal funds for career and technical education (CTE) programs by
more than four to one, and some districts did not want to accept
federal CTE funds because, in their view, complying with the
federal performance requirements was not worth the amount of funds
they would receive. In order to ensure that the financial value of
the Perkins III grants was large enough to motivate districts to
meet the Perkins III reporting and performance requirements, the
Arizona Career and Technical Division requires districts be in
Perkins III compliance in order to receive CTE-related state
funds, thereby creating a large incentive for local school
districts to comply with the Perkins III requirements. Indeed, one
district we spoke with lost Perkins III funding, but it was not
until the state linked Perkins III compliance to state funding,
and the district lost the rest of its CTE funding from the state,
that the district started to make significant improvement toward
meeting Perkins III requirements.
In addition, the grantor and grantee should understand the
trade-off between the financial risk-the possibility performance
will not improve sufficiently despite the resource investment-and
the potential return-what will be gained if performance goals are
met or exceeded-in order to determine whether to pursue any
particular performance improvement. Accountability provisions that
contain financial incentives and sanctions can shift risk between
the grantor and grantee. That is, the more the grant award depends
on performance, the greater the financial risk to the grantees: if
they invest but do not perform sufficiently, they do not get paid.
Conversely, in grants with limited or no performance
accountability provisions, the grantor bears the bulk of the
financial risk, since the grantee would receive the grant funds
regardless of the results achieved.
Ensure Effective Distribution
The ability of a performance accountability mechanism to influence
performance also depends on the effective distribution of
organizational rewards and penalties to individuals within the
organization who are directly responsible for the desired
performance.18 For example, Glendale Union School District
officials provide significant financial incentives to every school
employee with whom a student has contact, including teachers,
administrative staff, and other support staff-including the
maintenance staff and bus drivers. The district's philosophy is
that all employees influence the school's atmosphere and academic
achievement and therefore contribute to any success it enjoys.
Incentive funds are distributed based on a school's performance on
13 academic, involvement, and satisfaction-related measures.
According to a district official, the program, started 5 years
ago, has increased camaraderie and collaboration among school
employees, which the official said has contributed to academic
improvement.
In Pennsylvania, the state passes along a portion of the
state-earned federal incentive payments to the counties, according
to each county's proportionate share of the aggregate state CSE
expenditures and to reflect its relative score for each
performance measure, following the performance targets defined in
legislation. Pennsylvania codifies the performance expectations
and incentive payment procedures through cooperative agreements
with each county.
Execute Mechanisms Consistently
The grantor must execute the mechanisms consistently and as
designed to preserve the value of the mechanisms and to avoid
introducing unnecessary risk. For example, if rewards are not paid
as promised the grantee could learn that its additional efforts
are not worth the cost-or risk-and may not make the additional
effort to improve performance. Similarly, if rewards are paid
indiscriminately or if penalties are not levied as expected, the
grantee could learn that no additional effort or investment is
required in order to benefit. In both cases, the system breaks
down and the intended value of the accountability provision is
lost.
We have reported on an agency with the authority to levy penalties
for poor performance that resisted doing so. For example, the
Federal Transit Administration (FTA) has several enforcement tools
to deal with grantees' noncompliance, including warning letters,
suspension of funds, and grant termination. However,
traditionally, FTA had been reluctant to use these tools to
enforce compliance, opting instead to work with grantees in an
effort to continually promote transit development. Reviews also
showed that FTA's oversight was superficial and inconsistent and
that FTA seldom used its enforcement authority to compel grantees
to correct weaknesses, even those that were long-standing.
Consequently, federal dollars had been placed at risk. However, in
response to our 1992 report, FTA established a new enforcement
policy, developed detailed guidance on carrying out enforcement
actions, and has since demonstrated a greater willingness to use
these actions against grantees that do not comply with federal
transit requirements. The Department of Defense (DOD), on the
other hand, has paid billions in incentive and award fees for only
"acceptable, average, expected, good, or satisfactory"
performance. Despite paying billions in fees, DOD has little
evidence to support its belief that these fees improve contractor
performance and acquisition outcomes. The department has not
compiled data, conducted analyses, or developed performance
measures to evaluate the effectiveness of award and incentive
fees. Using accountability mechanisms in this manner undermines
their effectiveness as a motivational tool and marginalizes their
use in holding grantees and contractors accountable for
outcome-based results.19
Periodically Renegotiate and Revise Mechanisms and Measures
Organizations need to allow for and use the flexibility to revise,
update, or improve performance accountability mechanisms in order
to respond to changing needs. In the literature we reviewed, we
found a number of reasons for why accountability provisions may
need to be revised. For example, unintended consequences
associated with performance measures may be discovered only after
full implementation. Organizational priorities may change.
Technology may be introduced that substantially alters performance
expectations. In addition, expectations that were previously
considered stretch goals can become the norm over time-for
example, as productivity gains are realized rewarding such
performance may no longer make sense. Finally, efforts to
reevaluate and revise should consider whether established
accountability provisions are still effective at motivating
performance improvements.
For example, the DMA/MBHP contract demonstrates a situation in
which the entire accountability system experienced a revision to
adjust to contract progression. Incentives in this contract were
initially designed to motivate operational performance, such as
processing time for billing, and performance targets were revised
upward each year as performance improved. This upward revision
helped ensure that performance continued to improve. Once MBHP's
performance reached the highest levels of industry performance in
these areas, further improvements were no longer a priority.
As a result, DMA and MBHP used the annual review to revise the
incentive system from motivating operational improvement to
completing projects designed to improve performance in areas that
would add value to the services MBHP provides, such as a project
on providing behavioral health assistance to the homeless.
There are a number of ways to accommodate the need for periodic
revision. For example, congressional amendments to or
reauthorizations of grant programs allow policymakers the
opportunity to revisit and modify existing provisions and to add
flexibility for agencies that can lead to improved effectiveness.
Agencies can include renegotiation and revision policies in
regulations, guidance, and the terms and conditions of a grant
award. Providing for periodic revision may be particularly
important where performance measures are specified in legislation,
because agency flexibility to respond to changing needs is
significantly reduced. For example, as we have discussed, the
Child Support Performance and Incentive Act of 1998 specifies the
five performance measures, the performance targets, and the
percentage of incentive payments that states can earn for
performance. Initially, states made changes and saw improvement in
these areas. Recently, both state and federal program officials
have expressed concern about the long-term sustainability of such
aggressive targets. For example, program officials said that even
states that have in the past met the 90 percent performance target
for the paternity measure are concerned because more recent annual
rates have dropped back down closer to 80 percent. According to
officials, states initially conducted an extensive caseload
cleanup to improve performance on the five incentive measures when
the incentive program was enacted in 1998, and much of the backlog
of cases that could be addressed relatively easily has been.
However, since the measures and performance targets are
legislatively defined and the CSE program is permanently
authorized, the agency does not currently have the flexibility to
revise the measures or performance targets.
In contrast, state agencies, in negotiation with the Department of
Education, can periodically revise their Perkins III CTE
performance measures and targets during annual negotiations of
their state plans. At program introduction, program targets are
set through the negotiation process between states and OVAE. From
this process, performance targets negotiated initially reflect a
realistic level of what states can actually produce. Next, through
annual application updates, the legislation allows renegotiation
of performance levels with states. Among other factors, OVAE
officials attributed the program's success to this ongoing ability
to renegotiate and revise the program's measures. Although the
Perkins III legislation is similar to the Child Support
Performance and Incentive Act of 1998, in that Perkins defines the
four core indicators tied to the performance measures used in the
incentive program, it provides flexibility that the Child Support
Performance and Incentive Act of 1998 lacks. The flexibility to
revise or update the performance measures is built into the
Perkins III legislation.
Accountability systems by their very nature assume that
performance can be improved. However, performance improvements
depend on adequate time for and ability of participants to learn
from prior actions and use what they have learned to improve
performance from one period to the next.20 Depending upon the
complexity of the task, this process can take many cycles.
Therefore, accountability systems should not be abandoned
prematurely; rather, they should be assessed, revised, and
improved.
Ensure Appropriate Measurement Selection and Usage
Selecting and using appropriate types of performance measures is
important to the effective use of accountability mechanisms. We
have previously reported on general attributes of good performance
measures, noting that measures should be linked to agency goals
and missions; be clearly stated; include measurable targets; and
be objective, reliable, and balanced.21 Specifically, we found
four of these characteristics that highlight key features of
performance measures that can help ensure the successful linking
of performance measures and rewards and penalties:
o the performance being measured should be within the recipient's
sphere of influence,
o the performance measures should be suitable to the mechanism
evaluation cycle, and
o the performance measures and performance data should be tested.
Performance Should Be within Recipient�s Ability and Influence
Performance measures tied to rewards and penalties should
represent performance that can be sufficiently influenced by the
grant recipient's actions. Absent this linkage, the grantee may
have little motivation to change behavior to improve performance,
and the granting agency risks wasting funds by either rewarding
efforts that cannot reasonably be tied to grantee behavior or
penalizing a grantee for outcomes that even its best efforts may
not have prevented. For example, the Temporary Assistance to Needy
Families (TANF) bonus payments22 rewarded states for reducing
out-of-wedlock births. Several studies report, however, that there
does not appear to be a link between the existence of these
programs, or increases in efforts to deliver program services, and
the TANF bonus payment. Many state officials perceive the outcome
measure as inappropriate, relatively difficult to influence, or
both, and discourage attempts to do so. According to one study,
several states reported that they did not compete or did not
continue to compete for the bonus funds because, among other
reasons, their actions would not sufficiently affect the
out-of-wedlock birth rate; therefore they directed their efforts
to activities that were more directly under their influence.
In another example, the Perkins III CTE program has a financial
incentive system that assesses state performance through
performance measures that support its four core indicators-one of
which encourages participation in and completion of programs
leading to nontraditional employment.23 State and local officials
in Arizona said their ability to affect performance for this
indicator is very limited. They told us that although they have
tried to address the barriers to nontraditional employment, they
found that cultural and demographic influences have limited their
ability to improve performance every year. Because performance has
not improved as a result of their efforts, they focus most of
their energy on efforts to improve performance in the other three
core indicator areas, which reflect performance that is more
directly under their control.24
Measures Should Be Suitable to the Mechanism Cycle
Measures should assess performance that can be observed, achieved,
and reported frequently enough to inform the use of awards and
penalties on a timely basis. For example, an annual reward or
penalty should be tied to a measure that is also assessed
annually.
ORC uses performance measures that can be assessed in a relatively
short period of time and that support program outcomes. ORC holds
back 10 percent of ProFac's management fee each month. Each
quarter, ProFac has an opportunity to earn the holdback on the
basis of its performance during the prior quarter on 30 KPIs. For
example, 1 of the quarterly indicators tied to its overall
customer service objective is the "overall customer satisfaction
rate with project delivery." The quarterly assessment is based on
performance information gathered through customer satisfaction
surveys of local managers and facility management contacts for all
alteration projects, capital repairs, or both completed in the
previous quarter. Both ORC and ProFac officials credit the
frequency of evaluation for motivating ProFac to maintain high
performance throughout the year.
OVAE uses the timing of its grant funding distribution cycle to
its advantage in order to motivate states to meet federal
performance accountability requirements. OVAE disburses grant
funds in two pieces: a small portion in July and the remainder in
October. States that did not provide complete, timely performance
data, or missed their performance targets in the prior year, may
have "conditions" put on the July portion of the funding; if
conditions are not met during that quarter, the October funding is
withheld.
Measures and Data Should Be Tested
Performance measures that trigger accountability mechanisms should
be well functioning and time tested before they are linked to
rewards and penalties to minimize the potential for unintended
consequences. Although our literature review did not specify how
long this could take, one study in our review noted that many
leading companies use and test their measurement systems for years
before linking them to accountability provisions.
Performance data should also be tested to make sure they are
credible, reliable, and valid. Absent these attributes,
organizations lack the basis for sound decisions about rewards and
penalties. Data quality is so critical to performance
accountability and oversight of grants that several organizations
use it as the principal performance measure for performance-based
funding. Pinellas County, Florida, alters the case funding rates
paid to its ambulatory service contractor based on data quality.
This "altered funding rate" provision links case reimbursement
rates directly to data quality. For example, data that are
incomplete, illegible, inaccurate, altered, or lacking evidence of
medical necessity-and limit the county's ability to claim for
payment or use its data processing procedures-result in reduced
reimbursements to the ambulatory service contractor for the
affected cases. Pinellas County reports that as a result, ongoing
data quality issues are minimal. In another example, the Child
Support Performance and Incentive Act of 1998 prohibits the
payment of financial incentives to states for performance in
program areas where state data have failed an annual data
reliability test. This requirement ensures that incentive payments
are based on reliable and complete performance information.
Ensure Grantor and Grantee Technical Capacity
Grantor and grantee capacity-specifically, the knowledge about
performance accountability mechanisms and the ability to
effectively implement them-is critical to the effectiveness of
performance accountability systems. For example, when the Air
Force implemented its performance-based contracting program, it
found that employee training focusing on how the performance-based
aspects of the contracts should work were most critical.
Specifically, practices such as providing a step-by-step approach
to the process that outlined who should be involved at each step,
how much of their time and effort would be required at each step,
and what their specific roles and responsibilities would be were
critical to employees understanding what was needed to create
mechanisms to improve performance.
In addition, federal CSE staff in Region III provide a "Child
Support Enforcement Incentives 101" presentation to state and
county CSE staff throughout the region to explain how the
performance measures and incentive payments work. This training
presentation is tailored to the experience of CSE staff and the
demographics of the county, state, or both (large urban, rural,
large interstate caseloads, etc.) but strives to provide a clear
and consistent message: the everyday activities of CSE staff
directly affect the amount of child support available to children
and their families, and drive the amount of incentive payments the
county specifically, and the state in general, earns. The
presentation includes interactive exercises to show how each
employee's casework feeds into outcome-based program results.
Implement System in Stages
Organizations go through a number of stages designing, testing,
and revising measurement systems before linking them to
accountability mechanisms. This longer, phased implementation
allows organizations to ensure the system is effectively designed
before tying it to rewards and penalties. During these stages,
organizations can conduct pilot tests, create financial models,
and conduct behavioral modeling to understand and modify a system
prior to full implementation. For example, according to one
expert, the Tennessee Valley Authority completes a "readiness
test," an assessment of measurement effectiveness and suitability,
before allowing pay for performance or similar financial incentive
systems to be pinned to that measure. This helps avoid unintended
consequences associated with poorly designed measures. Phased
implementation also allows organizations to adjust to new demands
on their time and resources; set up or modify data collection
systems; and ensure the credibility, validity, and reliability of
the data before they are used to measure performance.
For example, the CSE incentive program was implemented in three
stages to allow states to learn about the new incentives and
performance measures. The five performance areas attached to
incentives were developed and legislatively defined in 1998. In
1999, the new data measures were used by the states and audited
for data reliability for the first time. In year one, one-third of
the total incentive funds were allocated based on the new formula
and the remaining funds were allocated based on the old system. In
year two, two-thirds of the funding was allocated using the new
system, and the remaining funds were allocated based on the old
system. In year three, all incentive funding was allocated
according to the new formula.
Collaboration and Oversight Also Key to Success
In addition to these strategies described above, we saw extensive
use of partnerships and collaborations and regular and effective
oversight and feedback. We have previously reported that these
practices are often associated with high-performing
organizations25 and organizations that effectively used
performance information to manage.
Designing and implementing accountability provisions in a
collaborative environment can help develop and encourage buy-in
and support and lead to improvements. For example, Arizona state
and local CTE officials said the state's focus has shifted from a
compliance-focused "audit," ensuring performance data were
properly collected and reported, to a true partnership in which
state and local officials work together to identify and replicate
successes, find solutions to challenges, and thereby improve
performance. State CTE staff spend several days each year meeting
with local CTE officials and providing regular assistance through
on-site technical assistance teams, phone calls, and e-mails.
Oversight and feedback are critical to creating and sustaining
effective performance accountability provisions. We have
previously reported on oversight practices, noting specifically
the value of feedback provided through performance monitoring
plans and tools such as site visits, document reviews, and
evaluations. For example, OVAE employs a number of tools to
provide feedback and assistance to states implementing the Perkins
III vocational education program. Among these tools are
o establishing state guidance that outlines how to meet the
Perkins III performance requirements,
o developing a peer-to-peer mentorship program among states and
with OVAE to share experiences and good practices,
o conducting monthly conference calls with state directors and
data specialists to discuss challenges and solutions to data
collection and quality,
o offering data quality "institutes" and conferences to share
performance measurement and data quality and collection practices,
and
o providing technical assistance to states.
An OVAE official said providing these types of oversight and
feedback activities generated ideas and discussion to help states
improve their performance; the state CTE officials with whom we
spoke agreed.
Various Opportunities Exist at the Federal Level to Enhance
Performance Accountability in Grants
The experiences with and strategies related to federal grant
accountability provisions described in this report suggest a
number of opportunities for Congress and the executive branch to
improve the design and implementation of performance
accountability mechanisms. First, a results-focused design can
help encourage performance accountability in general and
specifically provide for-or at least not prohibit-the use of
accountability mechanisms to encourage desired behavior. In
addition, the use of national program evaluation studies and
research and demonstration grants can provide valuable information
to assist in agency and congressional oversight of and knowledge
about accountability mechanisms. Because credible performance
information and performance measures form the basis for
well-functioning accountability provisions, it remains critical
for Congress and the executive branch to continue to encourage
their development and use. Finally, OMB and agencies can commit to
sharing good practices and lessons learned from experiences with
performance accountability provisions in federal grants-an
efficient and effective way to increase grantor and grantee
knowledge, understanding, and use of these provisions.
A Results-Focused Design Encourages Performance Accountability
Considering grant design features and their implications for
grantee flexibility and accountability can help policymakers
provide for appropriate accountability provisions, whatever type
of grant design is selected. We have previously reported that
policy options reflected in grant design collectively establish
(1) the degree of flexibility afforded to states or localities;
(2) the relevance of performance objectives for grantee
accountability; (3) whether accountability for performance rests
at the federal, state, or local level; and (4) prospects for
measuring performance through grantee reporting and oversight.26
Under a results-oriented approach, federal policymakers would
specify national goals and objectives in statute, enact a process
for establishing them, or adopt some combination of the two. As a
result, when designing or reauthorizing grants, it is important to
consider questions like the following:
o Is there a need for national performance objectives in this
policy area? If so, grantees may be required to use uniform
performance measures-as in the CSE program-to gauge progress. This
allows for comparisons across grantees, and the supporting
performance data collected from grantees have the advantage of
being program specific. However, uniform activities, objectives,
and measures may not exist or may not be desirable, especially
under flexible grant program designs. In these cases, Congress may
instead decide to allow grantees to establish their own program
objectives. For example, the Child Care and Development Block
Grant requires states to certify that they have requirements in
effect to protect the health and safety of children whose child
care is subsidized by the block grant. These requirements must
cover the areas of preventing and controlling for infectious
diseases, physical premise safety, and health and safety training.
However, the specificity and stringency of these requirements and
the manner in which they are enforced is left to the states. The
Perkins III legislation outlines several performance areas and
requires states to determine the measures they will use to
measures progress in these statutorily defined areas. Performance
targets for these measures are negotiated with OVAE. In these
cases, the federal role in monitoring the grants is generally
limited to collecting information on state and local program
efforts and accomplishments as well as evaluating and
disseminating information on best practices. Another option is to
grant temporary exemptions (waivers) from certain federal program
requirements to grantees that demonstrate that the flexibility
granted can lead to performance improvements. For example, Oregon
Option is an intergovernmental partnership that seeks to improve
performance on benchmarks for a broad variety of initiatives,
including childhood immunization, employment for the disabled,
wild salmon recovery, juvenile justice, welfare reform, and child
nutrition, by waiving administrative rules or seeking statutory
change.
o In all cases, what accountability provisions are needed to
support attainment of national performance objectives? These might
include constraints on activities and funds distribution or
operational objectives, standards, and criteria for performance.
These can be set for the program as a whole or delegated to the
level of government responsible for program management. Additional
considerations are as follows: What data are needed for grantee
accountability, and is it feasible to collect these data from
providers? Is it possible to collect data at the project level?
Will the contribution of federal funds be distinguishable from
state, local, and private funds? If the answer to several of these
questions is no, is additional information needed for program
oversight? If so, how will such information be gathered and
reported? The answers to questions such as these provide the basis
for setting grantee reporting requirements.
Careful Use of National Program Evaluation Studies and Research
and Demonstration Grants Can Help Assess Mechanism Performance
Congress has a number of opportunities to conduct oversight, such
as when it establishes or reauthorizes a new program, during the
annual appropriations process, and during hearings focused on
program and agency operations. Providing for-or at a minimum, not
prohibiting-performance accountability mechanisms can provide
timely, targeted performance information and help policymakers
ensure that federal grants focus on their goals, providing another
basis for congressional oversight.
National program evaluations have the potential to answer
questions about both overall program performance as well as the
effectiveness of performance accountability mechanisms, in terms
of their implementation, outcomes, impacts, and
cost-effectiveness. However, national programwide evaluations are
expensive in terms of dollars and time and frequently require
capacities and resources beyond those provided for program
management. Also, while evaluations of multiple sites provide
valuable information, programwide evaluation data are typically
periodic and often cover too few sites to support national
estimates of performance. In these cases, research and
demonstration projects often can provide better information on the
effectiveness of various service delivery methods and approaches.
Knowledge to support effective practice is well established in
some subject areas and can be incorporated into program provisions
(such as service standards) or in companion technical assistance
or knowledge dissemination programs.
Encourage Development and Use of Credible Performance Information
and Performance Measures
As we discussed earlier, performance accountability provisions
rely on a supply of credible, reliable, and valid data and
high-quality performance measures. We found organizations that
recognizing the importance of data quality, tied incentives to
increasing the supply of this type of information. Unfortunately,
as our work on PART27 and GPRA implementation shows, the
credibility of performance data has been a long-standing weakness.
OMB, through its development and use of PART, has provided
agencies with a powerful incentive for improving data quality and
availability. However, improving the supply of performance
information is in and of itself insufficient to sustain
performance management and achieve real improvements in management
and program results. Rather, it needs to be accompanied by a
demand for and use of that information by decision makers and
managers alike. Key stakeholder outreach and involvement is
critical to building demand and, therefore, success. Lack of
consensus by a community of interested parties on goals and
measures and the way that they are presented can detract from the
credibility of performance information and, subsequently, its use.
While congressional buy-in is critical to sustain any major
management initiative, it is especially important for performance
accountability given Congress's constitutional role in setting
national priorities and allocating the resources to achieve them.
Recognizing this, policymakers could use incentives to encourage
program partners to agree on performance measures and targets
against which performance will be judged.
Share Good Practices and Lessons Learned
We and others have frequently reported on the benefits of sharing
promising practices and lessons learned to promote performance
accountability in general in federal programs and program
partners. We believe sharing good practices related to the
effective design and implementation of performance accountability
mechanisms carries similar benefits. As noted earlier, some state
and local agencies' programs have used this type of information
sharing among themselves and their grantees and contractors as a
means of performance improvement.
OMB, as the focal point for overall management in the executive
branch, plays a key role in promoting performance improvement in
federal programs and has developed or contributed to a number of
tools to share information and encourage improvements to federal
grants and program performance. For example, www.grants.gov
includes information on grant opportunities, resources to assist
in writing grant proposals, and a newsletter highlighting recent
grant success stories, and www.results.gov has information on best
practices related to the President's Management Agenda
initiatives-one of which is Budget and Performance Integration
(BPI). Successful implementation of BPI depends significantly on
federal agencies' ability to ensure federal program partners work
toward program goals and are held accountable for results.
Expectmore.gov provides information on PART assessments and
improvement plans; these assessments consider, among other things,
whether the agency regularly collects timely and credible
performance information to manage its programs, and whether the
performance measurements are used to increase accountability.
OMB's own Web site also contains information on and examples of
what it considers to be high-quality PART performance measures;
discussion papers on measurement topics, such as how to
effectively measure what you are trying to prevent; and strategies
to address some of the challenges of measuring research and
development programs.
OMB hosts a number of standing work groups and
committees-comprising agency and OMB staff-to address important
grant-related issues, all of which could accommodate a more
specific focus on grants accountability provisions. For example,
OMB's Chief Financial Officer's Council has a standing grants
policy committee that focuses on grant application and reporting
streamlining. Agency BPI leads meet monthly and recently developed
a subgroup to share lessons learned related to efficiency measures
that balance effectiveness, quality, and cost. They also discuss
strategies to address the challenges of efficiency measures in the
grant context and to develop additional guidance for agencies in
this area.
In addition, OMB hosted a Block and Formula Grant workshop in
October 2005 for federal officials aimed at identifying and
sharing best practices in grants management and performance
measurement. OMB staff agreed that the workshop was a valuable,
efficient, and effective way to share information and lessons
learned and that collectively the participants increased their
knowledge and understanding of ways to enhance grant performance.
They also noted that the real difficulty comes in "what to do
next," in other words, implementing the strategies gleaned from
these sessions.
OMB staff told us that focusing specifically on performance
accountability provisions in grants is necessary and useful, but
that to date, they have focused their governmentwide efforts
primarily on encouraging and enhancing agency capacity to develop
high-quality, results-based program performance measures since
improving the quality of measures and data necessarily precedes
tying them to accountability provisions. The Block and Formula
Grant workshop addressed issues of measurement and accountability,
and several block grant programs have been working to strengthen
grantee accountability.
Conclusions
As the challenges of the 21st century grow, it will become
increasingly important for Congress, OMB, and executive agencies
to consider how the federal government can maximize performance
and results. This will be particularly important for federal grant
program managers, given the significant amount of federal
resources invested in these tools. Because many national
objectives can only be achieved through state, local, and
nongovernmental organizations, enhancing performance
accountability below the federal level is equally important. In
this report, we identify a variety of accountability mechanisms as
well as key strategies to enhance their use. Collectively, these
can help enhance and sustain performance accountability in grants
at all levels of government.
As the cases we described illustrate, rewards and penalties are
fundamental tools to help drive and motivate desired behaviors,
but performance accountability mechanisms are not one size fits
all; there is no universal transferable mechanism applicable to
all programs. The specific mechanisms used by agencies and
programs and highlighted throughout this report may not be
universally adopted by other federal agencies and programs seeking
to improve their own programs. Nevertheless, many can be tailored
to specific grant programs, and the key strategies can be adapted
to address the specific accountability challenges each agency
faces.
Like all successful change initiatives, the progress currently
under way to move from traditional fiscal accountability in grants
to greater accountability for performance will take time;
accountability provisions-and the performance measures associated
with them-can take many years to mature. Although some federal
programs are well on their way to collecting and reporting on
reliable, credible, and valid data that support high-quality
outcome goals agreed to by all program partners, many others are
still struggling with how to define appropriate outcome measures.
It will be critical to proceed thoughtfully and implement
performance accountability in phases, building in enough
opportunities to learn from mistakes and revise measures and
mechanisms to reap the benefits of performance management while
minimizing perverse incentives and unintended consequences.
As with all challenges, starting with small steps is often the
best way forward. Accountability provisions can be used to bring
program partners together to identify common ground. For example,
programs that struggle with defining appropriate outcome goals,
measures, and targets may wish to tie incentives to reaching
agreement on them. Those that struggle with poor data quality and
data definitions could reward grantees for progress in this area.
Performance accountability-especially in the early stages-must be
constructive, not punitive. Even if penalties are employed to
promote performance accountability, there should be a
constructive, collaborative approach to performance improvement
that precedes them. Tying performance to lower risk, nonfinancial
mechanisms may at first be more acceptable until performance
measures have been time tested and revised as needed and grantees
have had time to collect the necessary data to support the
measures. Above all, a collaborative process that includes
Congress, the executive branch, and grantees will be critical to
developing successful performance accountability systems.
Accountability provisions assume that performance can be
improved-but this requires information sharing and feedback. OMB
has a central role in overseeing the performance and
accountability in the federal government, and has used its role to
promote general results-oriented performance measurement and
management practices in federal grants through Web sites,
guidance, work groups, and workshops. Each of these tools and
strategies could be expanded on to specifically promote and
encourage performance accountability in federal grants, both among
related federal grant programs-programs that have a common
purpose-and federal grant types-such as categorical grants, block
grants, and funding streams. Sharing good practices and lessons
learned and providing feedback on performance are valuable
practices that can leverage resources to enhance knowledge and
further performance accountability. Leading practices can be
shared within and among agencies, grant programs, grantees, and
even grant types. OMB recognized the value in sharing information
on performance accountability mechanisms, but has not yet focused
on this issue.
Recommendation for Executive Action
We are therefore recommending that the Director of OMB encourage
and assist federal agencies in working with the Congress to expand
the effective use of performance accountability mechanisms,
focusing on the practices in this report, when federal grant
programs are being created or reauthorized. We further recommend
that OMB offer opportunities for knowledge transfer among federal
agencies and encourage agencies to share leading practices and
lessons learned in implementing grant accountability mechanisms.
Possible vehicles for the collection and dissemination of this
information include good practices guides and workshops and Web
sites such as results.gov, grants.gov, and expectmore.gov.
Agency Comments
On August 22, 2006, we provided a draft of this report to the
Director of OMB and the Secretaries of Education and Health and
Human Services. We also provided relevant sections of a draft of
this report to the grantees and contractors highlighted in this
report. We received technical comments from all three agencies,
which were incorporated as appropriate. In addition, OMB agreed
with our recommendation but suggested we broaden it to address the
role of federal agencies and Congress in the grant design and
reauthorization process. We agree, and have amended our
recommendation accordingly.
We are sending copies of this report to the Director of the Office
of Management and Budget, the Secretaries of Education and Health
and Human Services, and other interested parties. We will also
make copies available to others upon request. In addition, the
report is available at no charge on GAO's Web site at
http://www.gao.gov .
Please contact me on (202) 512-6543 or [email protected] if you
or your staff have any questions about this report. Contact points
for our Offices of Congressional Relations and Public Affairs may
be found on the last page of this report. Key contributors to this
report are acknowledged in appendix II.
Sincerely yours,
Bernice Steinhardt Director, Strategic Issues
Appendix I: Objectives, Scope, and Methodology
The objectives of this report were to identify (1) the challenges
to performance accountability in grants; (2) the kinds of
mechanisms that are being used to improve grant performance and
how; and (3) given the findings of questions 1 and 2, what
strategies the federal government can use to encourage the use of
these mechanisms, as appropriate.
To meet the first and second objectives, we interviewed experts in
grants and performance management, including individuals from the
following organizations: the School of Public Policy at the
University of Maryland, the University of Central Florida, the
John F. Kennedy School of Government at Harvard University, the
John C. Stennis Institute of Government at Mississippi State
University, the Public and International Affairs Department at
George Mason University, the Political Science Department at the
University of New Hampshire, Measurement International, and the
American Productivity and Quality Center.
Based on our literature review, we developed a coding scheme for
identifying (1) types of performance accountability mechanisms and
(2) strategies used to successfully design and implement these
mechanisms. We used these codes in a content analysis we conducted
on a subset of the documents we reviewed. We chose the documents
for content analysis based on the following criteria:
o discussed accountability systems, mechanisms, or both,
discussed general practices that facilitated to the effective use
of accountability mechanisms, or provided case examples;
o published in 1993 or later;
o found in major electronic databases; and
o published in the United States.
The content analysis was conducted by two analysts, with the
second analyst conducting a dependent review. Discrepancies in
coding were discussed and agreement reached between the two
analysts. Our analysis produced an inventory of performance
accountability mechanism types and five strategies used to
facilitate the effective design and implementation of performance
accountability mechanisms. See the bibliography for documents
included in our review.
To illustrate the mechanisms and strategies identified through our
content analysis, we used relevant case examples found in the
literature. To further illustrate the mechanisms and design and
implementation strategies, we also selected four additional case
illustrations-two federal grant programs and two nonfederal
contract cases. These four cases were selected based on our
literature review, interviews with experts, and reviews of prior
GAO work because they are good examples of where (1) a performance
mechanism was present and (2) there is reason to believe that
performance improved at least in part because of the mechanism.
To screen and develop the grant case illustrations, we interviewed
regional and headquarters federal agency officials and officials
at county/local offices. We also reviewed grant legislation,
program guidance, and prior studies. To develop contract case
illustrations, we interviewed officials both from the contracting
agencies and the contractors and reviewed the contract.
To address our third objective, we synthesized prior GAO work, and
we interviewed officials at the Office of Management and Budget.
We conducted our work from December 2005 through August 2006 in
accordance with generally accepted government auditing standards.
Appendix II: GAO Contact and Staff Acknowledgments
GAO Contact
Bernice Steinhardt (202) 512-6543 or [email protected]
Acknowledgments
Jackie Nowicki (Assistant Director) and Chelsa Gurkin (Senior
Analyst-in-Charge) managed this assignment. David Bobruff, Katie
Hamer, and Anne Marie Morillon made significant contributions to
all aspects of the work. Kate France significantly contributed to
the initial research and design of the assignment. In addition,
Tom Beall and Jay Smale provided methodological assistance, Amy
Rosewarne provided key assistance with message development, and
Donna Miller developed the report's graphics.
Bibliography
American Productivity & Quality Center. Achieving Organizational
Excellence Through the Performance Measurement System. Houston:
1999.
American Productivity & Quality Center. Measure What Matters:
Aligning Performance Measures with Business Strategy. Houston:
1999.
Ashworth, Karl, and others. "When Welfare-to-Work Programs Seem to
Work Well: Explaining Why Riverside and Portland Shine So
Brightly." Industrial & Labor Relations Review, vol. 59, iss. 1
(2005).
Ausink, John, and others. Implementing Performance-Based Services
Acquisition (PBSA): Perspectives from an Air Logistics Center and
a Product Center. A Documented Briefing prepared by the RAND
Corporation for the U.S. Air Force. 2002.
Ausink, John, Frank Camm, and Charles Cannon. Performance-Based
Contracting in the Air Force: A Report on Experiences in the
Field. A Documented Briefing prepared by the RAND Corporation for
the U.S. Air Force. 2001.
Baker, George. "Distortion and Risk in Optimal Incentive
Contracts." The Journal of Human Resources, vol. 37, no. 4 (2002).
Banta, Trudy W., and others. "Performance Finding Comes of Age in
Tennessee." The Journal of Higher Education, vol. 67, no. 1
(1996).
Conlon, Timothy. "Grants Management in the 21stCentury: Three
Innovative Policy Responses." IBM Center for the Business of
Government: Financial Management Series. Washington, D.C.: October
2005.
Courty, Pascal, and Gerald Marschke. "Measuring Government
Performance: Lessons from a Federal Job-Training Program." The
American Economic Review, vol. 82, no. 2 (1997).
Dodds, Dan R. "Performance Incentives Can Spark Greater
Productivity." School Administrator, vol. 55, iss. 3 (1998).
Domestic Working Group: Grant Accountability Project. Guide to
Opportunities for Improving Grant Accountability. Washington,
D.C.: October 2005.
Dye, Jane Lawler, and Harriet B. Presser. "The State Bonus to
Reward a Decrease in `Illegitimacy': Flawed Methods and
Questionable Effects." Family Planning Perspectives, vol. 31, no.
3 (1999).
Eberts, Randall, Kevin Hollenbeck, and Joe Stone. "Teacher
Performance Incentives and Student Outcomes." The Journal of Human
Resources, vol. 37, no. 4 (2002).
Garber, Alan M. "Evidence-Based Guidelines as a Foundation for
Performance Incentives." Health Affairs, vol. 24, no. 1 (2005).
Gordon, Stephen B. "Performance Incentive Contracting: Using the
Purchasing Process to Find Money Rather Than Spend It." Government
Finance Review, August (1998).
Hatry, Harry P., and others. How Federal Programs Use Outcome
Information: Opportunities for Federal Managers. Washington, D.C.:
IBM Center for the Business of Government, May 2003.
Hildebrandt, Gregory G. "The Use of Performance Incentives in DOD
Contracting." Acquisition Review Quarterly, Spring (1998).
Johnston, Jocelyn M., and Barbara S. Romzek. "State Social
Services Contracting: Exploring the Determinants of Effective
Contract Accountability." Public Administration Review, vol. 65,
iss. 4 (2005).
Kathuria, Sandeep. "An Overview of Share-in Savings Contracting."
Contract Management, vol. 45, no. 11 (2005).
King, Donald C., and Paul D. Tolchinsky. "Do Goals Mediate the
Effects of Incentives on Performance?" Academy of Management
Review, vol. 5, no. 3 (1980).
Lane, Nancy E. "Performance Incentives in The Massachusetts
Behavioral Health Program." Administration and Policy in Mental
Health, vol. 32, no. 4 (2005).
Marcus, Alfred A., Barry M. Mitnick, and Kiran Verma. "Making
Incentive Systems Work: Incentive Regulation in the Nuclear Power
Industry." Journal of Public Administration Research and Theory:
J-PART, vol. 9, no. 3 (1999).
Martin, Lawrence L. "Making Performance-Based Contracting Perform:
What the Federal Government Can Learn from State and Local
Governments." IBM Endowment for the Business of Government: New
Ways to Manage Series. Washington, D.C.: 2002.
Martin, Lawrence L. Performance-Based Contracting (PBC) for Human
Services: A Review of the Literature. Orlando: Center for
Community Partnerships, University of Central Florida, 2003.
Metzenbaum, Shelley H. "Performance Accountability: The Five
Building Blocks and Six Essential Practices." IBM Center for the
Business of Government: Managing for Performance and Results
Series. Washington, D.C.: 2006.
Rand Corporation. Organizational Improvement and Accountability:
Lessons for Education from Other Sectors. 1st ed. Santa Monica,
Calif.: 2004.
Sammer, Joanne. "Making Incentive$ Pay." Industry Week: Leadership
in Manufacturing. (Cleveland: Penton Media Inc., August 2002).
http://www.industryweek.com/ReadArticle.aspx?ArticleID=1121
(downloaded Mar. 9, 2006).
Tsimbinos, John M. "Are You Willing to Pay for Success?"
Bottomline, vol. 7, no. 6 (1990).
U.S. Department of Defense, Deputy Under Secretary of Defense for
Acquisition Reform. Incentive Strategies for Defense Acquisitions.
Washington, D.C.: 2001.
U.S. Department of Energy, Office of Inspector General. Inspection
of Selected Aspects of the Office of River Protection
Performance-Based Incentive Program. Washington, D.C.: 2001.
GAO�s Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in
meeting its constitutional responsibilities and to help improve
the performance and accountability of the federal government for
the American people. GAO examines the use of public funds;
evaluates federal programs and policies; and provides analyses,
recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at
no cost is through GAO's Web site ( www.gao.gov ). Each weekday,
GAO posts newly released reports, testimony, and correspondence on
its Web site. To have GAO e-mail you a list of newly posted
products every afternoon, go to www.gao.gov and select "Subscribe
to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies
are $2 each. A check or money order should be made out to the
Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are
discounted 25 percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
(202) 512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
[email protected] Automated answering system: (800) 424-5454 or
(202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400
U.S. Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW,
Room 7149 Washington, D.C. 20548
13We identified these strategies through our literature review, and
illustrate them with examples from the literature and from our additional
four case illustrations.
14 GAO-04-343SP , GAO-05-927 .
15Pub. L. No. 105-200.
16The five performance goals are (1) paternity establishment, (2) child
support order establishment, (3) collections on current support due, (4)
collections on arrears, and (5) cost-effectiveness.
17The Lewin Group, Study of the Implementation of the Performance-Based
Incentive System, October 2003.
18GAO, Results-Oriented Cultures: Creating a Clear Linkage between
Individual Performance and Organizational Success, GAO-03-488 (Washington,
D.C.: Mar. 14, 2003).
19GAO, Defense Acquisitions: DOD Has Paid Billions in Award and Incentive
Fees Regardless of Acquisition Outcomes, GAO-06-66 (Washington, D.C.: Dec.
19, 2005).
20GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing
Season Performance Measures, GAO-03-143 (Washington, D.C.: Nov. 22, 2002).
21 GAO-03-143 .
22The Personal Responsibility and Work Opportunity Reconciliation Act of
1996 authorized the Bonus to Reward Decrease in Illegitimacy Ratio, a
provision intended to motivate states to pursue nonmarital birth
prevention programs. This provision awarded up to $25 million in each of
fiscal years 1999 through 2002 to as many as five states showing the
largest reduction in nonmarital births.
23Nontraditional employment relates to the participation of students in
fields in which their gender constitutes less than 25 percent of the
individuals employed in that field (e.g., female students participating in
automotive repair programs).
24However, as discussed earlier, states can periodically revise their
Perkins III CTE performance measures and targets during annual
negotiations of their state plan.
25The other attributes of high-performing organizations are a clear,
well-articulated, and compelling mission; a focus on the needs of the
clients and customers; and the strategic management of people.
26 GAO/GGD-98-137 .
27OMB developed PART as a diagnostic tool meant to provide a consistent
approach to assessing federal programs during the executive budget
formulation process. PART covers four broad topics for all programs
selected for review: (1) program purpose and design, (2) strategic
planning, (3) program management, and (4) program results. We have
previously reported on PART in GAO, Performance Budgeting: PART Focuses
Attention on Program Performance, but More Can Be Done to Engage Congress,
GAO-06-28 (Washington, D.C.: Oct. 28, 2005).
(450454)
*** End of document. ***