[Analytical Perspectives]
[Performance and Management Assessments]
[2. Budget and Performance Integration]
[From the U.S. Government Printing Office, www.gpo.gov]
[[Page 9]]
2. BUDGET AND PERFORMANCE INTEGRATION
I. INTRODUCTION
The American taxpayer expects the Federal Government to implement
programs that will ensure the Nation's security and provide critical
services. Taxpayers deserve to have their money spent wisely to create
the maximum benefit. The Executive Branch should be held accountable for
program performance by the American people. For the Federal Government
to be held accountable, the American people must have clear, candid
information about each program's success and failures. The
Administration is providing this type of information. More importantly,
in all cases, the Administration is implementing detailed plans to
improve program performance.
The role of the President's Budget and Performance Integration (BPI)
Initiative is to ensure that Federal dollars produce the greatest
results. To accomplish this, agencies and OMB identify which programs
work, which are deficient, and what can be done to improve the
performance of them all. In some cases, it may be necessary to
reallocate funding to more effective programs. This and other decisions
about programs are ultimately made jointly by the Congress and the
President, but the analysis of program performance can help the
Executive and Legislative Branches make more informed decisions. To
expand the use of information about program performance, OMB is
launching ExpectMore.gov, a user-friendly website that provides the
public with performance information about Federal programs. (Greater
detail about ExpectMore.gov will be provided in a subsequent section.)
The Budget and Performance Integration Initiative measures its success
in two principal ways:
Improved Program Performance: Through the use of performance
assessments, programs will have the information they need to
improve their performance every year. The initiative requires
each agency to identify opportunities to improve program
management and design, and then develop and implement clear,
aggressive plans to get more for tax dollars every year.
Greater Investment in Successful Programs: Overall, scarce
resources need to be allocated to programs that benefit the
Nation most effectively and efficiently. Program performance
will not be the only factor in decisions about how much
funding programs receive. However, the Congress and the
President, equipped with information about program peformance
can consider performance to a greater degree in their
decision-making and invest primarily in programs that provide
the greatest return on the investment of taxpayer dollars. If
poor performing programs are unable to demonstrate improved
results, then that investment may be reallocated to programs
that can demonstrate greater success.
Currently, the Initiative is showing great progress toward the first
goal. Programs are becoming more efficient and more effective through
implementation of meaningful improvement plans.
Many programs are demonstrating improved results.
The Department of Veterans Affairs is reducing the time
veterans wait to get medical appointments. From 2001 to 2005,
the Veterans Health Administration (VHA) substantially reduced
the number of new veteran enrollees unable to schedule an
appointment for medical care from a high of 176,000 to 22,494.
VHA remains a leader in customer satisfaction, with an
inpatient satisfaction score of 84 out of 100 on the American
Customer Satisfaction Index, slightly higher than the score of
79 for comparable private sector services.
To reduce fatalities from automobile accidents, the National
Highway Traffic Safety Administration promoted greater seat
belt use among high-risk groups such as younger drivers, rural
populations, pick-up truck occupants, 8-15 year-old
passengers, occasional safety belt users, and motor vehicle
occupants in States with secondary safety belt use laws. As a
result, nationwide seat belt use increased from 73 percent in
2001 to 82 percent in 2005, an all-time high.
Agencies are also identifying the steps they will take to improve each
program's performance even more. All programs, regardless of whether
they perform poorly or well, should strive to perform better each year.
Progress toward the second goal of improving resource allocation is
slow. Overall high performers received larger funding increases than
those that did not perform as well, but in general, recommendations to
reduce funding for ineffective programs or those that can not
demonstrate results have been less successful.
II. HOW THE BUDGET AND PERFORMANCE INTEGRATION INITIATIVE WORKS
There are several aspects of the Initiative designed to maximize
program performance:
Assess performance with the PART (Program Assessment Rating
Tool)
[[Page 10]]
Publish a Scorecard to hold agencies accountable for
managing for results, addressing PART findings, and
implementing follow-up actions
Broadcast results on a new website, ExpectMore.gov
Implement inter-agency program improvement
Comprehensive Assessment by the Program Assessment Rating Tool (PART)
How do we ensure programs are improving every year? First, we assess
their current performance. In order to improve program outcomes, it is
critical to have a good understanding of how the program is currently
performing. To date, we have assessed the performance of 80 percent of
all Government programs using the PART.
------------------------------------------------------------------------
What is the PART and How is it Used?
The PART helps assess the management and performance of individual programs. With the PART, agencies and OMB
evaluate a program's purpose, design, planning, management, results, and accountability to determine its
overall effectiveness. Recommendations are then made to improve program results.
To reflect that Federal programs deliver goods and services using different mechanisms, the PART is customized
by program type. The seven PART types are: Direct Federal, Competitive Grant, Block/Formula Grant, Research and
Development, Capital Assets and Service Acquisition, Credit, and Regulatory. The PART types apply to both
discretionary and mandatory programs. ExpectMore.gov also classifies each program by its specific program area
(such as environment, transportation, education, etc) so we can accelerate the improved performance of programs
with similar missions.
Each PART includes 25 basic questions and there are additional questions tailored to different program types.
The questions are divided into four sections. The first section of questions gauges whether a program has a
clear purpose and is well designed to achieve its objectives. The second section evaluates strategic planning,
and weighs whether the agency establishes outcome-oriented annual and long-term goals for its programs. The
third section rates the management of an agency's program, including the quality of efforts to improve
efficiency. The fourth section assesses the results programs can report with accuracy and consistency.
The answers to questions in each of the four sections result in a numerical score for each section from 0 to 100
(100 being the best score). Because reporting a single weighted numerical rating could suggest false precision,
or draw attention away from the very areas most in need of improvement, numerical scores are combined and
translated into qualitative ratings. The bands and associated ratings are as follows:
------------------------------------------------------------------------
Rating Range
------------------------------------------------------------------------
Effective.................................................... 85-100
Moderately Effective......................................... 70-84
Adequate..................................................... 50-69
Ineffective.................................................. 0-49
------------------------------------------------------------------------
Regardless of overall score, programs that do not have acceptable performance measures or have not yet collected
performance data generally receive a rating of ``Results Not Demonstrated.'' This rating suggests that not
enough information and data are available to make an informed determination about whether a program is
achieving results.
PART ratings do not result in automatic decisions about funding. Clearly, over time, funding should be targeted
to programs that can prove they achieve measurable results. In some cases, a PART rating of ``Ineffective'' or
``Results Not Demonstrated'' may suggest that greater funding is necessary to overcome identified shortcomings,
while a funding decrease may be proposed for a program rated ``Effective'' if it is not a priority or has
completed its mission. However, most of the time, an ``Effective'' rating is an indication that the program is
using its funding well and that major changes are not needed.
------------------------------------------------------------------------
[[Page 11]]
Publish a Scorecard To Hold Agencies Accountable
Agencies are achieving greater results with the help of the habits and
discipline established through the Budget and Performance Integration
Initiative (BPI). These agencies recognize that the PART can be a useful
tool to drive the agencies to improved performance.
The President's Management Agenda established clear, Government-wide
goals or Standards for Success (http://results.gov/agenda/standards.pdf)
for several key areas, one of which is Budget and Performance
Integration. Agencies have developed and are implementing detailed,
aggressive action plans to achieve these goals. Most importantly,
agencies are held publicly accountable for adopting these disciplines.
The Standards for Success for the BPI Initiative are below:
Meets quarterly with senior agency managers to examine
reports that integrate financial and performance information
that covers all major responsibilities of the Department.
Agency achieves planned improvements in program performance
and efficiency in achieving results each year;
Develops strategic plans that contain a limited number of
outcome-oriented goals and objectives. Annual budget and
performance documents incorporate measures identified in the
PART and focus on the information used in the senior
management report described in the first criterion;
Demonstrates that it has performance appraisal and awards
systems for all Senior Executive Service (SES) and managers,
and more than 60 percent of the workforce, that effectively:
link to agency mission, goals, and outcomes; hold employees
accountable for results appropriate for their level of
responsibility; differentiate between various levels of
performance (i.e., multiple performance levels with at least
one summary rating above Fully Successful); and provide
consequences based on performance. In addition, at a beta
site, there is evidence that clear expectations are
communicated to employees; rating and awards data demonstrate
that managers effectively planned, monitored, developed and
appraised employee performance; and the site is ready to link
pay to the performance appraisal systems. The agency is
working to include all agency employees under such systems;
Reports the full cost of achieving performance goals
accurately in budget and performance documents and can
accurately estimate the marginal cost of changing performance
goals;
Has at least one efficiency measure for all PARTed
programs; and
Uses PART evaluations to direct program improvements, and
PART ratings and performance information are used consistently
to justify funding requests, management actions, and
legislative proposals. Less than 10 percent of agency programs
receive a Results Not Demonstrated rating for two years in a
row.
Each quarter, agencies received two ratings. First, they are rated on
their status in achieving the overall goals for each initiative. They
are then given a green, yellow or red rating to clearly announce their
performance. Green status is for success in achieving each of the
criteria listed earlier; yellow is for an intermediate level of
performance; and red is for unsatisfactory results.
Second, agency progress toward reaching the Budget and Performance
Integration standards is assessed separately. This is reviewed on a case
by case basis against the work plan and related time lines established
for each agency. Progress is also given a color rating. Green is given
when implementation is proceeding according to plans agreed upon with
the agencies; Yellow for when some slippage or other issues require
adjustment by the agency in order to achieve the initiative objectives
on a timely basis; and Red when the Initiative is in serious jeopardy.
In this case, it is unlikely to realize objectives absent significant
management intervention.
As of December 31, 2005, nine agencies achieved green status on the
Budget and Performance Integration Initiative Scorecard. The agencies at
green are:
1. Department of Energy
2. Department of Labor
3. Department of Transportation
4. Department of State
5. National Aeronautics and Space Administration
6. National Science Foundation
7. Small Business Administration
8. Social Security Administration
9. U.S. Agency for International Development
The Scorecard is an effective accountability tool to ensure agencies
manage the performance of their programs. Although a scorecard rating is
not directly linked to any specific consequences, it is quickly
understood at the highest levels of the Administration as an indicator
of an agency's strength or weakness.
The Government-wide scorecard reporting on individual agency progress
is published quarterly at http://results.gov/agenda/scorecard.html.
Broadcast Results on ExpectMore.gov
This year, a new website, ExpectMore.gov, will provide Americans with
candid information about which programs work, which do not, and what all
programs are doing to get better every year.
Up until now, Americans have had limited access to information on how
the Federal Government performs. In many cases, the Federal Government
performs well. In some cases, it performs better than the private
sector.
This site will contain PART summaries for all programs that have been
assessed to date. The site will provide the program information a
concerned citizen would need to assess a program's performance. Each
assessment includes a brief description of the program's purpose, its
overall rating, some highlights about its performance and the steps it
will take to improve in
[[Page 12]]
the future. For those interested in more information, there are links to
the detailed program assessment, as well as that program's website and
the assessment summaries of other similar programs. The detailed PART
assessment includes the answer to each PART question with an explanation
and supporting evidence. It also includes the performance measures for
the program along with current performance information. In addition,
there is an update on the status of follow-up actions to improve program
performance.
A visitor to the site may find, at least initially, programs are not
performing as well as they should or program improvement plans are not
sufficiently ambitious. We expect this site to change that. The website
will have a variety of benefits. It will:
Increase public attention to performance;
Draw greater scrutiny to agency action (or inaction) to
improve program results:
--Improvement plans will be transparent
--Statements about goals and achievements will
be clearer; and
Create demand for better quality and more timely performance
data.
Implement Inter-Agency Program Improvement
The Administration continues to look for new ways to improve the
performance of programs with similar purpose or design by using the PART
to analyze performance across agencies (i.e., cross-cutting analysis).
Cross-cutting analysis can improve coordination and communication by
getting managers from multiple agencies to agree to a common set of
goals and placing the focus on quantifiable results. This type of
analysis breaks down barriers across the Federal, State, and local
levels so all are working toward the same goal. Only topics that are
expected to yield meaningful results are selected for cross-cutting
analyses. This past year the Administration completed cross-cutting
analyses of block grant programs, Small Business Innovation Research,
and credit programs.
Block Grants. One of the most common tools used by the Federal
Government is the block grant, particularly in the social services area
where States and localities oftentimes award grants to local service
providers. Block grants are embraced for their flexibility to meet local
needs and criticized because accountability for results can be difficult
when funds are allocated based on formulas and population counts rather
than achievements or needs. In addition, block grants pose performance
measurement challenges precisely because they can be used for a wide
range of activities. The obstacles to measuring and achieving results
through block grants are reflected in PART scores: they receive the
second lowest average score of the seven PART types: 8 percent of block
grant programs assessed to date are rated ineffective, and 45 percent
are results not demonstrated.
The characteristics that distinguish high performing block grant
programs from low performing ones are:
Top management is committed to managing for results;
[[Page 13]]
Strong, outcome-oriented performance measures and goals are
used by management and grantees;
Performance information is relevant, transparent and
accessible so management and grantees can easily find out what
works and replicate it
Program performance is incorporated into managers' and
employees' performance appraisals.
The goal of this cross-cutting analysis was to share best practices
for block grant programs across agencies. During this past year, the BPI
Initiative led a seminar where multiple agencies learned lessons about
performance measurement, accountability, data collection, and reporting
for block grants.
All block grant programs will integrate the lessons from this work
into aggressive improvement plans that ensure:
Grantees and subgrantees strive to achieve outcome-oriented
goals;
Data on whether those goals are achieved are collected and
made public; and
Information about proven interventions and how to implement
them is shared widely.
The long term impact of this work will be visible over the coming
years as we monitor the ability of these programs to create better
outcomes for the citizens they serve.
Small Business Innovation Research (SBIR). The SBIR program sets aside
2.5 percent of Government research and development contract and grant
funding for small businesses. The goal of the program is to assist small
businesses in undertaking and obtaining the benefits of research and
development leading to commercial products, while assisting agencies in
achieving their missions. Approximately $2 billion was spent last year
in SBIR programs.
All Federal agencies with Research and Development budgets above $100
million per year must publish a list of technical topics that they would
like to support, after which small businesses are encouraged to submit
research funding proposals addressing opportunities in those areas.
First, agencies provide winning companies seed funding to explore the
feasibility of their projects and, if deemed promising after initial
investigation, funding is provided for subsequent research and
development. Awards generally are limited to less than $1 million per
project. Agencies monitor the progress of the selected projects and
report key data annually to the Small Business Administration.
A team, consisting of agency and OMB representatives, is carrying out
the following activities:
Assessing the program's impact;
Focusing on improving program administration;
Determining if legislative reform is needed;
Developing common long-term and annual measures; and
Developing a database that tracks commercialization and
sales in a consistent manner.
Credit Programs. The Federal Government is one of the world's largest
lenders. At the end of 2003, the Government held a financial asset
portfolio of nearly $1.5 trillion, including direct loans, loan
guarantees, defaulted loans, and non-credit debt owed to agencies. Many
agencies lack the data, processes, or overall understanding of the
credit lifecycle (origination, loan servicing/lender monitoring,
liquidation, and debt collection) needed to effectively assist intended
borrowers while also proactively reducing errors, risk, and cost to the
Government. Some credit program PART scores reflect these fundamental
inefficiencies. More information about the performance of credit
programs is available in chapter 7 in this volume.
The Budget and Performance Integration initiative identified the
``back office'' function of the five largest credit agencies
(Agriculture, Education, Housing and Urban Development, Small Business
Administration, and Veterans Administration) and Treasury as an
appropriate target for analysis. The Deputy Director for Management
created a Council to address improvements in these back office
functions. The Federal Credit Council convened its first meeting in
March 2005.
In order to create accountability in credit programs, the President's
Management Agenda scorecard has been expanded to include a set of
standards for credit program management. The standards include criteria
for red, yellow and green status related to:
loan origination;
servicing and/or lender monitoring; and
debt collection.
The first scorecard will be published subsequent to the President's
2007 Budget, with quarterly scorecard reports describing individual
agencies' milestones for addressing weaknesses.
Many agencies lack the systems and data to conduct regular analysis
consistent with minimum private sector standards, resulting in larger
than anticipated losses to the Government. For example, institution of
early warning systems to identify high-risk borrowers and provide
targeted intervention at agencies currently without such systems could
reduce defaults substantially, given the size of agency portfolios. The
Council is working to improve compliance with the provision of the Debt
Collection Improvement Act that bars certain borrowers through increased
reporting to, and use of, private credit bureaus. This permits better
identification of delinquent Federal debtors and avoids extending
additional credit to poor credit risks. Savings to the Government are
expected to be up to $100 million per year.
The Council has substantially completed the Sharing Lender Performance
Data (SLPD) portal that allows comparison of private lenders' default
and delinquency rates, and other portfolio data, across agencies. This
will result in better decisions to approve lender participation in
programs, provide benchmarks for ranking lenders, and could provide an
additional monitoring tool to reduce borrower defaults through early
action.
Initiatives of the Council aim to improve management functions and
have the potential to reduce delinquent debt by up to $10 billion, in
addition to substantial
[[Page 14]]
cost savings on the front end in the form of reduced administrative and
subsidy cost expenses.
Community and Economic Development Programs. The Federal Government
spends more than $16 billion annually to support local economic and
community development. In 2004, agencies and OMB participated in a
crosscutting review of the 35 Federal programs that make up this effort.
Based on PART analyses, input from agencies, and other program
information, the team identified common problems that reduced the
effectiveness of this Federal spending. They concluded that the
programs, taken together, were duplicative, not well-targeted, and in
many cases lacked clear goals, and a system to measure community
progress and evaluate program impacts.
Last year's budget proposed to consolidate 18 of the programs (which
spend about $4.8 billion) in a new Strengthening America's Communities
Initiative. For 2007, the Administration re-proposes program
consolidation--this time in HUD and Commerce. The consolidation will be
accompanied by three major reforms to make more effective use of these
resources by: 1) better targeting funds to places that lack the means to
create conditions for economic progress, 2) consolidating overlapping
and/or ineffective programs into flexible grants that include rewards
for community progress and results, and 3) coordinating the full set of
Federal economic and community development programs within a common
framework of goals, standards, and outcome measures.
III. RESULTS
As mentioned above, the BPI Initiative measures its success according
to two measures:
Improved Program Performance; and
Greater Investment in Successful Programs
There has been a good deal of success toward achieving goals of the
first measure. The BPI Initiative has caused agencies to think more
systematically about how they measure and improve program performance.
Though there are many factors that impact program performance, it is
clear that the BPI Initiative has framed the discussion around results.
Agencies have developed ways to measure their efficiency so they can
figure out how to do more with Americans' tax dollars.
This marks the fourth year that the PART was used to (1) assess
program performance, (2) take steps to improve program performance, and
(3) help link performance to budget decisions. To date, the
Administration has assessed 794 programs, which represent approximately
80 percent of the Federal budget. Over the next year, the Administration
will use the PART to assess the performance and management of most of
the remaining Federal programs.
With the help of the PART, we have improved program performance and
transparency. There has been a substantial increase in the total number
of programs rated either ``Effective'', ``Moderately Effective'', or
``Adequate''. This increase came from both re-assessments and newly
PARTed programs. The chart below shows the percentage of programs by
ratings category.
[[Page 15]]
The results demonstrate that the BPI initiative is having success
focusing Agencies' attention on program performance. For example,
approximately:
1 in 7 programs has improved its PART rating;
Half of programs rated Results Not Demonstrated have
improved their ratings;
80 percent of programs have acceptable performance
measures;
40 percent have achieved their long-term goals and 60
percent have achieved their annual goals; and
80 percent of programs have efficiency measures and about
half of them have achieved their efficiency targets.
Unfortunately, there has not been a similar level of accomplishment in
the second measure: Greater Investment in Successful Programs. Though
use of performance information has been limited, most in the Congress
are aware of the PART. This topic was discussed extensively in a
Government Accountability Office (GAO) report issued last year.
GAO recommends that OMB select PART reassessments and crosscutting
reviews based on factors including the relative priorities, costs, and
risks associated with clusters of related programs, and reflective of
congressional input. Additionally, GAO recommended OMB solicit
congressional views on the performance issues and program areas most in
need of review; the most useful performance data and the presentation of
those data. As mentioned above, OMB is using the PART to improve the
performance of similar programs in areas that are expected to yield
meaningful results. OMB and agencies are also actively soliciting the
views of the Congress in PART assessments, on improvement plans, and
oversight efforts.
IV. NEXT STEPS
The BPI Initiative has identified several activities to improve its
effectiveness over the coming year:
Ensure Plans are Aggressive and Result in Improved Performance.--
Rigorous follow-up on recommendations from the PART will accelerate
improvements in the performance of Federal programs. This will ensure
that the hard work done through the PART produces performance and
management improvements. Additionally, implementation of these plans
must be tracked and reported.
Expand Cross-Cutting Analyses.--Use the PART to facilitate cross-
cutting analysis where there is a higher return than approaching
programs individually. The goal of these efforts is to increase
efficiency and save
[[Page 16]]
dollars. We want to continue to build on the success of previous cross-
cuts. Congressional guidance will be a factor in choosing topics for the
next group of cross-cutting analyses.
Maximize ExpectMore.gov Impact.--The Federal Government should be
accountable to the public for its performance. This new web-based tool
will provide candid information on how programs are performing and what
they are doing to improve. The BPI Initiative will work to increase the
reach and impact of this valuable information to improve program
performance and accountability for results.
Note.--A table with summary information for all programs that have
been reviewed using the Program Assessment Rating Tool (PART) is
available at http://www.whitehouse.gov/omb/budget/fy2007/sheets/part.pdf
This table provides program ratings, section scores, funding levels, and
other information.