[Analytical Perspectives]
[Performance and Management Assessments]
[2. Budget and Performance Integration and the Program Assessment RatingTool (PART)]
[From the U.S. Government Printing Office, www.gpo.gov]
[[Page 9]]
2. BUDGET AND PERFORMANCE INTEGRATION
I. INTRODUCTION
Good Government--a government fiscally responsible to the people--must
have as one of its core purposes the achievement of results for the
taxpayers. Taxpayers expect the Federal Government to implement programs
that will ensure the Nation's security and provide critical services.
Taxpayers want their money spent wisely and used to gain maximum
benefit. Taxpayers have the right to hold the Federal Government
accountable for its actions. To exercise this right, the taxpayers must
have clear, candid, and up-to-date information about each program's
successes and failures. For the second straight year, the Administration
is providing this type of information to all Americans on
ExpectMore.gov, a user-friendly government website that describes which
programs are performing, which ones are not, and in both situations,
what is being done to improve them. (Greater detail about ExpectMore.gov
will be provided in a subsequent section.)
The Administration is making the Federal Government increasingly
effective by making program budget decisions based on program
performance. The objective of the President's Budget and Performance
Integration (BPI) Initiative is to ensure that Federal dollars produce
the greatest results. Under the BPI Initiative, agencies and OMB
identify which programs work well, which are deficient, and what can be
done to improve performance of each program. In some cases, the
Administration may find it necessary to reallocate funding from less
effective programs to more effective ones. The final decisions about the
scope of programs and the size of program budgets are ultimately made
jointly by the Congress and the President. The BPI Initiative provides
information on program performance to help the Executive and Legislative
branches make better, more informed decisions. Information about program
performance is now readily available and accessible to the public on
ExpectMore.gov.
The BPI Initiative measures a program's success in two principal ways:
Improved Program Performance: The initiative requires each
agency to identify opportunities to improve program management
and design, and then develop and implement clear, aggressive
plans to get more for tax dollars every year. Agencies have
ready access to program performance information by using the
results of the Program Assessment Rating Tool (PART)
assessments of each program, program evaluations,
investigations, audits, and analyses from a variety of
sources.
Greater Investment in Successful Programs: Overall, there
are now more program-funding needs and thus fewer resources to
be allocated to each funded program. These scarce resources
need to be allocated to programs that benefit the Nation most
effectively and efficiently. Though performance is not the
only factor used to decide the size of a program's budget,
Congress and the President can utilize information about a
program's effectiveness and efficiency in decision-making so
that taxpayer dollars are invested in programs that provide
the greatest return to the Nation. If poor performing programs
are unable to demonstrate improved results, then their
resources may be reallocated to programs that can demonstrate
greater success and returns to the taxpayer.
Currently, the BPI Initiative is showing great progress toward the
first goal. Programs are becoming more efficient and more effective
through implementation of meaningful improvement plans.
Many programs are demonstrating improved results. For example:
The Social Security Administration increased agency
productivity by 13.1 percent since 2001 through increased use
of information technology and improved business processes. SSA
would have required $800 million more in 2006 to process the
same work if productivity improvements had not been realized.
In 2005, the Bureau of Prisons reduced the construction cost
per bed in high security facilities, saving an estimated $54
million.
The Federal Transit Administration implemented its plan to
process Formula Grants faster. In the past, the highest
reported processing time for processing grants was 90 days.
FTA now expects to process such grants within only 36 days.
Agencies are identifying additional actions to improve the performance
of each of their programs. All agencies, regardless of whether their
programs perform poorly or well, strive for increased program
performance each year.
Progress toward the second goal of improving resource allocation has
been slow, but this year, the administration had greater success. We
have been successful in terminating some low-performing programs and
better at targeting resources to well-performing programs. In 2006,
seven programs were terminated, saving $230 million. Four programs were
reduced, saving $300 million. Though no decision is based purely on
performance, overall, high performing programs received larger funding
increases than those that did not perform as well.
[[Page 10]]
II. HOW THE BUDGET AND PERFORMANCE INTEGRATION INITIATIVE WORKS
Several aspects of the Budget Performance Integration (BPI) Initiative
are designed to maximize program performance. They include:
Assessment of performance with the PART (Program Assessment
Rating Tool);
Publishing a Scorecard to hold agencies accountable for
managing for results, addressing PART findings, and
implementing follow-up actions;
Broadcasting results to the public on ExpectMore.gov; and
Facilitating program improvement through interagency
collaboration and cooperation.
Comprehensive Assessment with the Program Assessment Rating Tool (PART)
How do we ensure that Federal programs are improving every year?
First, we assess their current performance. In order to improve a
program's outcomes, it is critical to have a good understanding of how
the program is currently performing. To date, we have assessed the
performance of nearly 1,000 programs, comprising 96 percent of all
Federal programs, using the PART.
History of the PART
The Federal Government spends trillions of dollars on programs
annually, but until the advent of the PART, there was not a uniform
basis for assessing how well these programs actually work. For example,
were the billions of taxpayer dollars the Federal Government spent on
foster care actually preventing the maltreatment and abuse of children?
Are Federal efforts to reduce air pollution successful? Previous
administrations from President Johnson to President Clinton and Congress
have grappled with this problem. Each prior administration has tried to
come up with means by which government programs are measured for
results. The most significant advance in bringing accountability to
government programs was the Government Performance and Results Act of
1993 (GPRA). This law requires Federal agencies to identify both annual
and long-term goals and collect and report performance data. For the
first time, agencies were required to explicitly identify measures and
goals for judging the performance of each of their programs and to
collect information on an annual basis in order to determine if they
were meeting those goals.
This Administration built upon GPRA requirements by creating the PART
(Program Assessment Rating Tools), an objective, evidence-based and
easy-to-understand questionnaire about program design, planning,
management, and performance. Objectivity is paramount to a PART rating.
For example, when the development of the PART began in 2002, the first
draft included a question relating to whether a particular program
served an appropriate federal role. Because many people believed that
the answer to that question would vary depending on the reviewer's
philosophical outlook, the question was removed.
Public and private sector entities have reviewed the PART. Private
sector reviewers have praised the PART assessment process for its
transparency and objectivity and have also raised concerns that OMB has
striven to address. For instance, some reviewers found assessments of
different programs lack consistency in the answers to the same
questions. OMB now audits all draft assessments to correct any obvious
inconsistencies. Reviewers also found that agencies did not always agree
with the final assessment of their programs. Agencies can now appeal to
a high level subcommittee of the President's Management Council to
dispute answers with which they disagree. To address concerns that OMB
and agencies were not doing enough to involve Congress in the assessment
process, agencies are now required to brief and consult their
Congressional appropriators, authorizers, and overseers before the
annual assessments begin.
The accompanying timeline provides a history of the development of the
PART.
[[Page 11]]
[[Page 12]]
------------------------------------------------------------------------
What is the PART and How is it Used?
The PART helps assess the management and performance of individual programs. With the PART, agencies and OMB
evaluate a program's purpose, design, planning, management, results, and accountability to determine its
overall effectiveness. Agencies then identify and complete follow-up actions to improve program results.
To reflect the fact that Federal programs deliver goods and services using different mechanisms, the PART is
customized by program type. The seven PART types are: Direct Federal, Competitive Grant, Block/Formula Grant,
Research and Development, Capital Assets and Service Acquisition, Credit, and Regulatory. The PART types apply
to both discretionary and mandatory programs. ExpectMore.gov also classifies each program by its specific
program area (such as environment, transportation, education, etc.) to facilitate comparison so we can
accelerate the improved performance of programs with similar missions.
Each PART includes 25 basic questions and there are additional questions tailored to the different program
types. The questions are divided into four sections. The first section of questions gauges whether a program
has a clear purpose and is well designed to achieve its objectives. The second section evaluates strategic
planning, and weighs whether the agency establishes outcome-oriented annual and long-term goals for its
programs. The third section rates the management of an agency's program, including the quality of efforts to
improve efficiency. The fourth section assesses the results programs can report with accuracy and consistency.
The answers to questions in each of the four sections result in a numerical score for each section from 0 to 100
(100 being the best score). Because reporting a single weighted numerical rating could suggest false precision,
or draw attention away from the very areas most in need of improvement, numerical scores are combined and
translated into qualitative ratings. The bands and associated ratings are as follows:
------------------------------------------------------------------------
Rating Range
------------------------------------------------------------------------
Effective.................................................... 85-100
Moderately Effective......................................... 70-84
Adequate..................................................... 50-69
Ineffective.................................................. 0-49
------------------------------------------------------------------------
Regardless of overall score, programs that do not have acceptable performance measures or have not yet collected
performance data generally receive a rating of ``Results Not Demonstrated.'' This rating suggests that not
enough information and data are available to make an informed determination about whether a program is
achieving results.
PART ratings do not result in automatic decisions about funding. Clearly, over time, funding should be targeted
to programs that can prove they achieve measurable results. In some cases, a PART rating of ``Ineffective'' or
``Results Not Demonstrated'' may suggest that greater funding is necessary to overcome identified shortcomings,
while a funding decrease may be proposed for a program rated ``Effective'' if it is not a priority or has
completed its mission. However, most of the time, an ``Effective'' rating is an indication that the program is
using its funding well and that major changes are not needed.
------------------------------------------------------------------------
Publish a Scorecard To Hold Agencies Accountable
Agencies are achieving greater results with the help of the habits and
disciplines established through the BPI Initiative. These agencies
recognize that the PART can be a useful tool to drive improvement in the
performance of their programs.
Agency success is judged by clear, Government-wide goals or standards
for Budget and Performance Integration. Agencies have developed and are
implementing detailed, aggressive action plans to achieve these goals.
Most importantly, agencies are held publicly accountable for adopting
these disciplines. To meet the Standards for Success for the BPI
Initiative, an agency must:
Demonstrate that senior agency managers meet at least
quarterly to examine reports that integrate financial and
performance information that covers all major responsibilities
of the Department;
Have strategic plans that contain a limited number of
outcome-oriented goals and objectives. Annual budget and
performance documents incorporate measures identified in the
PART and focus on the information used in the senior
management report described in the first criterion;
Report the full cost of achieving performance goals
accurately in budget and performance documents and accurately
estimate the marginal cost of changing performance goals;
[[Page 13]]
Have at least one efficiency measure for all PARTed
programs;
Use PART evaluations to direct program improvements and hold
managers accountable for those improvements, and PART findings
and performance information are used consistently to justify
funding requests, management actions, and legislative
proposals; and
Have less than 10 percent of agency programs receive a
Results Not Demonstrated rating for two years in a row.
Each quarter, agencies receive two ratings. First, they are rated on
their status in achieving the overall goals for each initiative. They
are then given a green, yellow or red rating to clearly announce their
performance. Green status is for success in achieving each of the
criteria listed earlier; yellow is for an intermediate level of
performance; and red is for unsatisfactory performance.
Second, agency progress toward reaching the Budget and Performance
Integration standards is assessed separately. This is reviewed on a
case-by-case basis against the work plan and related time lines
established for each agency. Progress is also given a color rating.
Green is given when implementation is proceeding according to plans
agreed upon with the agencies; Yellow for when some slippage or other
issues require adjustment by the agency in order to achieve the
initiative objectives on a timely basis; and Red when the Initiative is
in serious jeopardy. In this case, it is unlikely to realize objectives
absent significant management intervention.
As of December 31, 2006, fifteen agencies achieved green status on the
Budget and Performance Integration Initiative Scorecard. The agencies at
green are:
1. Department of Agriculture
2. Department of Commerce
3. Department of Education
4. Department of Energy
5. Department of Justice
6. Department of Labor
7. Department of Transportation
8. Department of State
9. General Services Administration
10. National Aeronautics and Space Administration
11. National Science Foundation
12. Small Business Administration
13. Smithsonian
14. Social Security Administration
15. U.S. Agency for International Development
The Scorecard is an effective accountability tool to ensure agencies
manage the performance of their programs. Although a scorecard rating is
not directly linked to any specific consequences, it is quickly
understood at the highest levels of the Administration as an indicator
of an agency's strength or weakness.
The Government-wide scorecard reporting on individual agency progress
is published quarterly at www.results.gov/agenda/scorecard.html.
Broadcast Results on ExpectMore.gov
ExpectMore.gov provides Americans with candid information about which
programs work, which do not, and what all programs are doing to get
better every year.
Up until the launch of ExpectMore.gov last year, Americans had limited
access to information on how well the Federal Government performed. Now,
every American can see for themselves how their government is
performing. In many cases, the Federal Government performs well. In some
cases, it performs better than the private sector.
ExpectMore.gov contains PART summaries for all programs that have
been assessed to date. The site provides the program information that a
concerned citizen would need to assess a program's performance. Each
assessment includes a brief description of the program's purpose, its
overall rating, some highlights about its performance and the steps it
will take to improve in the future. For individuals interested in more
information, the site also provides links to the detailed program
assessment, as well as that program's website and the assessment
summaries of other similar programs. The detailed PART assessment
includes the answer to each PART question with an explanation and
supporting evidence. It also includes the performance measures for the
program along with current performance information. In addition, there
is an update on the status of follow-up actions to improve program
performance.
A visitor to the site may find, at least initially, programs are not
performing as well as they should or program improvement plans are not
sufficiently ambitious. We expect this site to help change that. The
website has a variety of benefits, including:
Increased public attention to performance;
Greater scrutiny of agency action (or inaction) to improve
program results:
--Improvement plans will be transparent
--Statements about goals and achievements will be clearer; and
Demand for better quality and more timely performance data.
[[Page 14]]
Implement Inter-Agency Program Improvement
The Administration continues to look for new ways to improve the
performance of programs with similar purposes or designs by using the
PART to analyze performance across agencies (i.e., cross-cutting
analysis) and State and local levels. Cross-cutting analysis can improve
coordination and communication by getting managers from multiple
agencies to agree to a common set of goals and placing the focus on
quantifiable results. This type of analysis breaks down barriers across
the Federal, State, and local levels so that all entities work toward
the same goal. Only topics that are expected to yield meaningful results
are selected for cross-cutting analyses. This past year the
Administration completed cross-cutting analysis of the government's math
and science programs as part of the ACC (Academic Competitiveness
Council).
Academic Competitiveness Council. The ACC set out to identify all
Federal education programs with a science, technology, engineering, and
math focus; clarify the goals of these programs; identify the extent to
which the programs have undergone independent, external evaluation based
on sound, scientific principles and have quantitative evidence of
achieving their goals; and identify better ways to measure and evaluate
these programs and efficiently integrate and coordinate Federal spending
on Science, Technology, Engineering, and Mathematics (STEM) education
programs.
The ACC first identified 109 STEM education programs funded in 2006
for a total of $3.13 billion. Within that total, elementary and
secondary programs received approximately $640 million (20 percent of
the total), postsecondary programs, including graduate and postdoctoral
programs, nearly $2.4 billion (76 percent) and informal education and
outreach programs close to $103 million (4 percent). The group agreed on
common goals for the programs, but found that few had been rigorously
evaluated and determined to be effective. These programs, like many
managed by the Federal Government, must do more to gather and report
evidence of what activities are most effective at achieving common
goals.
III. RESULTS
As mentioned above, the BPI Initiative measures its success according
to two measures:
Improved Program Performance; and
Greater Investment in Successful Programs
There has been greater success in achieving the goals of the first
measure. The BPI Initiative has caused agencies to think more
systematically about how they measure and improve program performance.
Though there are many factors that impact program performance, it is
clear that the BPI Initiative has framed the discussion around results.
Agencies have developed ways to measure their efficiency so they can
figure out how to achieve more with Americans' tax dollars.
This marks the fifth year that the PART was used to (1) assess program
performance, (2) take steps to improve program performance, and (3) help
link performance to budget decisions. To date, the Administration has
assessed nearly 1000 programs, representing approximately 96 percent of
the Federal budget. Over the next year, the Administration will use the
PART to assess the performance and management of most of the remaining
Federal programs.
[[Page 15]]
With the help of the PART, we have improved program performance and
transparency. There has been a substantial increase in the total number
of programs rated either ``Effective'', ``Moderately Effective'', or
``Adequate''. This increase came from both re-assessments and newly
PARTed programs. The chart below shows the percentage of programs by
ratings category.
These results demonstrate that the BPI Initiative has been very
successful in focusing Agencies' attention on program performance. For
example, approximately:
14 percent of programs improved their performance rating
overall;
80 percent of programs have acceptable performance measures;
74 percent have achieved their long-term goals and 80
percent have achieved their annual goals; and
90 percent of programs have efficiency measures and about
half of them have achieved their efficiency targets.
Unfortunately, there has not been a similar level of accomplishment in
the second measure: Greater Investment in Successful Programs. Though
congressional use of performance information has been limited, most in
the Congress are aware of the PART. This topic was discussed extensively
in a Government Accountability Office (GAO) report issued last year.
GAO recommends that OMB select PART reassessments and crosscutting
reviews based on factors that include the relative priorities, costs,
and risks associated with clusters of related programs, and reflect
congressional input. Additionally, GAO recommended OMB solicit
congressional views on the performance issues and program areas most in
need of review; the most useful performance data and the presentation of
those data. As mentioned above, OMB is using the PART to improve the
performance of similar programs in areas that are expected to yield
meaningful results. OMB and agencies are also actively soliciting the
views of the Congress in PART assessments, on improvement plans, and
oversight efforts.
IV. NEXT STEPS
The BPI Initiative has identified several activities to improve its
effectiveness over the coming year:
Ensure Plans are Aggressive and Result in Improved Performance.--
Rigorous follow-up on recommendations from the PART will accelerate
improvements in the performance of Federal programs. This will ensure
that the hard work done through the PART produces performance and
management improvements. Additionally, implementation of these plans
must be tracked and reported.
Expand Cross-Cutting Analyses.--Use the PART to facilitate cross-
cutting analysis where there is a higher return than approaching
programs individually. The goal of these efforts is to increase
efficiency and save dollars, building on the success of previous cross-
cutting analyses. Congressional guidance will be a factor in
[[Page 16]]
choosing topics for the next group of cross-cutting analyses.
Maximize ExpectMore.gov Impact.--The Federal Government should be
accountable to the public for its performance. This web-based tool
provides candid information on how programs are performing and what they
are doing to improve. The BPI Initiative will work to increase the reach
and impact of this valuable information to improve program performance
and accountability for results.
Note.--A table with summary information for all programs that have
been reviewed using the Program Assessment Rating Tool (PART) is
available at: www.whitehouse.gov/omb/budget/fy2008/sheets/part.pdf. This
table provides program ratings, section scores, funding levels, and
other information. Additionally, a complete data file and data model of
all assessments on ExpectMore.gov is available at: www.whitehouse.gov/
omb/expectmore/whatsnew.htm. This is a comma-separated values file that
academics and researchers can use to analyze performance data.