[Analytical Perspectives]
[Performance and Management Assessments]
[2. Performance Improvement Initiative]
[From the U.S. Government Printing Office, www.gpo.gov]
[[Page 7]]
======================================================================
PERFORMANCE AND MANAGEMENT ASSESSMENTS
========================================================================
[[Page 9]]
2. PERFORMANCE IMPROVEMENT INITIATIVE
I. INTRODUCTION
The American people expect the Federal government to implement
programs that will ensure the Nation's security, provide critical
national level services and produce meaningful results. To hold
government accountable for its performance, taxpayers must have clear
and candid information about the successes and failures of all Federal
programs. For the third straight year, the Administration is providing
this type of information to the public on ExpectMore.gov, a user-
friendly government website that allows public access to government
programs. ExpectMore.gov describes which government programs are
performing, which ones are not, and in both situations, what is being
done to improve them.
The objective of the President's Performance Improvement Initiative
(PII) (formerly the Budget and Performance Integration Initiative) is to
ensure that Federal dollars produce the greatest results possible. The
Initiative provides information on program performance to help the
President and Congress make better, more informed decisions about the
programs.
The PII focuses on performance in two principal ways:
Improved Program Performance: The initiative requires each
agency to identify opportunities to improve program management
and design, and then develop and implement clear, aggressive
plans to get more from tax dollars every year. Agencies have
ready access to program performance information from a variety
of sources such as the Program Assessment Rating Tool (PART)
and other independent program evaluations, investigations,
audits, and analyses.
Greater Investment in Successful Programs: Although
performance is not the only factor used to decide the size of
a program's budget, Congress and the President can utilize
information about a program's effectiveness and efficiency in
decision-making so that taxpayer dollars are invested in
programs that provide the greatest return to the Nation. If
poorly performing programs are unable to demonstrate improved
results, then their resources may be reallocated to programs
that can demonstrate greater success and returns to the
taxpayer.
Currently, the PII is showing great progress toward helping programs
become more efficient and more effective through implementation of
meaningful improvement plans.
Many programs are demonstrating improved results. For example:
Social Security Administration (SSA): SSA increased agency
productivity by 15.5 percent since 2001 through increased use
of information technology and improved business processes. SSA
would have required $980 million more in 2007 to process the
same work if productivity improvements had not been realized.
High Intensity Drug Trafficking Areas (HIDTA): The HIDTA
program improved the way it measures success by implementing a
system for tracking and analyzing performance data. Using this
information, more drug trafficking organizations were
dismantled for less money. In 2005, 2,183 Drug Trafficking
Organizations were dismantled for $80,000 each. By 2006, 2,332
were dismantled for $76,000 each.
Administration on Aging (AoA): AoA improved its outreach and
services to elderly Americans who suffer from disease and
disability. In 2006, there were 18 States that improved
targeting to those living below the poverty level, serving an
additional 80,000 elderly individuals who lived in poverty.
Over 345,000 elderly and disabled individuals, who due to
their physical conditions would otherwise be living in nursing
homes, can continue to live in their own homes and stay
connected to their communities. This is an increase of more
than 52,200 nursing home-eligible individuals since 2003.
Federal Bureau of Prisons (BOP): In 2006 as a part of its
``Greening Prisons'' initiative, the BOP piloted renewable
energy technologies in several prisons and generated savings
of $1.1 million. As a result, in 2006 and 2007, BOP entered
into 18 new national Energy Savings Performance Contracts with
energy services companies to generate additional savings.
Agencies are identifying additional actions to improve the performance
of each of their programs. For example:
Progress toward the second PII goal of improving resource allocation
has been limited, but this year, the Administration had more success in
terminating some low-performing programs and targeting those resources
to well-performing programs. In 2008 seven programs were terminated,
saving $156 million and six programs were reduced, saving $1.120
billion. Though no decision is based purely on performance, overall,
high performing programs received larger funding increases than those
that did not perform as well.
[[Page 10]]
II. HOW THE PERFORMANCE IMPROVEMENT INITIATIVE WORKS
Several aspects of the Performance Improvement Initiative are designed
to maximize program performance. They include:
Comprehensively assessing performance using the PART;
Publishing quarterly Scorecards to hold agencies accountable
for managing for results, addressing PART findings, and
implementing improvement plans;
Broadcasting results to the public on ExpectMore.gov; and
Facilitating program improvement through interagency
collaboration and cooperation.
Comprehensive Assessment with the Program Assessment Rating Tool (PART)
How do we ensure that Federal programs are improving every year?
First, we assess their current performance. In order to improve a
program's outcomes, it is critical to have a good understanding of how
the program is currently performing. To date, we have assessed the
performance of more than 1,000 programs, comprising 96 percent of all
Federal programs, using the PART.
History of the PART
The Federal Government spends trillions of dollars on programs
annually, but until the advent of the PART, there was not a uniform
basis for assessing how well these programs actually work. For example,
are the billions of taxpayer dollars the Federal Government spends on
foster care actually preventing the maltreatment and abuse of children?
Are Federal efforts to reduce air pollution successful? Previous
administrations from President Johnson to President Clinton and Congress
have grappled with this problem. Each prior administration has tried to
come up with means by which government programs can be measured for
results. The most significant advance in bringing accountability to
government programs was the Government Performance and Results Act of
1993 (GPRA). This law requires Federal agencies to identify both annual
and long-term goals and to collect and report performance data. For the
first time, agencies were required to explicitly identify measures and
goals for judging the performance of each of their programs and to
collect information on an annual basis in order to determine whether
they were meeting those goals.
This Administration built upon GPRA requirements by creating the PART
(Program Assessment Rating Tool), an objective, evidence-based and easy-
to-understand questionnaire about program design, planning, management,
and performance. Objectivity is paramount to a PART rating. For example,
when the development of the PART began in 2002, the first draft included
a question relating to whether a particular program served an
appropriate federal role. Because many people believed that the answer
to that question would vary depending on the reviewer's philosophical
outlook, the question was removed.
Public and private sector entities have reviewed the PART. Private
sector reviewers have praised the PART assessment process for its
transparency and objectivity and also have raised concerns that OMB has
striven to address. For instance, some reviewers found that assessments
of different programs lacked consistency in the answers to the same
questions. OMB now audits all draft assessments to correct any obvious
inconsistencies. Reviewers also found that agencies did not always agree
with the final assessment of their programs. Agencies can now appeal to
a high level subcommittee of the President's Management Council to
dispute answers with which they disagree. To address concerns that OMB
and agencies were not doing enough to involve Congress in the assessment
process, agencies are now required to brief and consult their
Congressional appropriators, authorizers, and overseers before the
annual assessments begin.
The accompanying timeline provides a history of the development of the
PART.
[[Page 11]]
[[Page 12]]
------------------------------------------------------------------------
What is the PART and How is it Used?
The PART helps assess the management and performance of individual programs. With the PART, agencies and OMB
evaluate a program's purpose, design, planning, management, results, and accountability to determine its
overall effectiveness. Agencies then identify and complete follow-up actions to improve program results.
To reflect the fact that Federal programs deliver goods and services using different mechanisms, the PART is
customized by program type. The seven PART types are: Direct Federal, Competitive Grant, Block/Formula Grant,
Research and Development, Capital Assets and Service Acquisition, Credit, and Regulatory. The PART types apply
to both discretionary and mandatory programs. ExpectMore.gov also classifies each program by its specific
program area (such as environment, transportation, education, etc.) to facilitate comparison and accelerate the
improved performance of programs with similar missions.
Each PART includes 25 basic questions and additional questions tailored to the different program types. The
questions are divided into four sections. The first section of questions gauges whether a program has a clear
purpose and is well designed to achieve its objectives. The second section evaluates strategic planning, and
weighs whether the agency establishes outcome-oriented annual and long-term goals for its programs. The third
section rates the management of an agency's program, including the quality of efforts to improve efficiency.
The fourth section assesses the results programs can report with accuracy and consistency.
The answers to questions in each of the four sections result in a numerical score for each section from 0 to 100
(100 being the best score). Because reporting a single weighted numerical rating could suggest false precision,
or draw attention away from the very areas most in need of improvement, numerical scores are combined and
translated into qualitative ratings. The bands and associated ratings are as follows:
------------------------------------------------------------------------
Rating Range
------------------------------------------------------------------------
Effective.................................................... 85-100
Moderately Effective......................................... 70-84
Adequate..................................................... 50-69
Ineffective.................................................. 0-49
------------------------------------------------------------------------
Regardless of overall score, programs that do not have acceptable performance measures or have not yet collected
performance data generally receive a rating of ``Results Not Demonstrated.'' This rating suggests that not
enough information and data are available to make an informed determination about whether a program is
achieving results.
PART ratings do not result in automatic decisions about funding. Clearly, over time, funding should be targeted
to programs that can prove they achieve measurable results. In some cases, a PART rating of ``Ineffective'' or
``Results Not Demonstrated'' may suggest that greater funding is necessary to overcome identified shortcomings,
while a funding decrease may be proposed for a program rated ``Effective'' if it is not a priority or has
completed its mission. However, most of the time, an ``Effective'' rating is an indication that the program is
using its funding well and that major changes are not needed.
------------------------------------------------------------------------
Publish a Scorecard to Hold Agencies Accountable
Agencies are achieving greater results with the help of the habits and
disciplines established through the Performance Improvement Initiative
(PII). These agencies recognize that the PART can be a useful tool to
drive improvement in the performance of their programs.
Agency success is judged by clear, Government-wide goals or standards
consistent with the Program Improvement Initiative. Agencies have
developed and are implementing detailed, aggressive improvement plans to
achieve these goals. Most importantly, agencies are held publicly
accountable for adopting these disciplines. To meet the Standards for
Success for the PII, an agency must:
Demonstrate that senior agency managers meet at least
quarterly to examine reports that integrate financial and
performance information that covers all major responsibilities
of the Department;
Have strategic plans that contain a limited number of
outcome-oriented goals and objectives. Annual budget and
performance documents incorporate measures identified in the
PART and focus
[[Page 13]]
on the information used in the senior management report
described in the first criterion;
Report the full cost of achieving performance goals
accurately in budget and performance documents and accurately
estimate the marginal cost of changing performance goals;
Have at least one efficiency measure for all PARTed
programs;
Use PART evaluations to direct program improvements and hold
managers accountable for those improvements, and PART findings
and performance information are used consistently to justify
funding requests, management actions, and legislative
proposals; and
Have less than 10 percent of agency programs receive a
Results Not Demonstrated rating for two years in a row.
Each quarter, agencies receive two ratings--status and progress.
First, they are rated on their status in achieving the overall goals for
each initiative. They are given a green, yellow or red rating to clearly
announce their performance. Green status is for success in achieving
each of the criteria listed above; yellow is for an intermediate level
of performance; and red is for unsatisfactory performance.
Second, agency progress on the Program Improvement Initiative
standards is assessed separately. Agency progress is reviewed on a case-
by-case basis against the work plan and related time lines established
for each agency. Progress is also given a color rating. Green is given
when implementation is proceeding according to plans agreed upon with
the agencies; yellow for when some slippage or other issues require
adjustment by the agency in order to achieve the initiative objectives
on a timely basis; and red when the Initiative is in serious jeopardy of
not realizing its objectives without significant management
intervention.
As of September 30, 2007, fourteen agencies achieved green status on
the Program Improvement Initiative Scorecard. The agencies at green are:
1. Department of Agriculture
2. Department of Commerce
3. Department of Education
4. Department of Energy
5. Environmental Protection Agency
6. Department of Justice
7. Department of Labor
8. Department of Transportation
9. General Services Administration
10. National Aeronautics and Space Administration
11. National Science Foundation
12. Small Business Administration
13. Smithsonian Institution
14. Social Security Administration
The Scorecard is an effective accountability tool to ensure agencies
manage the performance of their programs. Although a scorecard rating is
not directly linked to any specific consequences, it is quickly
understood at the highest levels of the Administration as an indicator
of an agency's strength or weakness.
The Government-wide scorecard reporting on individual agency progress
is published quarterly at www.results.gov/agenda/scorecard.html.
Broadcast Results on ExpectMore.gov
ExpectMore.gov provides Americans with candid information about which
programs work, which do not, and what all programs are doing to get
better every year.
Up until the launch of ExpectMore.gov last year, Americans had limited
access to information on how well the Federal Government performed. Now,
Americans can see for themselves how their government programs are
performing. In many cases, the Federal Government performs well. In some
cases, it performs better than the private sector.
ExpectMore.gov contains summaries of PART results for all programs
that have been assessed to date. The site provides program information
that a concerned citizen could use to assess a program's performance.
Each assessment includes a brief description of the program's purpose,
its overall rating, some highlights about its performance and the steps
it will take to improve in the future. For individuals interested in
more information, the site also provides links to the detailed program
assessment, as well as that program's website and the assessment
summaries of other similar programs. The detailed PART assessment
includes the answer to each PART question with an explanation and
supporting evidence. It also includes the performance measures for the
program along with current performance information. In addition, there
is an update on the status of follow-up actions to improve program
performance.
A visitor to the site may find, at least initially, that programs are
not performing as well as they should or program improvement plans are
not sufficiently ambitious. We expect this site to help change that. The
website has a variety of benefits, including:
Increased public attention to performance;
Greater scrutiny of agency action (or inaction) to improve
program results:
--Improvement plans are transparent
--Statements about goals and achievements are clearer; and
Demand for better quality and more timely performance data.
Implement Inter-Agency Program Improvement
The Administration continues to look for new ways to improve the
performance of programs with similar purposes or designs by using the
PART to analyze performance across agencies (i.e., cross-cutting
analysis) and State and local levels. Cross-cutting analysis can improve
coordination and communication by encouraging managers from multiple
agencies to agree to a common set of goals and by placing the focus on
quantifiable results. Cross-cutting analysis breaks down barriers across
the Federal, State, and local levels so that all entities work toward
the same goal. Only topics that are expected to yield meaningful results
are se
[[Page 14]]
lected for cross-cutting analyses. To date, the Administration completed
cross-cutting analyses of the government's math and science programs,
community and economic development programs, import and food safety
programs, and others.
III. RESULTS
As mentioned above, the PII measures its progress according to two key
principles:
Improved Program Performance; and
Greater Investment in Successful Programs
There has been greater success in leading agencies to think more
systematically about how they measure and improve program performance.
Though there are many factors that impact program performance, it is
clear that the PII has framed the discussion around results. Agencies
have developed ways to measure their efficiency so they can figure out
how to achieve more with Americans' tax dollars.
2009 marks the sixth year that the PART was used to (1) assess program
performance, (2) take steps to improve program performance, and (3) help
link performance to budget decisions. To date, the Administration has
assessed more than 1,000 programs, representing approximately 96 percent
of the Federal budget. The Administration will use the PART to assess
the performance and management of the remaining Federal programs.
With the help of the PART, we have improved program performance and
transparency. There has been a substantial increase in the total number
of programs rated either ``Effective'', ``Moderately Effective'', or
``Adequate''. This increase came from both re-assessments and newly
PARTed programs. The chart below shows the percentage of programs by
ratings category.
These results demonstrate that the PII has been very successful in
focusing Agencies' attention on program performance. For example,
approximately:
89 percent of programs established or clarified their long-
term and annual performance goals to focus on the outcomes
that are important to the American people.
82 percent of programs are achieving their performance
goals.
73 percent of programs are measuring their efficiency, a
relatively new activity for Government programs.
70 percent of programs are improving efficiency annually,
producing more value per dollar spent.
55 percent of programs that were initially unable to
demonstrate results have improved their overall performance
rating.
Unfortunately, there has not been a similar level of accomplishment in
the second measure: Greater Investment in Successful Programs. Though
Congressional use of performance information has been limited, most in
the Congress are aware of the PART. This topic was discussed extensively
in recent debates in the Senate.
[[Page 15]]
Senator Wayne Allard introduced an amendment to cut funding for
programs funded in the Labor, HHS, and Education 2008 Appropriations
Bill rated as ``Ineffective'' by 10 percent across the board. In
advocating his amendment, Senator Allard said:
These assessments represent the combined wisdom of career officials.
This is not a political process. These are objective evaluations
done by career officials at agencies and OMB, and are based on
evidence of that program's performance. While a program's overall
rating should not be the sole determinant of funding, Congress
should prioritize funding programs that perform well. Ineffective
programs in particular should be scrutinized to determine whether
the resources they use could be better spent elsewhere and whether
their goals could be achieved through other means.
Senator Allard brought warranted focus on programs that aren't
performing as they should. In arguing against the amendment, Senator Tom
Harkin said:
The Program Assessment Rating Tool . . . is intended to help assess
the management and performance of individual programs. So it is not
just a question of whether the program works, it also evaluates
whether Congress has designed the program in a clear manner and
whether Federal agencies do a good job managing the program.
Both Senators went on to have a substantive debate about how programs
were performing and how to get them to perform better. And soon
thereafter, in arguing for his own amendment, Senator John Cornyn said:
The Office of Management and Budget has recently reviewed over a
thousand programs. As this chart indicates, upon a review of 1,016
Federal Government programs, they have concluded that 22 percent of
those programs rated either as ineffective or they are unable to
determine whether they are effective. In other words, they are
unable to find evidence that they are effective. They have not
conclusively determined them as ineffective, but they have concluded
that 22 percent of the Federal Government programs are either
ineffective or the results are not demonstrated. Anybody who is
interested anywhere in the world--certainly in the United States--
can look at the information on this ExpectMore.gov Web site and
inform themselves, as I am sure they would want to, about what the
Federal Government is doing and not doing on their behalf.
This debate on Senator Allard's amendment was an important one. It
shows increasing attention to the objective rating of program
performance.
IV. NEXT STEPS
The PII has identified several activities to improve program
effectiveness over the coming year:
Ensure Program Goals are Adequate and Improvement Plans are Aggressive
and Result in Improved Performance.--Review of all completed PARTs and
program goals, as well as rigorous follow-up on recommendations from the
PART will accelerate improvements in the performance of Federal
programs. This will ensure that the hard work done through the PART
produces performance and management improvements. Additionally,
implementation of improvement must be tracked and reported.
Appoint Agency Performance Improvement Officers.--To ensure successful
implementation of the new policy of the Federal Government embodied in
Executive Order 13450 to spend taxpayer dollars effectively, and more
effectively each year, each agency will appoint Performance Improvement
Officers. Performance Improvement Officers are responsible for
coordinating the performance improvement activities of their agencies,
including:
Developing and improving the agency's strategic plans,
annual performance plans, and annual performance reports, as
well as ensuring the use of such information in agency budget
justifications;
Ensuring program goals are aggressive, realistic, and
accurately measured;
Regularly convening agency program management personnel to
assess and improve program performance and efficiency; and
Assisting the head of the agency in the development and use
within the agency of performance measures in personnel
performance appraisals, particularly those of program
managers, to ensure real accountability for greater
effectiveness.
Expand Cross-Cutting Analyses.--Use the PART to facilitate cross-
cutting analysis where there is a higher return than approaching
programs individually. The goal of these efforts is to increase
efficiency and save dollars, building on the success of previous cross-
cutting analyses. Congressional guidance will be a factor in choosing
topics for the next group of cross-cutting analyses.
Maximize ExpectMore.gov Impact.--The Federal Government should be
accountable to the public for its performance. This web-based tool
provides candid information on how programs are performing and what they
are doing to improve. The PII Initiative will work to increase the reach
and impact of this valuable information to improve program performance
and accountability for results.
[[Page 16]]
Note.--A table with summary information for all programs that have
been reviewed using the Program Assessment Rating Tool (PART) is
available at: www.whitehouse.gov/omb/expectmore/part.pdf. This table
provides program ratings, section scores, funding levels, and other
information. Additionally, a complete data file and data model of all
assessments on ExpectMore.gov is available at: www.whitehouse.gov/omb/
expectmore/whatsnew.htm. This is a comma-separated values file that
academics and researchers can use to analyze performance data.