[Analytical Perspectives]
[Performance and Management Assessments]
[2. Performance Improvement Initiative]
[From the U.S. Government Printing Office, www.gpo.gov]




[[Page 7]]



                                     

                                     

  ======================================================================

                 PERFORMANCE AND MANAGEMENT ASSESSMENTS

========================================================================

   

[[Page 9]]

 
                  2. PERFORMANCE IMPROVEMENT INITIATIVE

                             I. INTRODUCTION

  The American people expect the Federal government to implement 
programs that will ensure the Nation's security, provide critical 
national level services and produce meaningful results. To hold 
government accountable for its performance, taxpayers must have clear 
and candid information about the successes and failures of all Federal 
programs. For the third straight year, the Administration is providing 
this type of information to the public on ExpectMore.gov, a user-
friendly government website that allows public access to government 
programs. ExpectMore.gov describes which government programs are 
performing, which ones are not, and in both situations, what is being 
done to improve them.
  The objective of the President's Performance Improvement Initiative 
(PII) (formerly the Budget and Performance Integration Initiative) is to 
ensure that Federal dollars produce the greatest results possible. The 
Initiative provides information on program performance to help the 
President and Congress make better, more informed decisions about the 
programs.
  The PII focuses on performance in two principal ways:
    Improved Program Performance: The initiative requires each 
          agency to identify opportunities to improve program management 
          and design, and then develop and implement clear, aggressive 
          plans to get more from tax dollars every year. Agencies have 
          ready access to program performance information from a variety 
          of sources such as the Program Assessment Rating Tool (PART) 
          and other independent program evaluations, investigations, 
          audits, and analyses.
    Greater Investment in Successful Programs: Although 
          performance is not the only factor used to decide the size of 
          a program's budget, Congress and the President can utilize 
          information about a program's effectiveness and efficiency in 
          decision-making so that taxpayer dollars are invested in 
          programs that provide the greatest return to the Nation. If 
          poorly performing programs are unable to demonstrate improved 
          results, then their resources may be reallocated to programs 
          that can demonstrate greater success and returns to the 
          taxpayer.
  Currently, the PII is showing great progress toward helping programs 
become more efficient and more effective through implementation of 
meaningful improvement plans.
  Many programs are demonstrating improved results. For example:
    Social Security Administration (SSA): SSA increased agency 
          productivity by 15.5 percent since 2001 through increased use 
          of information technology and improved business processes. SSA 
          would have required $980 million more in 2007 to process the 
          same work if productivity improvements had not been realized.
    High Intensity Drug Trafficking Areas (HIDTA): The HIDTA 
          program improved the way it measures success by implementing a 
          system for tracking and analyzing performance data. Using this 
          information, more drug trafficking organizations were 
          dismantled for less money. In 2005, 2,183 Drug Trafficking 
          Organizations were dismantled for $80,000 each. By 2006, 2,332 
          were dismantled for $76,000 each.
    Administration on Aging (AoA): AoA improved its outreach and 
          services to elderly Americans who suffer from disease and 
          disability. In 2006, there were 18 States that improved 
          targeting to those living below the poverty level, serving an 
          additional 80,000 elderly individuals who lived in poverty. 
          Over 345,000 elderly and disabled individuals, who due to 
          their physical conditions would otherwise be living in nursing 
          homes, can continue to live in their own homes and stay 
          connected to their communities. This is an increase of more 
          than 52,200 nursing home-eligible individuals since 2003.
    Federal Bureau of Prisons (BOP): In 2006 as a part of its 
          ``Greening Prisons'' initiative, the BOP piloted renewable 
          energy technologies in several prisons and generated savings 
          of $1.1 million. As a result, in 2006 and 2007, BOP entered 
          into 18 new national Energy Savings Performance Contracts with 
          energy services companies to generate additional savings.
  Agencies are identifying additional actions to improve the performance 
of each of their programs. For example:
  Progress toward the second PII goal of improving resource allocation 
has been limited, but this year, the Administration had more success in 
terminating some low-performing programs and targeting those resources 
to well-performing programs. In 2008 seven programs were terminated, 
saving $156 million and six programs were reduced, saving $1.120 
billion. Though no decision is based purely on performance, overall, 
high performing programs received larger funding increases than those 
that did not perform as well.

[[Page 10]]

          II. HOW THE PERFORMANCE IMPROVEMENT INITIATIVE WORKS

  Several aspects of the Performance Improvement Initiative are designed 
to maximize program performance. They include:
    Comprehensively assessing performance using the PART;
    Publishing quarterly Scorecards to hold agencies accountable 
          for managing for results, addressing PART findings, and 
          implementing improvement plans;
    Broadcasting results to the public on ExpectMore.gov; and
    Facilitating program improvement through interagency 
          collaboration and cooperation.

 Comprehensive Assessment with the Program Assessment Rating Tool (PART)

  How do we ensure that Federal programs are improving every year? 
First, we assess their current performance. In order to improve a 
program's outcomes, it is critical to have a good understanding of how 
the program is currently performing. To date, we have assessed the 
performance of more than 1,000 programs, comprising 96 percent of all 
Federal programs, using the PART.

                           History of the PART

  The Federal Government spends trillions of dollars on programs 
annually, but until the advent of the PART, there was not a uniform 
basis for assessing how well these programs actually work. For example, 
are the billions of taxpayer dollars the Federal Government spends on 
foster care actually preventing the maltreatment and abuse of children? 
Are Federal efforts to reduce air pollution successful? Previous 
administrations from President Johnson to President Clinton and Congress 
have grappled with this problem. Each prior administration has tried to 
come up with means by which government programs can be measured for 
results. The most significant advance in bringing accountability to 
government programs was the Government Performance and Results Act of 
1993 (GPRA). This law requires Federal agencies to identify both annual 
and long-term goals and to collect and report performance data. For the 
first time, agencies were required to explicitly identify measures and 
goals for judging the performance of each of their programs and to 
collect information on an annual basis in order to determine whether 
they were meeting those goals.
  This Administration built upon GPRA requirements by creating the PART 
(Program Assessment Rating Tool), an objective, evidence-based and easy-
to-understand questionnaire about program design, planning, management, 
and performance. Objectivity is paramount to a PART rating. For example, 
when the development of the PART began in 2002, the first draft included 
a question relating to whether a particular program served an 
appropriate federal role. Because many people believed that the answer 
to that question would vary depending on the reviewer's philosophical 
outlook, the question was removed.
  Public and private sector entities have reviewed the PART. Private 
sector reviewers have praised the PART assessment process for its 
transparency and objectivity and also have raised concerns that OMB has 
striven to address. For instance, some reviewers found that assessments 
of different programs lacked consistency in the answers to the same 
questions. OMB now audits all draft assessments to correct any obvious 
inconsistencies. Reviewers also found that agencies did not always agree 
with the final assessment of their programs. Agencies can now appeal to 
a high level subcommittee of the President's Management Council to 
dispute answers with which they disagree. To address concerns that OMB 
and agencies were not doing enough to involve Congress in the assessment 
process, agencies are now required to brief and consult their 
Congressional appropriators, authorizers, and overseers before the 
annual assessments begin.
  The accompanying timeline provides a history of the development of the 
PART.

[[Page 11]]





[[Page 12]]

------------------------------------------------------------------------

                                      What is the PART and How is it Used?



The PART helps assess the management and performance of individual programs. With the PART, agencies and OMB
 evaluate a program's purpose, design, planning, management, results, and accountability to determine its
 overall effectiveness. Agencies then identify and complete follow-up actions to improve program results.

To reflect the fact that Federal programs deliver goods and services using different mechanisms, the PART is
 customized by program type. The seven PART types are: Direct Federal, Competitive Grant, Block/Formula Grant,
 Research and Development, Capital Assets and Service Acquisition, Credit, and Regulatory. The PART types apply
 to both discretionary and mandatory programs. ExpectMore.gov also classifies each program by its specific
 program area (such as environment, transportation, education, etc.) to facilitate comparison and accelerate the
 improved performance of programs with similar missions.

Each PART includes 25 basic questions and additional questions tailored to the different program types. The
 questions are divided into four sections. The first section of questions gauges whether a program has a clear
 purpose and is well designed to achieve its objectives. The second section evaluates strategic planning, and
 weighs whether the agency establishes outcome-oriented annual and long-term goals for its programs. The third
 section rates the management of an agency's program, including the quality of efforts to improve efficiency.
 The fourth section assesses the results programs can report with accuracy and consistency.

The answers to questions in each of the four sections result in a numerical score for each section from 0 to 100
 (100 being the best score). Because reporting a single weighted numerical rating could suggest false precision,
 or draw attention away from the very areas most in need of improvement, numerical scores are combined and
 translated into qualitative ratings. The bands and associated ratings are as follows:




------------------------------------------------------------------------
                            Rating                               Range
------------------------------------------------------------------------
Effective....................................................     85-100

Moderately Effective.........................................      70-84

Adequate.....................................................      50-69

Ineffective..................................................       0-49
------------------------------------------------------------------------






Regardless of overall score, programs that do not have acceptable performance measures or have not yet collected
 performance data generally receive a rating of ``Results Not Demonstrated.'' This rating suggests that not
 enough information and data are available to make an informed determination about whether a program is
 achieving results.

PART ratings do not result in automatic decisions about funding. Clearly, over time, funding should be targeted
 to programs that can prove they achieve measurable results. In some cases, a PART rating of ``Ineffective'' or
 ``Results Not Demonstrated'' may suggest that greater funding is necessary to overcome identified shortcomings,
 while a funding decrease may be proposed for a program rated ``Effective'' if it is not a priority or has
 completed its mission. However, most of the time, an ``Effective'' rating is an indication that the program is
 using its funding well and that major changes are not needed.


------------------------------------------------------------------------

                                     

            Publish a Scorecard to Hold Agencies Accountable

  Agencies are achieving greater results with the help of the habits and 
disciplines established through the Performance Improvement Initiative 
(PII). These agencies recognize that the PART can be a useful tool to 
drive improvement in the performance of their programs.
  Agency success is judged by clear, Government-wide goals or standards 
consistent with the Program Improvement Initiative. Agencies have 
developed and are implementing detailed, aggressive improvement plans to 
achieve these goals. Most importantly, agencies are held publicly 
accountable for adopting these disciplines. To meet the Standards for 
Success for the PII, an agency must:
    Demonstrate that senior agency managers meet at least 
          quarterly to examine reports that integrate financial and 
          performance information that covers all major responsibilities 
          of the Department;
    Have strategic plans that contain a limited number of 
          outcome-oriented goals and objectives. Annual budget and 
          performance documents incorporate measures identified in the 
          PART and focus

[[Page 13]]

          on the information used in the senior management report 
          described in the first criterion;
    Report the full cost of achieving performance goals 
          accurately in budget and performance documents and accurately 
          estimate the marginal cost of changing performance goals;
    Have at least one efficiency measure for all PARTed 
          programs;
    Use PART evaluations to direct program improvements and hold 
          managers accountable for those improvements, and PART findings 
          and performance information are used consistently to justify 
          funding requests, management actions, and legislative 
          proposals; and
    Have less than 10 percent of agency programs receive a 
          Results Not Demonstrated rating for two years in a row.
  Each quarter, agencies receive two ratings--status and progress. 
First, they are rated on their status in achieving the overall goals for 
each initiative. They are given a green, yellow or red rating to clearly 
announce their performance. Green status is for success in achieving 
each of the criteria listed above; yellow is for an intermediate level 
of performance; and red is for unsatisfactory performance.
  Second, agency progress on the Program Improvement Initiative 
standards is assessed separately. Agency progress is reviewed on a case-
by-case basis against the work plan and related time lines established 
for each agency. Progress is also given a color rating. Green is given 
when implementation is proceeding according to plans agreed upon with 
the agencies; yellow for when some slippage or other issues require 
adjustment by the agency in order to achieve the initiative objectives 
on a timely basis; and red when the Initiative is in serious jeopardy of 
not realizing its objectives without significant management 
intervention.
  As of September 30, 2007, fourteen agencies achieved green status on 
the Program Improvement Initiative Scorecard. The agencies at green are:

1.          Department of Agriculture
2.          Department of Commerce
3.          Department of Education
4.          Department of Energy
5.          Environmental Protection Agency
6.          Department of Justice
7.          Department of Labor
8.          Department of Transportation
9.          General Services Administration
10.         National Aeronautics and Space Administration
11.         National Science Foundation
12.         Small Business Administration
13.         Smithsonian Institution
14.         Social Security Administration
  The Scorecard is an effective accountability tool to ensure agencies 
manage the performance of their programs. Although a scorecard rating is 
not directly linked to any specific consequences, it is quickly 
understood at the highest levels of the Administration as an indicator 
of an agency's strength or weakness.
  The Government-wide scorecard reporting on individual agency progress 
is published quarterly at www.results.gov/agenda/scorecard.html.

                   Broadcast Results on ExpectMore.gov

  ExpectMore.gov provides Americans with candid information about which 
programs work, which do not, and what all programs are doing to get 
better every year.
  Up until the launch of ExpectMore.gov last year, Americans had limited 
access to information on how well the Federal Government performed. Now, 
Americans can see for themselves how their government programs are 
performing. In many cases, the Federal Government performs well. In some 
cases, it performs better than the private sector.
  ExpectMore.gov contains summaries of PART results for all programs 
that have been assessed to date. The site provides program information 
that a concerned citizen could use to assess a program's performance. 
Each assessment includes a brief description of the program's purpose, 
its overall rating, some highlights about its performance and the steps 
it will take to improve in the future. For individuals interested in 
more information, the site also provides links to the detailed program 
assessment, as well as that program's website and the assessment 
summaries of other similar programs. The detailed PART assessment 
includes the answer to each PART question with an explanation and 
supporting evidence. It also includes the performance measures for the 
program along with current performance information. In addition, there 
is an update on the status of follow-up actions to improve program 
performance.
  A visitor to the site may find, at least initially, that programs are 
not performing as well as they should or program improvement plans are 
not sufficiently ambitious. We expect this site to help change that. The 
website has a variety of benefits, including:
    Increased public attention to performance;
    Greater scrutiny of agency action (or inaction) to improve 
          program results:
     --Improvement plans are transparent
     --Statements about goals and achievements are clearer; and
    Demand for better quality and more timely performance data.

               Implement Inter-Agency Program Improvement

  The Administration continues to look for new ways to improve the 
performance of programs with similar purposes or designs by using the 
PART to analyze performance across agencies (i.e., cross-cutting 
analysis) and State and local levels. Cross-cutting analysis can improve 
coordination and communication by encouraging managers from multiple 
agencies to agree to a common set of goals and by placing the focus on 
quantifiable results. Cross-cutting analysis breaks down barriers across 
the Federal, State, and local levels so that all entities work toward 
the same goal. Only topics that are expected to yield meaningful results 
are se

[[Page 14]]

lected for cross-cutting analyses. To date, the Administration completed 
cross-cutting analyses of the government's math and science programs, 
community and economic development programs, import and food safety 
programs, and others.

                              III. RESULTS

  As mentioned above, the PII measures its progress according to two key 
principles:
    Improved Program Performance; and
    Greater Investment in Successful Programs
  There has been greater success in leading agencies to think more 
systematically about how they measure and improve program performance. 
Though there are many factors that impact program performance, it is 
clear that the PII has framed the discussion around results. Agencies 
have developed ways to measure their efficiency so they can figure out 
how to achieve more with Americans' tax dollars.
  2009 marks the sixth year that the PART was used to (1) assess program 
performance, (2) take steps to improve program performance, and (3) help 
link performance to budget decisions. To date, the Administration has 
assessed more than 1,000 programs, representing approximately 96 percent 
of the Federal budget. The Administration will use the PART to assess 
the performance and management of the remaining Federal programs.
  With the help of the PART, we have improved program performance and 
transparency. There has been a substantial increase in the total number 
of programs rated either ``Effective'', ``Moderately Effective'', or 
``Adequate''. This increase came from both re-assessments and newly 
PARTed programs. The chart below shows the percentage of programs by 
ratings category.


  These results demonstrate that the PII has been very successful in 
focusing Agencies' attention on program performance. For example, 
approximately:
    89 percent of programs established or clarified their long-
          term and annual performance goals to focus on the outcomes 
          that are important to the American people.
    82 percent of programs are achieving their performance 
          goals.
    73 percent of programs are measuring their efficiency, a 
          relatively new activity for Government programs.
    70 percent of programs are improving efficiency annually, 
          producing more value per dollar spent.
    55 percent of programs that were initially unable to 
          demonstrate results have improved their overall performance 
          rating.
  Unfortunately, there has not been a similar level of accomplishment in 
the second measure: Greater Investment in Successful Programs. Though 
Congressional use of performance information has been limited, most in 
the Congress are aware of the PART. This topic was discussed extensively 
in recent debates in the Senate.


[[Page 15]]


  Senator Wayne Allard introduced an amendment to cut funding for 
programs funded in the Labor, HHS, and Education 2008 Appropriations 
Bill rated as ``Ineffective'' by 10 percent across the board. In 
advocating his amendment, Senator Allard said:

    These assessments represent the combined wisdom of career officials. 
      This is not a political process. These are objective evaluations 
       done by career officials at agencies and OMB, and are based on 
     evidence of that program's performance. While a program's overall 
       rating should not be the sole determinant of funding, Congress 
     should prioritize funding programs that perform well. Ineffective 
     programs in particular should be scrutinized to determine whether 
     the resources they use could be better spent elsewhere and whether 
             their goals could be achieved through other means.

  Senator Allard brought warranted focus on programs that aren't 
performing as they should. In arguing against the amendment, Senator Tom 
Harkin said: 

    The Program Assessment Rating Tool . . . is intended to help assess 
    the management and performance of individual programs. So it is not 
      just a question of whether the program works, it also evaluates 
      whether Congress has designed the program in a clear manner and 
        whether Federal agencies do a good job managing the program. 

  Both Senators went on to have a substantive debate about how programs 
were performing and how to get them to perform better. And soon 
thereafter, in arguing for his own amendment, Senator John Cornyn said: 

      The Office of Management and Budget has recently reviewed over a 
     thousand programs. As this chart indicates, upon a review of 1,016 
    Federal Government programs, they have concluded that 22 percent of 
      those programs rated either as ineffective or they are unable to 
       determine whether they are effective. In other words, they are 
       unable to find evidence that they are effective. They have not 
    conclusively determined them as ineffective, but they have concluded 
       that 22 percent of the Federal Government programs are either 
      ineffective or the results are not demonstrated. Anybody who is 
     interested anywhere in the world--certainly in the United States--
      can look at the information on this ExpectMore.gov Web site and 
     inform themselves, as I am sure they would want to, about what the 
         Federal Government is doing and not doing on their behalf.
  This debate on Senator Allard's amendment was an important one. It 
shows increasing attention to the objective rating of program 
performance.

                             IV. NEXT STEPS

  The PII has identified several activities to improve program 
effectiveness over the coming year:
  Ensure Program Goals are Adequate and Improvement Plans are Aggressive 
and Result in Improved Performance.--Review of all completed PARTs and 
program goals, as well as rigorous follow-up on recommendations from the 
PART will accelerate improvements in the performance of Federal 
programs. This will ensure that the hard work done through the PART 
produces performance and management improvements. Additionally, 
implementation of improvement must be tracked and reported.
  Appoint Agency Performance Improvement Officers.--To ensure successful 
implementation of the new policy of the Federal Government embodied in 
Executive Order 13450 to spend taxpayer dollars effectively, and more 
effectively each year, each agency will appoint Performance Improvement 
Officers. Performance Improvement Officers are responsible for 
coordinating the performance improvement activities of their agencies, 
including:
    Developing and improving the agency's strategic plans, 
          annual performance plans, and annual performance reports, as 
          well as ensuring the use of such information in agency budget 
          justifications;
    Ensuring program goals are aggressive, realistic, and 
          accurately measured;
    Regularly convening agency program management personnel to 
          assess and improve program performance and efficiency; and
    Assisting the head of the agency in the development and use 
          within the agency of performance measures in personnel 
          performance appraisals, particularly those of program 
          managers, to ensure real accountability for greater 
          effectiveness.
  Expand Cross-Cutting Analyses.--Use the PART to facilitate cross-
cutting analysis where there is a higher return than approaching 
programs individually. The goal of these efforts is to increase 
efficiency and save dollars, building on the success of previous cross-
cutting analyses. Congressional guidance will be a factor in choosing 
topics for the next group of cross-cutting analyses.
  Maximize ExpectMore.gov Impact.--The Federal Government should be 
accountable to the public for its performance. This web-based tool 
provides candid information on how programs are performing and what they 
are doing to improve. The PII Initiative will work to increase the reach 
and impact of this valuable information to improve program performance 
and accountability for results.


[[Page 16]]


  Note.--A table with summary information for all programs that have 
been reviewed using the Program Assessment Rating Tool (PART) is 
available at: www.whitehouse.gov/omb/expectmore/part.pdf. This table 
provides program ratings, section scores, funding levels, and other 
information. Additionally, a complete data file and data model of all 
assessments on ExpectMore.gov is available at: www.whitehouse.gov/omb/
expectmore/whatsnew.htm. This is a comma-separated values file that 
academics and researchers can use to analyze performance data.