[Senate Hearing 109-923]
[From the U.S. Government Publishing Office]



                                                        S. Hrg. 109-923

                   AUTOPILOT BUDGETING: WILL CONGRESS
                       EVER RESPOND TO GOVERNMENT
                           PERFORMANCE DATA?

=======================================================================

                                HEARING

                               before the

                FEDERAL FINANCIAL MANAGEMENT, GOVERNMENT
                     INFORMATION, AND INTERNATIONAL
                         SECURITY SUBCOMMITTEE

                                 of the

                              COMMITTEE ON
                         HOMELAND SECURITY AND
                          GOVERNMENTAL AFFAIRS
                          UNITED STATES SENATE


                       ONE HUNDRED NINTH CONGRESS

                             SECOND SESSION

                               __________

                             JUNE 13, 2006

                               __________

        Available via http://www.access.gpo.gov/congress/senate

       Printed for the use of the Committee on Homeland Security
                        and Governmental Affairs
















                      U.S. GOVERNMENT PRINTING OFFICE
29-503 PDF                    WASHINGTON  :  2007
---------------------------------------------------------------------
For sale by the Superintendent of Documents, U.S. Government
Printing Office Internet:  bookstore.gpo.gov Phone:  toll free (866)
512-1800; DC area (202) 512-1800 Fax: (202)512-2250 Mail: Stop SSOP,
Washington, DC 20402-0001 










        COMMITTEE ON HOMELAND SECURITY AND GOVERNMENTAL AFFAIRS

                   SUSAN M. COLLINS, Maine, Chairman
TED STEVENS, Alaska                  JOSEPH I. LIEBERMAN, Connecticut
GEORGE V. VOINOVICH, Ohio            CARL LEVIN, Michigan
NORM COLEMAN, Minnesota              DANIEL K. AKAKA, Hawaii
TOM COBURN, Oklahoma                 THOMAS R. CARPER, Delaware
LINCOLN D. CHAFEE, Rhode Island      MARK DAYTON, Minnesota
ROBERT F. BENNETT, Utah              FRANK LAUTENBERG, New Jersey
PETE V. DOMENICI, New Mexico         MARK PRYOR, Arkansas
JOHN W. WARNER, Virginia

           Michael D. Bopp, Staff Director and Chief Counsel
             Michael L. Alexander, Minority Staff Director
                  Trina Driessnack Tyrer, Chief Clerk


FEDERAL FINANCIAL MANAGEMENT, GOVERNMENT INFORMATION, AND INTERNATIONAL 
                         SECURITY SUBCOMMITTEE

                     TOM COBURN, Oklahoma, Chairman
TED STEVENS, Alaska                  THOMAS CARPER, Delaware
GEORGE V. VOINOVICH, Ohio            CARL LEVIN, Michigan
LINCOLN D. CHAFEE, Rhode Island      DANIEL K. AKAKA, Hawaii
ROBERT F. BENNETT, Utah              MARK DAYTON, Minnesota
PETE V. DOMENICI, New Mexico         FRANK LAUTENBERG, New Jersey
JOHN W. WARNER, Virginia             MARK PRYOR, Arkansas

                      Katy French, Staff Director
                 Sheila Murphy, Minority Staff Director
            John Kilvington, Minority Deputy Staff Director
                       Liz Scranton, Chief Clerk
























                            C O N T E N T S

                                 ------                                
Opening statements:
                                                                   Page
    Senator Coburn...............................................     1
    Senator Carper...............................................     6

                               WITNESSES
                         Tuesday, June 13, 2006

Hon. Clay Johnson III, Deputy Director for Management, U.S. 
  Office of Management and Budget................................     8
Eileen Norcross, Government Accountability Project, Mercartus 
  Center at George Mason University..............................     9
Adam Hughes, Director for Federal Fiscal Policy, OMB Watch.......    11

                     Alphabetical List of Witnesses

Hughes, Adam:
    Testimony....................................................    11
    Prepared statement with attachments..........................    89
Johnson, Hon. Clay III:
    Testimony....................................................     8
    Prepared statement with attachments..........................    25
Norcross, Eileen:
    Testimony....................................................     9
    Prepared statement...........................................    49

                                APPENDIX

``A Working Paper in Government Accountabiity'' by Eileen 
  Norcross and Kyle McKenzie.....................................    61






















 
                   AUTOPILOT BUDGETING: WILL CONGRESS
                      EVER RESPOND TO GOVERNMENT
                           PERFORMANCE DATA?

                              ----------                              


                         TUESDAY, JUNE 13, 2006

                                     U.S. Senate,  
          Subcommittee on Federal Financial Management,    
       Government Information, and International Security  
                      of the Committee on Homeland Security and    
                                            Governmental Affairs,  
                                                    Washington, DC.
    The Subcommittee met, pursuant to notice, at 2:57 p.m., in 
room 342, Dirksen Senate Office Building, Hon. Tom Coburn 
(Chairman of the Subcommittee) presiding.
    Present: Senators Coburn and Carper.

              OPENING STATEMENT OF SENATOR COBURN

    Senator Coburn. Good afternoon. The Federal Financial 
Management Subcommittee of the Homeland Security and 
Governmental Affairs Committee will come to order. Senator 
Carper will be here in a moment. We apologize for the delay. 
There was an official photo. We also have a conflict. There is 
a briefing ongoing now by the Secretary of Defense and the 
Secretary of State, which will limit Senator Carper's time with 
us. So we are going to go on and go forward so we have it in 
the record. I apologize for the conflicting schedules.
    Americans have a crazy idea, that they should get something 
for their money, even when the money is spent by the 
government. It is a simple concept, and in policy-speak we call 
it performance-based budgeting. I know I am new in the Senate, 
but I am still surprised by the level of resistance in 
Washington to holding people accountable by measuring their 
performance. And it is a difficult thing to do. A 
multitrillion-dollar government imposing some sort of 
standardized outcome evaluation is difficult at best, and what 
it implies is that the tool will be very crude. But that does 
not say we should not attempt to make measurements, and I want 
to be one of many who should commend both Mr. Johnson and the 
Bush Administration, and the President himself, for being the 
first to attempt to do it.
    It is not novel. It is required in the competitive business 
environment that we find ourselves worldwide. It is being used 
effectively in many State governments, and it is something that 
is long overdue. The Performance Assessment Rating Tool (PART) 
was first introduced by the President 4 years ago as a tool to 
review the strengths and weaknesses of government programs to 
influence funding and programmatic decisions. The annual PART 
reports offer needed sunshine in government and provide good 
data for government managers to improve their programs.
    Today, the Office of Management and Budget has reviewed 793 
programs, which account for $1.47 trillion in taxpayer money. 
Almost a third of these programs have proven to not meet up to 
standards based on the PART analysis. I have already admitted 
that it is a blunt tool. One-third of $1.5 trillion is $500 
billion. Maybe this is why PART scores so far have created a 
stir not only among the agencies but among the Members of 
Congress who make budgeting decision.
    Some Members of Congress want to stick their heads in the 
sand and keep funding pet programs on autopilot year after 
year. To my amazement, just last week, the Appropriations 
Subcommittee that funds the Departments of Labor and Health and 
Human Services passed language prohibiting the use of PART 
assessments on those agencies. They may not like PART's 
message, but they should not shoot the messenger. This sort of 
Orwellian immunization against any hint that our favorite 
programs may not be performing up to the idealized, utopian 
goals of their Congressional champions is one of the reasons 
why Americans are mad at Congress.
    The approval ratings for Congress are in the tank, and this 
prohibition of accountability for failing government is why the 
voters who fork over their hard-earned dollars every year may 
just have something new to say come this November. I am not 
sure why so many of my colleagues are afraid of assessment 
tools on performance. It may reflect their own performance.
    As part of our investigation for this hearing, we learned 
that low PART ratings do not always mean that OMB will 
recommend a budget cut or a cut in the program or a 
recommendation to go on the terminations list. In some cases, 
programs rated ineffective have had budget reductions 
recommended. But in other cases, the reason they were low was 
because they were not funded appropriately to begin with, and 
therefore, they could not accomplish what they were intended to 
because they did not have adequate funding.
    Each program is unique, and I do not know that a PART score 
should be the last word. But I do know that measurement of 
performance is something that every member of a Congressional 
authorizing or Sppropriations committee should be reading and 
using to inform their oversight work. Congress consistently 
neglects the duty to conduct oversight of Federal programs and 
spending. Instead, we spend most of the time passing spending 
bills that ignore PART ratings, the President's termination 
list, or any other performance data as if the spending were on 
autopilot. Congress might as well write a blank check.
    By 2008, OMB will have applied PART to the entire 
government. In the last 4 years, OMB has scored 793 government 
programs. Here are the results: 15 percent were found to be 
effective; 29 percent were found to be moderately effective; 28 
percent were rated adequate; 4 percent were found to be 
ineffective--that is one in every 25 programs--25 percent could 
not demonstrate results to get a rating and were labeled 
results not demonstrated.
    I do not believe the spin that results not demonstrated can 
mean that the program is either good or bad; we just do not 
have enough information to tell. On the contrary, the results 
not demonstrated designation is a red flag marking a program so 
poorly conceived by us or so directionalist that that 
unaccountability seems to have been built into it by design. 
Programs rated ineffective or results not demonstrated account 
for $152 billion in budget authority. Imagine what we could do 
with $152 billion right now. The ideas are endless.
    Outside of Washington, DC, any business or family with 
finite resources sets priorities and creates a budget based on 
the actual amount of bang they get from their buck. It is only 
inside the Beltway where that kind of information is not 
considered relevant, and in fact, some are even attempting to 
ban the collection of such information. But then, it is only 
Washington where you never have to declare bankruptcy, and debt 
is allowed to grow on the backs of future generations with 
impunity.
    Let me give you one case study, and my co-chairman on this 
will disagree, but my firm believing is the following: We held 
a hearing last year on the Advanced Technology Program that was 
created in 1988 to subsidize high-risk research and 
development. This program has never demonstrated results. What 
it has demonstrated is corporate welfare. Its 2002 PART report, 
that the majority of ATP grants go to multibillion dollar 
corporations and that the GAO has found that ATP projects are 
very similar to private sector R&D undertaken without a 
government subsidy. An amendment to eliminate this funding that 
was offered last year lost by a vote of 68 to 29. In the end, 
Congress wasted a portion of $79 million last year for that 
program. The 2007 Senate budget resolution promises to fund the 
program at almost twice that amount.
    It would be one thing if we were operating in a surplus. 
Then, we could have a legitimate debate about whether to keep 
failing programs, hoping that they would improve, or to give 
that surplus back to the taxpayers. But that is not where we 
are today. With a debt burden of $25,000 per man, woman, and 
child, we simply cannot afford to keep funding programs that 
cannot prove their worth. Non-defense discretionary spending 
has increased 45 percent since 2001. The President has 
requested a $2.8 trillion budget, and that does not include any 
of the so-called emergency, ``supplemental bills in our 
future,'' nor does it include the late night pork barrel frenzy 
each time Congress schedules an appropriations bill vote.
    Entitlement spending will tank our economy if we do not do 
something to get spending under control. The question remains: 
How do we get Congress to act? I would like to see OMB sell 
their PART terminations list more aggressively, forcibly sell 
the reforms and savings to Congress, fight for the cuts by 
taking the terminations list to the American people with the 
power of the bully pulpit. The President should veto spending 
bills that continue to issue blank checks for failing programs.
    There is a bit of hope on the horizon. I was encouraged to 
see that the House Appropriations Committee wrote in their 2006 
budget savings report that the only way to establish 
accountability in the budget process is to stop spending on 
programs that have outlived their usefulness or could be 
delivered more effectively at the State or local level. I will 
believe that when I see it, but I welcome any help that we can 
get.
    The best place to start is by immediately defunding all 
programs on the termination list and adopting other PART 
recommendation reductions. Granted, the list only cuts $20 
billion from a $2.8 trillion budget, but we have got to start 
somewhere. What is more, we should suspend the creation of any 
new program until further notice or it is compared to the 
existing programs that it is meant to supplement. We need 
sunset legislation that would phase out government agencies on 
a timed basis, where we force ourselves to look at them and to 
reauthorize them.
    These are challenging times, and we can no longer afford to 
run on a budget that is on cruise control. I want to thank our 
witnesses for being here.
    [The prepared statement of Chairman Coburn follows:]
             OPENING PREPARED STATEMENT OF CHAIRMAN COBURN
    Americans have a crazy idea: They should get something for their 
money, even when the money is spent by government. It's a simple 
concept--in policy-speak, we call it ``performance-based budgeting.'' I 
know I'm new in the Senate, but I'm still surprised by how much 
resistance there is in Washington to performance-based budgeting.
    Now, to be fair, taking a multi-trillion dollar government and 
imposing some sort of standardized outcome evaluation on it is 
difficult at best. So I concede that any instrument we use will be a 
blunt instrument. But I want to commend President Bush for being the 
first to try.
    The Performance Assessment Rating Tool (PART) was first introduced 
by the President 4 years ago as a tool to review the strengths and 
weaknesses of government programs to influence funding and programmatic 
decisions.
    The annual PART reports offer needed sunshine in government and 
provide good data for government managers to improve their programs. To 
date, the Office of Management and Budget has reviewed 793 programs 
which account for $1.47 trillion in taxpayer money. Almost a third of 
these programs have proven either totally ineffective or are not 
demonstrating results. One-third of $1.5 trillion is $500 billion.
    Maybe this is why the PART scores have created a stir--not only 
among the agencies, but among the Members of Congress who make 
budgeting decisions. Some Members of Congress want to stick their head 
in the sand and keep funding their pet programs, as if on autopilot, 
year after year.
    Just last week the House Appropriations subcommittee that funds the 
Departments of Labor, Education and Health and Human Services passed 
language prohibiting the use of PART assessments on those agencies. 
They may not like PART's message, but they shouldn't shoot the 
messenger. This sort of Orwellian immunization against any hint that 
our favorite programs may not be performing up to the idealized utopian 
goals of their Congressional champions is why Americans are mad at 
Congress. The approval ratings for Congress are in the tank, and this 
prohibition of accountability for failing government is why the voters 
who fork over their hard-earned dollars every year may just have 
something to say come November.
    I'm not sure why some of my colleagues are so afraid of PART. As 
part of our investigation for this hearing, we learned that low PART 
ratings don't always mean that OMB will recommend a budget cut or put 
the program on the Terminations List. In some cases, programs rated 
``ineffective'' had budget reductions, but in other cases their budgets 
increased. Each program is unique and I don't know that a PART score 
should be the last word, but I do know that the PART is something every 
member of a Congressional authorizing or Appropriations committee 
should be reading and using to inform their oversight work.
    You see, Congress consistently neglects the duty to conduct 
oversight of Federal programs and spending. Instead, we spend most of 
the time passing spending bills that ignore PART ratings, the 
President's terminations list and any other performance data. It is as 
if we're spending on ``auto pilot''--Congress might as well just write 
a blank check.
    By 2008, OMB will have applied PART to the entire government. In 
the last 4 years OMB has scored 793 government programs. Here are the 
results: Just 15 percent were found to be ``effective''; 29 percent 
were rated ``moderately effective''; 28 percent were rated 
``adequate''; 4 percent were found to be ``ineffective''; and 24 
percent cannot demonstrate results to even get a rating and were 
labeled ``results not demonstrated''! Don't believe the spin that 
``results not demonstrated'' could mean that the program is either good 
or bad, we just don't have enough information to tell. On the 
contrary--the ``results not demonstrated'' designation is a red flag 
marking a program so poorly conceived or directionless that 
unaccountability seems to have been built into it by design.
    Programs rated ``ineffective'' or ``results not demonstrated'' 
account for $152 billion in budget authority. Imagine what we could do 
with $152 billion.
    Outside of Washington DC, any business or family with finite 
resources sets priorities and creates a budget based on the actual 
amount of bang they get for their hard-earned buck. It is only inside 
the beltway where that kind of information isn't considered relevant 
and in fact, some are trying hard to ban the collection of such 
information. But then, it's only in Washington where you never have to 
declare bankruptcy and debt is allowed to grow on the backs of future 
generations with impunity.
    Let me give you one case study. We held a hearing last year on the 
Advanced Technology Program. The program was created by Congress in 
1988 to subsidize high-risk research and development. The program 
cannot demonstrate results. It is corporate welfare. The 2002 PART 
reported that the majority of ATP grants go to multimillion dollar 
corporations and that the GAO has found that ATP projects are very 
similar to private sector R&D undertaken without a government subsidy. 
An amendment to eliminate funding for ATP that I offered last year was 
voted down in the Senate 68-29. In the end, Congress wasted another $79 
million last year for the program. The 2007 Senate budget resolution 
promises to fund the program at almost twice that amount.
    It would be one thing if we were operating in a surplus. Then we 
could have a legitimate debate about whether to keep funding failing 
programs hoping they will improve or to give that surplus back to the 
taxpayers. But that's not where we are today, with a debt burden of 
$25,000 per man, woman and child in America. We simply cannot afford to 
keep funding programs that cannot prove their worth.
    Nondefense discretionary spending has increased over 45 percent 
since 2001. The President has requested a $2.8 trillion budget and that 
doesn't include any so called ``emergency'' supplemental spending bills 
in our future, nor does it include the late-night pork-barrel frenzy 
each time Congress schedules an Appropriations bill vote. Entitlement 
spending will tank our economy if we don't do something to get spending 
under control.
    The question remains, how do we get Congress to act? I would like 
to see OMB sell their PART and Terminations List more aggressively:

      Forcefully sell these reforms and savings to Congress.
      Fight for these cuts, by taking the terminations list to 
the American people with the power of the bully pulpit.
      The President should veto spending bills that continue to 
issue blank checks to failing programs.

    There's a bit of hope on the horizon--I was encouraged to see that 
the House Appropriations Committee wrote in their 2006 Budget Savings 
report that ``the only way to establish accountability in the budget 
process is to stop spending on programs that have outlived their 
usefulness or could be delivered more effectively at the State or local 
level.'' I'll believe it when I see it, but I welcome any help we can 
get.
    The best place to start is by immediately defunding all programs on 
the Terminations List and adopting the other PART reduction 
recommendations. Granted, the list only cuts $20.4 billion from a $2.8 
trillion budget, but we've got to start somewhere. What's more, we 
should suspend the creation of any new program until further notice. We 
need ``sunset'' legislation that would phase out every single 
government agency, department or program after a certain deadline if 
the Congress fails to act or if the program consistently performs 
poorly. These are challenging times and we can no longer budget on 
cruise control.
    I want to thank our witnesses for being here today and for the time 
they spent preparing testimony.

    Again, I apologize for the lateness of our attendance, and 
Senator Carper, you are recognized.

              OPENING STATEMENT OF SENATOR CARPER

    Senator Carper. Thank you.
    Senator Coburn. And I have already explained that you will 
probably have to attend the briefing that is ongoing.
    Senator Carper. Thanks, Mr. Chairman. To our witnesses, 
welcome. It is good to see each of you. We appreciate you 
joining us and providing your testimony today. As the Chairman 
mentioned, Secretary of State Rice and Secretary Rumsfeld are 
briefing us as we speak over in the Capitol, and I want to slip 
out in a little bit and hear what they have to say and 
hopefully rejoin you before you leave.
    Mr. Chairman, thank you for holding this hearing. It is an 
important hearing, as we both know. And as we have discussed in 
any number of our similar hearings in the past over the last 
couple of years, our country is facing a large budget deficit 
for as far as the eye can see, and we are just about to embark 
on another appropriations season here in Congress, where we 
will be called on to make some difficult decisions about what 
to do with relatively scarce Federal resources.
    At the same time as GAO and other observers have pointed 
out again, and again, we are at a crossroads in our history, 
where we need to decide what we want our government to do in 
the 21st Century. Nearly 5 years after the attacks of September 
11, 2001, we have a whole new set of needs, a whole new set of 
priorities that must be balanced against some of our older 
needs and priorities in scores of popular programs. And with 
the challenge of retiring baby boomers, guys like me, our 
generation on the horizon, we just cannot afford to do all of 
the things that we might want to do.
    That is why initiatives like OMB's Program Assessment and 
Rating Tool (PART) are interesting and, I think, important. We 
should never be afraid of taking a hard look at Federal 
programs, my programs, Senator Coburn's programs, whatever, to 
determine whether or not they are accomplishing what was 
intended for them to accomplish when we first created them. And 
in this day and age, we simply cannot afford to allow either 
poorly conceived or poorly managed programs to continue without 
reform or, frankly, for a program that has run its course and 
achieved its goals, to continue draining resources from other, 
newer priorities.
    That said, we need to be certain that PART or whatever 
mechanism we use to make these evaluations is in itself 
effective. I think to be effective, a program like PART must be 
totally separated from politics and ideology, at least to the 
extent we can make that happen. It must be closely coordinated 
with existing mechanisms agencies and Congress use to align the 
budget with program goals and outcomes such as the older 
government Performance and Results Act. And perhaps just as 
importantly, we also need to make sure that a program's 
intended beneficiaries outside of Washington have a say before 
an evaluation is actually completed.
    Let me just add in closing, if I could, Mr. Chairman, that 
we are not going to close the budget deficit, we know, by 
reducing spending on a program here or eliminating a program 
there, although every little bit helps. But even if a program 
were to eliminate every single one of the programs receiving 
failing grades through PART, I still think the savings would 
cover just a fraction of our budget deficit, but they would 
cover a portion of our budget deficit.
    Non-defense discretionary spending, which is the target of 
many of the spending reductions and program eliminations in the 
President's budget proposals, make up a relatively small 
percentage of the Federal budget. I am sure we can find ways to 
improve the management of some of the funding in that 16 
percent or even to find and eliminate waste and inefficient use 
of resources within that 16 percent.
    If we truly want to tackle the fiscal problems facing us 
right now, however, we, and that is the Congress and I think 
the Administration needs to take a look at the entire budgetary 
picture. We need to look on both the spending and on the 
revenue side, and we need to make some tough choices.
    Thank you, Mr. Chairman. Again, we look forward to your 
testimony today.
    Senator Coburn. Thank you, Senator Carper.
    I am going to ask the witnesses to limit their verbal 
testimony to 5 minutes. Your complete written statements will 
be made a part of the official hearing record, and we will hold 
our questions until you have given your testimony.
    Let me first introduce Clay Johnson III, Deputy Director 
for Management at OMB, and in his capacity, he has provided the 
government-wide leadership to the Executive Branch agencies to 
improve agency and program performance. Formerly, he served as 
Assistant to the President for Presidential Personnel, 
responsible for the organization that identifies and recruits 
approximately 4,000 senior officials, middle management 
personnel, and part-time board and commission members. At OMB, 
he oversees PART process.
    Eileen Norcross, Senior Research Fellow, Government 
Accountability Project, The Mercatus Center at George Mason 
University; she joined that center as a research fellow in 
January 2003. Her research areas include the U.S. budget, the 
use of performance budgeting in the Federal Government, tax and 
fiscal policy, and environmental regulation. She is one of the 
leading experts on performance-based budgeting, and her 
scholarship plays a vital role in the debate on PART and the 
importance of measuring outcomes.
    Adam Hughes is the Director for Federal Fiscal Policy at 
OMB Watch. He oversees Federal budget and tax policy, income 
and wealth trends, and government performance issues at OMB 
Watch. Senator Carper and myself very much appreciate the work 
that OMB Watch has done in their pursuit of transparent and 
accountable government and for the support of the Federal 
Funding Accountability and Transparency Act that we both 
authored. This bill would create an online public database that 
itemizes Federal funding so taxpayers can see how their money 
is being spent.
    I want to welcome you all. I will recognize Mr. Johnson 
first.

     TESTIMONY OF CLAY JOHNSON III,\1\ DEPUTY DIRECTOR FOR 
        MANAGEMENT, U.S. OFFICE OF MANAGEMENT AND BUDGET

    Mr. Johnson. Mr. Chairman, Senator Carper, thank you very 
much.
---------------------------------------------------------------------------
    \1\ The prepared statement of Mr. Johnson with attachments appears 
in the Appendix on page 25.
---------------------------------------------------------------------------
    The title of this hearing is Will Congress Ever Respond to 
Program Performance Data? In preparing my response, I rephrased 
that to Does Congress Care Whether Programs Work or Not? My 
answer is ``I am not sure, but I sure hope so.'' I believe that 
taxpayers want Congress to ensure that they, the taxpayers, get 
what they pay for. I believe that we all, to widely varying 
degrees, however, want Federal programs to do what they are 
supposed to do and get better every year.
    I believe that money is tight, as you all have pointed out, 
and the biggest opportunity we have to add new services and 
expand existing services to more citizens is through causing 
our existing programs to work better, not spend more money. I 
believe that career employees want to be held accountable for 
how their programs perform. They tell me this in focus groups. 
And I also believe that career employees care about how their 
programs perform.
    Because of this, I believe it is important to have certain 
things. I believe it is important to have clear outcome goals 
for each Federal program. We do not have that now. I believe it 
is important to have Federal program performance information 
that is objective, as objective and reliable as possible. I 
believe that we need to have lots of transparency about how 
well programs are performing. If we do all of this in the dead 
of night, it cannot be used to hold people accountable.
    I believe that we need lots of debate about these 
performance assessments and how to make them better. As you 
said, Mr. Chairman, program assessment is going to be a blunt 
instrument, particularly in the early years. And it will only 
get better every year, but a blunt instrument is better than no 
instrument at all. I also believe it is important to have lots 
of discussion about how to help programs work better. We talk a 
lot about using the PART to make budget decisions. I believe 
the primary use of PART information is to help programs get 
better. If we cut programs, we might save $10 billion here or 
$15 billion there per year. If we cause 1 percent improvement 
in program performance each year, that is $28 billion a year. 
Two percent is obviously twice that.
    After 5 years of effort, not 5 months, comprehensive 
program performance information is still time consuming and 
very hard to come by. We have program outcome goals, 
performance information, and lots of transparency, which other 
countries and several States are working to adopt, and most 
good government groups applaud. What we do not have from most 
Members of Congress is a lot of constructive debate about these 
assessments and how to improve and use them to improve program 
performance. We have asked for feedback. We have asked for 
engagement by Congress but have not gotten it.
    Currently, a majority of Appropriations subcommittees have 
no objection to the way agencies use performance information to 
justify their budgets. Some of these subcommittees actually use 
the PART to justify program funding in their bills. A few 
Members of Congress have advanced greater use of performance 
information in decisionmaking. Congressmen Platts and Tanner 
have proposed separate pieces of legislation, while Senators 
like you, Senator Coburn, and Senators Carper, Ensign, and 
Allard have spoken out on the subject, and Congressmen Cuellar, 
Conaway, and Diaz-Balart have spoken out on it as well.
    But these expressions of interest in program performance 
are the exceptions. There is a big, a huge opportunity for 
Congress to challenge programs to clearly define success and 
their plan for achieving it, and then to hold agencies 
accountable for doing what they said they were going to do.
    That concludes my remarks, and I look forward to any 
questions.
    Senator Coburn. Ms. Norcross.

TESTIMONY OF EILEEN NORCROSS,\1\ SENIOR RESEARCH FELLOW FOR THE 
   GOVERNMENT ACCOUNTABILITY PROJECT, THE MERCATUS CENTER AT 
                    GEORGE MASON UNIVERSITY

    Ms. Norcross. Thank you, Chairman Coburn, Senator Carper, 
for inviting me to testify today on Autopilot Budgeting: Will 
Congress Ever Respond to Government Performance Data? Our work 
in the Government Accountability Project at the Mercatus Center 
at George Mason University focuses closely on performance 
information in government, and I note that the views expressed 
in my testimony are not an official position of the university.
---------------------------------------------------------------------------
    \1\ The prepared statement of Ms. Norcross appears in the Appendix 
on page 49.
---------------------------------------------------------------------------
    I would like to submit for the record our paper on the 
results of the fiscal year 2007 PART for your reference.
    Senator Coburn. Without objection, the document will be 
included in the record.\2\
---------------------------------------------------------------------------
    \2\ The Working Paper in Government Accountability appears in the 
Appendix on page 61.
---------------------------------------------------------------------------
    A program is a tool to achieve a policy goal. Do economic 
development programs lead to prosperous communities? Are 
homeland security programs protecting the Nation? Congress 
needs to know the answers to these questions in order to make 
decisions about how to spend resources. Without performance 
information, Congress cannot reliably accomplish its policy 
aims. Not knowing its consequences, Congress has created 
anywhere from 180 to 342 programs dealing with economic 
development in over 24 agencies; 44 job training programs in 
nine agencies.
    Program duplication on this scale tells us that Congress is 
not sure which programs are reaching their goals. It has no way 
of comparing programs around common outcomes. Not knowing if a 
job training program is employing people means not spending 
money on programs that are employing people. Not evaluating 
programs on a regular basis prevents the program from 
effectively reaching grantees or delivering results; 
performance information from its dialogue between agencies, the 
Executive Branch, and Congress around jointly defined 
objectives.
    Congress took the initiative in 1993, when it passed GPRA. 
GPRA has encouraged the development of performance measures and 
data, but it was not until OMB's Program Assessment Rating Tool 
that real progress towards developing measures was made. That 
is because the Administration does not just require the 
information; it uses it. Congress has identified the need for 
performance information. It must now commit to using it. 
Otherwise, measuring and gathering data is a paper exercise.
    For the past 2 years, the President has issued a major 
savings and reforms report detailing his reasons for 
terminating or reducing funding for programs. Of the 154 
recommended for termination or reduction in funds last year, 54 
were PARTed. The document indicates where PART played a role. 
Other factors include lack of a Federal role, obsolescence, or 
completion of mission.
    The Administration uses PART along with other information 
and does not limit itself to the evaluations. It does not 
automatically reward satisfactory programs or cancel 
underperforming ones. By contrast, the House Committee on 
Appropriations report ``On Time and Under Budget'' lists 53 
programs that were terminated. It only offers explanations for 
three of the terminations. We do not know if the remainder were 
terminated because they were underperformers or politically 
easy choices.
    The Administration's report gives a rationale for each 
recommendation. The House report only provides a list. 
Ultimately, the goal is not to randomly kill programs. Making 
judgments about how to fund agency activity should be 
constructive, not destructive. Performance information helps 
make policy effective. We want to know what works, what does 
not, and why.
    The only way to give budgetary decisions credibility is to 
base them on a reliable evaluation of their performance. Is 
PART that system? PART's methodology has been criticized. 
Improvements can and should be made. But what is important 
about PART is not the ratings; it is the Management 101 
questions PART asks of agency activity. Is the program purpose 
clear? Is it effectively targeted? Has it demonstrated progress 
towards its goals? These questions are the substance of PART. 
These are the questions Congress should be asking before 
allocating resources.
    PART has a few virtues. It has identified and catalogued 
agency activity. It is transparent. It holds programs 
accountable to the same standards. It measures outcomes. Once 
strength often cited as a weakness: PART rates programs on 
statutory limitations. Though a source of frustration for 
agencies, here, PART provides a service by identifying those 
aspects of a program that are barriers to success. The hope is 
that Congress review the statute to see if it is preventing the 
program from meeting its objectives.
    Some limitations of PART: It rates programs against their 
own performance. We would like to see PART advanced to compare 
like activities. In some cases, scores may not fully reflect 
program performance, and there is a potential for different 
budget examiners to reach different conclusions. We do not 
believe Congress should adopt PART wholesale. We hope Congress 
would consider using the kinds of questions PART is asking as 
the basis of developing its own method of evaluating agency 
activity based on common outcomes.
    Indiscriminate cancellation of programs discredits the 
budgetary process. We leave program managers confused about why 
their programs failed. Programs need to deliver according to 
clear expectations and be given a chance to perform. When you 
do not meet the expectations, reduction in funding or 
termination should be the result. It should not be a surprise.
    We believe performance information is best used in 
conjunction with other criteria. All of these form the basis 
against which Congress should continually scrutinize agency 
activity. Efforts to advance what PART has set in motion can 
only aid Congress in its work and give the American people 
confidence that our Nation's problems are being solved. Thank 
you.
    Senator Coburn. Mr. Hughes.

TESTIMONY OF ADAM HUGHES,\1\ DIRECTOR OF FEDERAL FISCAL POLICY, 
                           OMB WATCH

    Mr. Hughes. Chairman Coburn, thank you for having me here 
today and for holding this hearing. As you mentioned, I am the 
Federal Fiscal Policy Director at OMB Watch. OMB Watch was 
founded in the 1980s and has spent over 20 years advocating for 
government accountability, transparency, and access to 
government information, and citizen participation in 
governmental processes.
---------------------------------------------------------------------------
    \1\ The prepared statement of Mr. Hughes with attachments appears 
in the Appendix on page 89.
---------------------------------------------------------------------------
    OMB Watch believes citizens must take an active role in 
holding their government accountable and that the Federal 
Government, when supported by sensible fiscal policy, can 
develop effective programs and safeguards that meet the 
public's needs.
    The issue of government performance, as you mentioned 
earlier, has taken on added importance during the Bush 
Administration, as a combination of factors, some avoidable and 
some not, have plunged the Federal Government into debt. Large 
and sustained deficits over the past 5 years have made 
efficient use of government resources all the more important.
    In light of the anticipated budget crunch due to the baby 
boomers' retirement over the coming decades, the fiscal 
situation in this country will only deteriorate further. 
Performance measurement can therefore become a particularly 
attractive alternative for those who want to set Federal 
priorities based on the current fiscal prospects of a strained 
and shrinking revenue base.
    OMB Watch has been commenting on government performance 
issues for the better part of its existence. We have spent 
increased time and resources analyzing the Government 
Performance and Results Act and the Program Assessment Rating 
Tool over the last 10 years, as government itself has spent 
more time focusing on performance and results. We are strongly 
supportive of improving the Federal Government's capacity to 
meet the public's needs. OMB Watch has worked to protect and 
improve that capacity, and we have been open to the possibility 
of using performance measurement as one means for achieving 
those ends.
    We bring a strong belief in the importance and potential of 
government itself to the work we do, and because of that 
belief, we, perhaps maybe more than anyone else, want 
government to be responsible to community needs, spend money 
wisely, and accomplish its goals. We are advocates for 
government and therefore have a strong motivation to see 
government programs succeed.
    PART, however, is a very poor mechanism for measuring 
program performance and results, introducing biases and skewed 
ideological perspective into a model claiming to present 
consistent and objective performance data and evaluations of 
government programs. Oftentimes, the PART actually decreases 
the efficiency and effectiveness of government through 
increased administrative burdens, distracted managers, and 
compliance costs.
    Ironically, we feel the PART mechanism itself does not 
produce the right type of results to support and improve 
government. We believe PART ratings should not be directly 
connected to the budgeting process of Congress because of 
significant deficiencies within the mechanisms, namely, the 
substantial biases and limitations embedded within the tool and 
the additional distortion and manipulation we have observed in 
OMB's actual application of the PART.
    Based on our studies of the PART and our longstanding 
commitment to open, accountable government that is responsive 
to the public's needs, I would like to make three points today. 
First, we feel the PART continues a troubling trend we have 
seen in other recent Executive Branch proposals and even some 
Congressional proposals, namely, a trend towards increasing the 
power of the White House and the Executive Branch even into 
some areas that have been constitutionally designed to be 
committed to Congress.
    Second, the PART is a limited and distorted tool that 
should not be used for either management of programs or for 
budget and appropriations decisions. In both the design of the 
tool and the process by which the tool is implemented, PART 
systematically ignores the reality and the complexity of 
Federal programs and judges them based on standards that are 
often deeply incompatible with the purposes those very programs 
are expected to serve. As one agency contact memorably 
explained to us, PART assessments are akin to a baseball coach 
walking to the mound to remove his star player and then 
chastising him for not kicking enough field goals.
    My third point is that there is a better way. Specifically, 
Congress already has the means to investigate and produce far 
more sophisticated analyses of the usefulness, effectiveness, 
and results of government programs in a deliberative way, 
including the opportunity for input from a wide array of 
stakeholder interests. The openness of the Legislative Branch 
allows the Congress to be informed and make better decisions, 
but it also serves to balance competing agendas and 
perspectives from both inside and outside Congress.
    The oversight and evaluation process is one of the primary 
if not the primary role for the Legislative Branch. While the 
oversight function of Congress may not be as robust as it once 
was because of significantly shorter legislative sessions and 
delays due to sharply divided political climates, the capacity 
to judge the results of government programs already exists 
within the existing structures, structures that we feel do not 
carry the significant limitations, biases, and negative 
consequences of the PART.
    In conclusion, we all agree that everyone in government, 
the President, agencies and departments, and their staffs, and 
especially Congress, needs to be focused on achieving results 
in a fair, effective, and balanced way. However, this job 
should most of all fall on Congress, which already has the 
necessary tools and resources in place to do the most robust 
and equitable review of the entire Federal Government.
    Relying too heavily on PART ratings will not only gradually 
remove Congress from its funding and oversight responsibilities 
but will also continue to close the door on opportunities for 
outside stakeholder interests, the views of the public, to be 
infused into the Congressional budgeting and evaluation 
process. The limited perspective of the PART is one of its most 
glaring deficiencies. While subjectivity and bias will almost 
always creep into any rating system, the PART does not have a 
mechanism for balancing out the results of its one-size-fits-
all, Executive Branch-focused perspective.
    While the expansion of the Executive Branch powers has been 
present in our government since the turn of the last century, 
the overreach of those powers into areas historically and 
constitutionally given to Congress, the structuring of 
programs, appropriating and authorizing of revenues, and 
oversight of government is a disturbing trend. Because of this, 
PART should not be taken with just a grain of salt or even a 
hefty dose of skepticism. Unless the tool design and 
implementation systems are significantly modified, the PART 
ratings should probably be largely ignored by Congress.
    Thank you. I look forward to your questions.
    Senator Coburn. Thank you.
    I wonder if either of you might want to comment on Mr. 
Hughes' testimony. It is certainly different than what we heard 
from either Mr. Johnson or Ms. Norcross, and I have several 
questions for Mr. Hughes as well, but I thought--Mr. Johnson, 
would you like to comment?
    Mr. Johnson. Yes; several other countries around the world 
think the PART is great; other States in America think the PART 
is great. Most good government groups think it is great. It is 
an instrument. It has had blunt; will get better every year.
    Most people that observe Congress, that have been around 
Congress a long time, believe that the Executive Branch is more 
interested in how well programs work than Congress is. David 
Walker has said that in hearings; so have Dick Armey and 
others. I would bet you agree. It is very hard to produce 
performance information and program assessments. What the 
Administration has done with PART is a place to start. We have 
been working 5 years on this. I do not believe Congress is 
going to invest 5 years to put together the information that we 
have right now. The PART information is a starting point for 
building better mechanisms to holding agencies and programs 
accountable for what they do.
    So I believe, in spite of its flaws, that PART is an 
excellent tool. It is a wonderful beginning. It is the product 
of 5 years of effort. I do not see this as a power grab by the 
Executive Branch. The subject of this hearing is why won't 
Congress pay attention to PART, so I don't think Congress is 
actually reeling with this onslaught of performance information 
from the Executive Branch. Our challenge is to get them to pay 
attention to it.
    Senator Coburn. Thank you. Ms. Norcross.
    Ms. Norcross. What is the alternative to not using 
performance information? PART has given us--at least we have 
moved the discussion away from the policy preferences of an 
administration towards evaluating programmatic activity. So I 
do not know what the option would be. Should we revert back to 
a system where we simply do not use performance information, 
expect it, gather it, or analyze it? And if there is discomfort 
with OMB performing these assessments, perhaps Congress should 
undertake that.
    I understand Congress only engages about 7 percent of its 
time in oversight. So the current legislative mechanisms that 
are supposed to be engaged in this activity are not working up 
to speed. So that would simply be my response is if Congress is 
supposed to be evaluating these programs, where is the evidence 
that it is, in fact, evaluating them and providing guidance to 
agencies along the lines of performance?
    Senator Coburn. All right; thank you.
    Mr. Hughes, you mentioned that there is significant bias 
and distortion and manipulation. Would you give me examples of 
bias, please?
    Mr. Hughes. Sure. There are a number of different types of 
biases that can be involved in this. One is the perspective of 
the OMB officer. The budget officer at OMB is the person who 
has the final say on what the language will be for the answers 
to the questions, how that language that is written will 
translate into a yes or a no or a few of the modified answers 
that are possible now under the PART and also how those yeses 
and noes get translated not only into the numeric raw score but 
also into the actual rating.
    There are a lot of inconsistencies between the guidelines 
that have been laid down for what raw score equals what rating 
and what the programs that have been reviewed actually get. 
That is one type of bias, and that is from a kind of 
implementation perspective. There are other biases in the 
actual design of the tool. I think that the format under which 
it was designed, which was designed to be accessible to people 
who may not be policy experts or who want to just know, like 
you say, come and look and see whether the government is 
getting results and whether the program is working, that 
necessitates that certain things are left out.
    One of those things is whether the Congress has designed a 
program to have multiple goals. Many programs in the Federal 
Government are designed to have multiple goals. That sort of 
thing is not taken into consideration within the PART. 
Oftentimes, those goals can be conflicting. That does not 
necessarily mean that it is a bad design. That just means that 
it is a complex program. And that kind of complexity is lost in 
the way the tool was designed to apply to people who may not be 
policy experts.
    Senator Coburn. Are you saying that there could be another 
PART program that would take into account for that? What is 
wrong by demanding a clear program mission from agencies?
    Mr. Hughes. Certainly nothing.
    Senator Coburn. And questioning how a program fulfills that 
goal; is there anything wrong with those two things?
    Mr. Hughes. No.
    Senator Coburn. So you do not disagree that a PART program 
might be designed better to take out more bias, but you do not 
disagree with the fact that knowing what a program's goal is 
and measuring performance against that goal, it should be an 
effective tool. You would not disagree with that?
    Mr. Hughes. No, theoretically, I agree with you.
    Senator Coburn. The one problem I had with your testimony 
is the problem I have with the rest of Congress is we are lazy. 
And the fact is that this is the 37th oversight hearing of this 
Subcommittee. Go find another one that has done that. And the 
point is that ideally, Congress does have the responsibility, 
but they do not live up to it. And so, what we are working with 
is in a vacuum, is Congress ideally should be doing this. I do 
not disagree with you, but they are not.
    And to have a blunt tool that is getting better, even 
though it can be criticized, and I think Mr. Johnson would 
agree that it is subject to some criticism, as is any 
assessment tool when you first start using it. But to say we 
should not have them doing it because it is Congress' role--I 
agree; that is why I am doing it; that is why we have done 37 
of them, begs the question of how do we motivate Congress to do 
oversight?
    So if we are critical of this one, answer me the question 
how I motivate my peers to do the appropriate thing when it 
comes to authorizing a program, and in that authorizing, saying 
we are going to measure it and then having the incentive to 
have Congress do the oversight to see whether or not they have 
a goal, and they are meeting that goal.
    Mr. Hughes. That is, of course, a very difficult question, 
one that I will probably be very insufficient in answering, 
giving a satisfactory answer for. I think that the oversight 
role of Congress, and you are correct, of course, in citing the 
fact that Congress does not really do oversight any more. That 
is indicative of larger things about our political system, 
about the way that the electoral process works, about the 
importance of fundraising. There are multiple things that are 
in there that actually have nothing to do with whether Congress 
should do oversight or not that are enormous problems that 
would be difficult to tackle.
    I think your leadership on this issue is important. I think 
we need to have more folks in Congress who are paying attention 
to these sorts of issues. I do not know if there is a magic 
bullet procedural change or a statute or something that we 
could do that would make it so that Congress would be forced to 
do oversight more. I do think that some of the suggestions that 
have been made in front of this Subcommittee in the past about 
taking the Program Assessment Rating Tool or a modified version 
of it outside of the Office of Management and Budget, perhaps 
maybe having the Government Accountability Office do it or 
establishing a committee within Congress that would provide 
oversight in that regard. I think those ideas are worth 
exploring.
    I do not think that you can just remove the PART the way it 
exists now and give it to GAO and have it work well. I think 
there are design flaws that need to be corrected, that need to 
be adjusted, and I am, of course, sympathetic to the point that 
if you change it too much, the previous reviews would not be as 
useful. But it is not necessarily just a problem with the way 
that the tool gets done at OMB. We think there are deficiencies 
within the way it was designed as well.
    Senator Coburn. Well, you would be agreeable, then, to 
submit to this Subcommittee the things that you think are 
deficient in the design so that we can look at that?
    Mr. Hughes. Sure, and that was reflected in my written 
testimony. There is a section on that.
    Senator Coburn. One of the problems with oversight is that 
a lot of agencies do not respond to our questions. Let us say 
we had oversight, and they do not respond. The only way you can 
solve that is either have somebody who can squeeze them on 
their money, or we have to squeeze them until they respond. But 
that requires the sausage-making process to be able to 
accomplish that.
    The thing that is disconcerting is I have little faith that 
Congress is going to step up to the bar until they are 
absolutely forced to through a financial disaster to make the 
hard choices. Congress wants to avoid hard choices, and as long 
as they do not feel the pinch, they will not make the hard 
choices. And that is why 2016 is going to be a very tough year 
for this country, because that is when the pinch starts, the 
big pinch. And so, having an assessment tool, blunt, maybe 
somewhat biased, maybe somewhat distorted is, in my mind, 
better than nothing at all.
    Mr. Johnson, and you may not care to comment on this, but 
you might comment on the motivation behind it: The House 
Subcommittee on Labor/HHS put a prohibition in their bill this 
past week that precludes any money from being spent on the PART 
assessment. Any comments on the motivation behind that or what 
you see? I am not trying to create a problem for you with the 
Subcommittee, but how did we get there?
    Mr. Johnson. Well, there is one unelected staff member who 
is opposed to the PART. He worked on the Treasury/
Transportation bill last year and put a similar prohibition in 
there. He was at HUD before that, and he disagreed with HUD's 
use of the PART, and he was at OMB before that. One unelected 
staff member is responsible for the provision. The chairman of 
the committee had no knowledge that it was in the bill. It is 
inexplicable to me that language like that is in the bill. That 
is my only comment.
    Senator Coburn. OK; one of the other things, Mr. Hughes, 
with your testimony which I find, well, less than congruent is 
the statement that the PART increases the White House's power. 
And the problem with that is Congress ignores the PART 
assessment. We have been able to do nothing with the PART 
assessment. Even when I look at all of it, and I look at the 
agencies, and I have done the oversight, and I try to get 
somebody to do something about it, Congress ignores it.
    So there is not a power grab there, because Congress is not 
paying any attention to it. So explain to me your reasoning 
behind--is it a potential? Because it is certainly not, in 
fact, acted out. There is no effect of the PART right now on 
the Congress, because they ignore it.
    Mr. Hughes. I actually would agree with you, and I would 
say that would probably be a poor choice of words on my part. I 
do think it is a potential problem. Let us do a for instance. 
Let us suppose that Congress will appropriate funds according 
to whatever the rating on the PART is. Why do we even need 
Congress? Let's just let OMB do it. So I think it is a slippery 
slope. I think that particularly with respect to budgeting, we 
have been working more in trying to explore the management side 
of it as well, of PART, and the usefulness within agencies.
    I think there is more potential for a productive use of the 
information there. I do not think that you can look at a PART 
score and say, OK, well, I know how to fund programs now 
because of these problems. So the way I chose my words is 
probably poor. I do not think it is a problem right now; as you 
say, you are correct, that Congress does not pay attention to 
them.
    Senator Coburn. Well, but let me create a scenario for you. 
Let us say that Congress is doing great oversight on 
everything. We are sunsetting things; we are reauthorizing 
them; we are bringing them back up; we really know what we are 
doing and that we are doing a good job of that. Let's make that 
assumption. That is an absolute lie, but let's make that 
assumption.
    Would you deny the fact that the Administration should have 
a performance tool themselves to measure what the goal is of 
the program and whether or not they are meeting that goal as a 
management tool to become more effective in carrying out the 
will of the Congress?
    Mr. Hughes. No; I think the problem exists when the tool 
that the Administration designs, or it does not even have to be 
this one, the Executive Branch designs portrays itself as an 
unbiased, objective evaluation of how programs and management 
are going at agencies when, in fact, it is anything but that. 
So I do not think that--again, in theory, that this is 
necessarily a problem. But with this particular instance, it is 
kind of like a wolf in sheep's clothing. You have a situation 
where they are saying we are doing this; it is systematic; it 
is transparent; it is on the Web; the public can view it; this 
is an innate good.
    But the kind of things that we worry about are the things 
that are not transparent within the PART, that you do not 
necessarily see up front when you look at the one-page review. 
That is where you get into a tricky situation, and it is 
perfectly fine for the Executive Branch to have their own 
systems and whatever they like, but the problem occurs when 
they try to sell that to Congress as the one objective 
evaluator.
    Senator Coburn. But they have not. They have just said 
since you are not doing one, we are going to do one, and here 
is what we have found, and here is what our recommendation is. 
We still control the purse strings, and it is obvious from the 
PART assessment that Congress has totally ignored the 
Administration when it comes to evaluating programs. So that is 
not seen as a risk to me whatsoever.
    Mr. Hughes. Well, that is encouraging to hear.
    Senator Coburn. Well, they have not.
    Mr. Hughes. Well, I would say that they have not succeeded.
    Senator Coburn. I think it is very discouraging to hear, 
because they are not looking at the other as well.
    Mr. Hughes. Fair enough.
    Senator Coburn. They are paying attention to nothing and 
continue it. One of the battles I have, and I will share it in 
the Subcommittee, is there are a lot of bills that I block; 
they are authorizing bills. And I go to Members of the Senate, 
and I say these are the things that I have problems with. And 
they say, well, why do you have problems? And I say, well, you 
have not looked at the programs that are already there before 
you authorize another program, and you have not said we are 
going to eliminate this program and put this one in. You are 
authorizing another program to do the same thing that is 
already happening without deauthorizing another program.
    And what I get told: Well, we do not do things that way. 
Well, the American people do things that way. Business does 
things that way. States do things that way. Why should Congress 
not do it? So, really, we are shooting the messenger here. The 
messenger--there is a vacuum in terms of oversight, and we now 
have an Administration that has attempted, whether we think 
their tool is good or not. And you do not doubt that the tool 
is getting better as they have used it? They are using a tool 
that is improving, that does have maybe some bias and does have 
some risk for manipulation in it, but the fact is it is the 
only thing available right now, especially since this 
Subcommittee has time getting even agencies to come and testify 
before it or to give us information.
    Mr. Hughes. I will respond with two points. One, your 
shooting the messenger analogy, I think that may be part of our 
criticism of it, but our problem with it is that when the 
messenger leaves with his message, and when he gets to his 
destination, he is carrying two different messages. There is a 
problem with the transmission along the way, and that is 
something that is important to realize, regardless of where the 
criticisms are being pointed at.
    I think the second thing is, and I sympathize with your 
frustrations about oversight in Congress, and that is certainly 
something that we would like to see a ton more of. I think you 
can kind of get around some of the rhetoric around what 
government--we have all these programs, and they do not do 
anything that is important. If we had more oversight, if we had 
more openness about what the government actually does, I think 
people will actually have a greater appreciation of things.
    Senator Coburn. Right.
    Mr. Hughes. So I think our criticism--try to be focused on 
this particular instance of PART, the way that this PART 
assessment works. I do not think that it should be thrown in 
the garbage can. I think that it is very important that people 
in Congress and people in the agencies and the public know that 
this should be, despite the fact that there is not a lot going 
on elsewhere, this should be a really tiny part about 
evaluating how government works. That would be my caveat 
about--I am sympathetic to the fact that it is not going on 
elsewhere, but try not to latch on to it and say this is the 
tool, and this is what is going to get us there.
    Senator Coburn. Nobody has in Congress. Would all three of 
you agree that some type of assessment of goals and measurement 
against the goals changes expectations of program managers?
    Mr. Johnson. I agree totally.
    Senator Coburn. Ms. Norcross.
    Ms. Norcross. Totally correct.
    Senator Coburn. Mr. Hughes.
    Mr. Hughes. In my limited experience, I would say that is 
right.
    Senator Coburn. Ms. Norcross, you have some experience with 
performance tools in New Zealand, and I also know that South 
Korea has adopted assessment programs. Could you comment on 
those two things?
    Ms. Norcross. Morris McTeague, with whom I work at the 
Government Accountability Program, has direct experience with 
the New Zealand experience in developing performance 
information systems and applying them to remedy some of New 
Zealand's budget crises. And if I could answer that question 
later, I could get you more information in specific on some of 
the reforms that they have undertaken. We are right now doing 
an analysis of that.
    Senator Coburn. OK.
    Ms. Norcross. So I could provide that for you.
    Senator Coburn. Mr. Johnson, the question of bias in the 
instrument that you use, give us an example of three or four of 
the questions that PART asks about programs.
    Mr. Johnson. Well, it asks if the program has a clear 
definition of--this is not exact wording, but it asks about do 
you have a clear definition of success? Does it have a good way 
of measuring your performance relative to that? Is it meeting 
its performance goals? It asks about the quality of management 
the program has. Do the program have an efficiency goal? Is it 
Management 101, or it is Accountability 101?
    These assessments are put together by the agency and OMB, 
not by OMB alone. The agency and OMB are supposed to agree on 
the program performance goals. Just as agencies are afraid to 
disagree with Congress, agencies are sometimes afraid to 
disagree with OMB about its assessment, But if they really 
disagree with the assessment, agencies can submit their 
disagreements to an appeals board that I chair and that is made 
up of deputy secretaries from four or five agencies. We get a 
number of appeals every year, and we review them. Some of them, 
we approve, and some of them, we reject.
    And we also conduct what we call consistency checks, where 
we review if the PART follows the rules we have for answering 
the questions. We also review whether the answers in a PART are 
consistent with each other.
    As we look also at programs dealing with the same subject 
across agencies, we pull all the relevant program assessments 
together when we start doing a cross-cut analyses to make sure 
we are equally attentive to the issues, equally focused on the 
quality of the performance measures and so forth, because we 
are going to be using this information to compare one program 
to another. We use cross-cutting analyses to see if there is 
something an ineffective program can learn from an effective 
program dealing with the same topic.
    So there is a lot of effort to make the assessments 
consistent; to make the information reliable; and to remove 
bias. There is bias in anything a human being does. So I have 
no doubt that these are not perfect instruments, but they will 
get better over time, and the assessments that we have done in 
the last 2 years are better than the assessments we did in the 
first 2 years.
    Senator Coburn. So, in other words, the programs set their 
own outcome measurements.
    Mr. Johnson. The program does. OMB and the program staff 
have to agree that the performance measures are acceptable.
    Senator Coburn. And then, they measure themselves against 
it.
    Mr. Johnson. They then determine the metrics they will use 
to measure performance and how to collect the data, and how 
often to collect that data.
    Senator Coburn. And so, where is the bias in that?
    Mr. Johnson. I do not know; just because----
    Senator Coburn. If they are participating in setting the 
goal, and they are participating in setting the metrics, and 
they are the ones doing the measurement of the metrics, where 
is the bias?
    Mr. Johnson. I do not know where there is unusual bias. I 
know that there is a bias in anything that human beings are 
involved in. So I do not know what specifically Mr. Hughes is 
talking about.
    Senator Coburn. In teaching to the test, a problem across 
agencies as they respond to PART questions, Mr. Hughes wrote in 
his testimony that agency officials told him they gamed the 
system to avoid negative scores and consequences. Do you think 
that is true? Is there something in the program to help 
alleviate? We know everybody games when they are being measured 
to an extent. Are there things in the PART assessment system 
that take that into account?
    Mr. Johnson. I know that agencies like to be green. They 
really like to be green, and they really like to have good PART 
scores. And so, they do a lot of things to please OMB and to 
get good scores and to look good on that scorecard.
    Senator Coburn. Does that carry out into changed programs 
and changed management to make the programs more effective to 
deliver better process and therefore better response by the 
government to the very people they are supposed to be helping?
    Mr. Johnson. Well, teaching to the test, gaming the PART to 
get a good score and providing only superficial analysis does 
not help the program work better. One approach that we have to 
improve the quality of program performance is shine a real big 
light on it all, which is one of the primary reasons we took 
all this assessment information, summarized it, and put it on 
the Website ExpectMore.Gov for all the world to see and for 
people to look up and say that is not the way I know the 
program works. An employee can look at it, or someone served by 
the program can look at it and say, well, that program does not 
work very well; it is ineffective as far as I am concerned, and 
they can complain to the agency or complain to OMB or complain 
to their Senator or Congressman.
    Shining a lot of light on how the program is assessed, on 
what performance measures are used, and on what the performance 
information says can drive improvements in the measures that 
are used, the data that are used, and the quality assessment. 
So that is why I believe it is so important to have No. 1 on 
your list posted on this sign, which is transparency. You can 
have all of this information, but unless we shine a really big 
light on it, it will not get better over time. That is why we 
took it with all of its warts, with all of its dimples, put it 
out there. Now, let us begin the process with agencies and with 
Congress, I would hope, to improve this program performance 
information, to make these assessments better and to make our 
plans to help program perform better more aggressive.
    Senator Coburn. Let me follow up on that for a minute. So 
Mr. Hughes can go to every PART assessment and via the 
government Website can look at the goals, the metrics, the 
measurement of the metrics, the response, and the rating.
    Mr. Johnson. Right.
    Senator Coburn. In other words, nothing is hidden. 
Everything that comes to develop that, that can be accessed by 
OMB Watch, so they can see all that.
    Mr. Johnson. Right.
    Senator Coburn. Right; OK.
    Mr. Johnson. There is a one-page summary of every PART on 
the ExpectMore.gov. There are links, at the bottom to the 
detailed PART, which is multiple pages. It is written in OMB-
speak and has historical information and more detailed 
information. That is the meat of the assessment. The summary 
and all the details is available on ExpectMore.gov.
    Senator Coburn. But they can get access----
    Mr. Johnson. Yes, sir.
    Senator Coburn [continuing]. If they need to as well.
    And so, given your emphasis on transparency, are we to 
assume that you are going to be very accepting of our OMB 
transparency bill that Senator Carper and I have.
    Mr. Johnson. That is on the contracting information.
    Senator Coburn. Online grants, contracting, everything.
    Mr. Johnson. We love transparency, and we are working very 
effectively with your staff to figure out----
    Senator Coburn. I understand that.
    Mr. Johnson [continuing]. The best way to get there as soon 
as possible. We are big on transparency and shining the light 
of day on performance to a strengthen accountability.
    Senator Coburn. Any other comments from any of our 
panelists?
    Mr. Hughes. If I could just respond to some of the stuff 
that we have been talking about, the bias in the data and how 
the data, which data is important and which data is not 
important, there is a tension between outcomes and outputs in 
any type of performance management initiative. The PART focuses 
on outcomes, which is certainly a good goal. We think it is 
more of a broad government-wide goal, maybe something that 
should be included in something like the GPRA, the Government 
Performance and Results Act.
    A lot of times, you cannot judge the effectiveness of 
programs based purely on outcomes, and I will give you a couple 
of examples. One program that is run out of, I believe it is 
the National Park Service, is an office that works as a 
consultant with local communities to transform the neglected or 
unused areas into public space: Parks, playgrounds, those sorts 
of things. They have collected, long before PART came along. 
And another thing that you should know is that agencies have 
collected performance data before PART came along. It is not 
like they were not doing this to begin with, and then, all of a 
sudden, OMB says you have to do it.
    Senator Coburn. Some were not.
    Mr. Hughes. Some were not; that is correct; not all of 
them.
    They had a couple of standards by which they judged whether 
they are doing a good job in this program that acts as a 
consultant. One is through surveys with the local communities 
that they consult with: Were you satisfied? Did people use the 
parks? Did you like the services we provided? Another way is 
they used to collect data about based on the amount of money 
that they were given, how many square acres of parks did they 
create? How many miles of jogging trails, those sorts of 
things.
    Those are outputs, the second part. The survey part could 
be both. OMB, in the PART process, wanted them to focus on 
outcomes. And one of the things that they said should be an 
outcome was, Are the people living in the community healthier? 
And I think that is a perfectly good goal. I think people 
should be healthier. But the program in the National Park 
Service has no way to force people to go and jog in the park. 
All they can do is say this is the money we got to create parks 
for communities. These are the parks we created. These are the 
people we worked with and what they thought of what we did.
    That is one instance, one example of multiple examples of 
the difference between outcomes and outputs and how certain 
programs are not necessarily structured to focus completely on 
outcomes, or maybe the outcomes are beyond their control. And 
one other example I will share about bias within the PART is 
the Appalachian Regional Commission. This is a program that 
Congress decided to be a patchwork, to cover the holes between 
other programs that were working in similar issue areas.
    Senator Coburn. I have been trying to get rid of it for 10 
years.
    Mr. Hughes. I am aware of that. [Laughter.]
    And I do not have a personal perspective on the Regional 
Commission myself, but Congress designed this program to fill 
in the blanks, in the holes between programs, and the PART 
assessment said that this is not a unique program, which of 
course, it is not, because Congress designed it not to be 
unique. It was designed to be duplicative, because the evidence 
that Congress had seen at the time said that there are things 
that are being missed.
    And we can talk more about, maybe we should have just 
pulled all of the programs together and redesigned them so that 
the holes are not missed, and that is certainly something that 
OMB is trying to do.
    Senator Coburn. The whole point is you raise the question 
about what Congress has not done so that they will do it 
better. And to say that it is a blunt--it is a blunt tool, but 
it raises it up to a level so that somebody has to now--let's 
address this, and we have not addressed the Appalachian 
Regional Commission. What we have done is we have let it 
continue to do exactly what it does, and the danger with that 
is: One, we are not efficient; two, we could design a program 
that helps a whole lot more people with the same dollars; or 
three is we could help the same amount of people with a whole 
lot less dollars, which gives us dollars to help somebody else 
somewhere.
    So the point is that is a commission that I am very well 
informed on, and I believe even the blunt tool will show that 
we could be much wiser as Congress to make the goals of that 
program more effective. I believe outcomes is the measure. I 
believe the American people want outcomes as the measure. But 
part of it is laziness on our part. When we write a program, 
and this is something I am critical of Congress, we ought to be 
very specific about what our intentions are, and we are not. We 
ought to be very specific about what we expect, and we are not. 
We ought to be very specific on how we want to measure whether 
we got what we expected, and we are not.
    So a lot of the problems do not have anything to do with 
you all in front of us; they have to do with Congress not being 
good legislators so that we design a system that can be looked 
at later and say did we go after what we intended to go after? 
Did we accomplish what we intended? And did we do it in a way 
or within the cost parameters that we thought it would?
    And so, the real criticism is not at OMB. They are dealing 
with what we have dealt them. The real criticism is for us in 
not being specific enough in terms of--and you can ask staff: 
When I write a bill, I want it all the way down to the T. I 
want limited discretion, because if we are going to write a 
bill and do not know enough about it, we should not be writing 
the bill until we get the information to write a bill 
correctly. And I do not know many people who would disagree 
with that. It is just easier to write it loose and let somebody 
else worry with the details, and that is called lazy 
legislating.
    Mr. Hughes. That has been our experience working with you 
on the transparency bill as well. I think, though, that it is 
not necessarily as easy as you might make it out to seem. There 
is another example: The Consumer Product Safety Commission was 
ruled down on the PART review for not using cost-benefit 
analysis in its regulatory rulemaking. It is actually 
prohibited by Supreme Court decision from using cost-benefit 
changes like that in their rulemaking.
    That is not something that the program can control. And I 
agree with you that it is good that even with the Appalachian 
Regional Commission example that these things are brought up to 
Congress. But as you have said many times, with the lack of the 
kind of investigatory role that Congress is playing now into 
how programs are made, we are concerned that the information 
that we all admit has some biases and those sorts of things 
will be taken as a snapshot, and the investigation will not be 
done to get underneath what the rating is.
    So the ineffective for the Consumer Product Safety 
Commission, it should not be said we should get rid of it. But 
it is the bias in the tool that gives you the ineffective.
    Senator Coburn. But experience tells us that is not 
happening, because Congress is not paying attention to PART, 
and they are not paying attention to their own. They are 
ignoring them both. So your fear is unfounded, because we are 
not using it.
    Mr. Hughes. Well, we would like to be vigilant.
    Senator Coburn. We should do both, though. We should be 
using theirs plus our own, and that is the point. Outcomes, to 
me, is the measurement, not outputs. And outcomes, if we design 
something to have an outcome, then, we ought to know what that 
outcome measurement is, and then, we ought to hold agencies 
accountable to be to that outcome. And I will just give you a 
great example: How about the incidence of HIV reduction in this 
country, which has not happened, and then, we spend money on 
flirting classes? And there is no connection between the two. 
In other words, if somebody is going to measure outcome, we 
ought to be asking why, with all of the money we are spending 
on HIV that we are not seeing a reduction in the incidence of 
new HIV cases in this country.
    And yet, nobody is measuring the performance against that 
outcome, and that is an outcome that makes a difference in 
lives. It is not outputs; yes, we are spending a lot of money, 
but we are not measuring outcomes, and therefore, we are not 
getting the ability to make the programmatic changes that need 
to be made on the congressional side to accomplish that.
    If the court prohibits a program from operating well, OK, 
if it prohibits a program from operating well, that tells us we 
have a problem in the design of the program.
    Mr. Hughes. Well, I would disagree with that 
classification. I do not think it is prohibiting it from 
operating well. I think it is saying that the program needs to 
take certain considerations into account when it does operate. 
It needs to say there are certain things, equity issues within 
programs in the Federal Government that are important to take a 
look at. It is not necessarily that the Supreme Court is 
putting up a roadblock in front of them getting the job done. 
The Supreme Court is making a value judgment about how the 
program should operate.
    Senator Coburn. Which is not the Supreme Court's job. The 
Supreme Court's job is to interpret the laws and the 
Constitution and the treaties, not to tell Congress how to run 
the budget of the country, and that is----
    Mr. Hughes. And I would also say to you, too, that it is 
not OMB's job to tell you how to run the budget of the country 
either.
    Senator Coburn. No, we agree.
    Mr. Hughes. Yes.
    Senator Coburn. You will not disagree with me on that at 
all. I believe we have abdicated our responsibility, and the 
reason OMB is having to do this is because we have not. But I 
have no heartburn with somebody doing it somewhere. At least we 
have some information with which to make a decision.
    I want to thank each of you for being here. I would like a 
little more formal response from your organization on specifics 
on how you would definitely change an assessment tool program 
for agencies and what we might be able to accomplish that would 
limit, and I want that as justifiable constructive criticism so 
that when we look at PART, we can have your thoughts in detail 
on how we can assess that and maybe make recommendations.
    Mr. Hughes. We look forward to that opportunity.
    Senator Coburn. All right.
    Thank you all so much for being here. The hearing is 
adjourned.
    [Whereupon, at 3:43 p.m., the Subcommittee adjourned.]



                            A P P E N D I X

                              ----------                              

[GRAPHICS NOT AVAILABLE IN TIFF FORMAT] 





                                 
