[Senate Hearing 110-817]
[From the U.S. Government Publishing Office]



                                                        S. Hrg. 110-817

   IMPROVING FEDERAL PROGRAM MANAGEMENT USING PERFORMANCE INFORMATION

=======================================================================

                                HEARING

                               before the

                FEDERAL FINANCIAL MANAGEMENT, GOVERNMENT
                   INFORMATION, FEDERAL SERVICES, AND
                  INTERNATIONAL SECURITY SUBCOMMITTEE

                                 of the

                              COMMITTEE ON
                         HOMELAND SECURITY AND
                          GOVERNMENTAL AFFAIRS
                          UNITED STATES SENATE


                       ONE HUNDRED TENTH CONGRESS

                             SECOND SESSION

                               __________

                             JULY 24, 2008

                               __________

       Available via http://www.gpoaccess.gov/congress/index.html

       Printed for the use of the Committee on Homeland Security
                        and Governmental Affairs






                  U.S. GOVERNMENT PRINTING OFFICE
44-582                    WASHINGTON : 2009
-----------------------------------------------------------------------
For sale by the Superintendent of Documents, U.S. Government Printing 
Office Internet: bookstore.gpo.gov Phone: toll free (866) 512-1800; DC 
area (202) 512-1800 Fax: (202) 512-2104  Mail: Stop IDCC, Washington, DC 
20402-0001













        COMMITTEE ON HOMELAND SECURITY AND GOVERNMENTAL AFFAIRS

               JOSEPH I. LIEBERMAN, Connecticut, Chairman
CARL LEVIN, Michigan                 SUSAN M. COLLINS, Maine
DANIEL K. AKAKA, Hawaii              TED STEVENS, Alaska
THOMAS R. CARPER, Delaware           GEORGE V. VOINOVICH, Ohio
MARK L. PRYOR, Arkansas              NORM COLEMAN, Minnesota
MARY L. LANDRIEU, Louisiana          TOM COBURN, Oklahoma
BARACK OBAMA, Illinois               PETE V. DOMENICI, New Mexico
CLAIRE McCASKILL, Missouri           JOHN WARNER, Virginia
JON TESTER, Montana                  JOHN E. SUNUNU, New Hampshire

                  Michael L. Alexander, Staff Director
     Brandon L. Milhorn, Minority Staff Director and Chief Counsel
                  Trina Driessnack Tyrer, Chief Clerk


FEDERAL FINANCIAL MANAGEMENT, GOVERNMENT INFORMATION, FEDERAL SERVICES, 
                AND INTERNATIONAL SECURITY SUBCOMMITTEE

                  THOMAS R. CARPER, Delaware, Chairman
CARL LEVIN, Michigan                 TOM COBURN, Oklahoma
DANIEL K. AKAKA, Hawaii              TED STEVENS, Alaska
BARACK OBAMA, Illinois               GEORGE V. VOINOVICH, Ohio
CLAIRE McCASKILL, Missouri           PETE V. DOMENICI, New Mexico
JON TESTER, Montana                  JOHN E. SUNUNU, New Hampshire

                    John Kilvington, Staff Director
                  Katy French, Minority Staff Director
                       Monisha Smith, Chief Clerk

















                            C O N T E N T S

                                 ------                                
Opening statement:
                                                                   Page
    Senator Carper...............................................     1

                               WITNESSES
                        Thursday, July 24, 2008

Hon. Martin O'Malley, Governor of Maryland.......................     4
Bernice Steinhardt, Director for Strategic Issues, U.S. 
  Government Accountability Office...............................    18
Marcus C. Peacock, Deputy Administrator, U.S. Environmental 
  Protection Agency, and Former Official at U.S. Office of 
  Management and Budget..........................................    20
Donald F. Kettl, Director, Fels Institute of Government, and 
  Robert A. Fox Professor of Leadership, University of 
  Pennsylvania...................................................    23
James Dyer, Chief Financial Officer, and Performance Improvement 
  Officer, U.S. Nuclear Regulatory Commission....................    36
Scott Pace, Associate Administrator for Program Analysis and 
  Evaluation, and Performance Improvement Officer, National 
  Aeronautics and Space Administration...........................    38
Daniel Tucker, Performance Improvement Officer, and Deputy 
  Assistant Secretary for the Budget, U.S. Department of Veterans 
  Affairs........................................................    40

                     Alphabetical List of Witnesses

Dyer, James:
    Testimony....................................................    36
    Prepared statement...........................................    92
Kettl, Donald F.:
    Testimony....................................................    23
    Prepared statement...........................................    86
O'Malley, Hon. Martin:
    Testimony....................................................     4
    Prepared statement...........................................    49
Pace, Scott:
    Testimony....................................................    38
    Prepared statement...........................................    97
Peacock, Marcus C.:
    Testimony....................................................    20
    Prepared statement...........................................    84
Steinhardt, Bernice:
    Testimony....................................................    18
    Prepared statement...........................................    55
Tucker, Daniel:
    Testimony....................................................    40
    Prepared statement...........................................   100

                                APPENDIX

Questions and Responses for the Record from:
    Governor O'Malley............................................   107
    Ms. Steinhardt...............................................   112
    Mr. Peacock..................................................   117
    Mr. Tucker...................................................   128
Prepared statement from Robert Shea, Associate Director for 
  Government Performance, Office of Management and Budget........   131
Charts submitted for the Record by Governor O'Malley.............   136

 
   IMPROVING FEDERAL PROGRAM MANAGEMENT USING PERFORMANCE INFORMATION

                              ----------                              


                        THURSDAY, JULY 24, 2008

                                 U.S. Senate,      
        Subcommittee on Federal Financial Management,      
               Government Information, Federal Service,    
                              and International Security,  
                          of the Committee on Homeland Security    
                                        and Governmental Affairs,  
                                                    Washington, DC.
    The Subcommittee met, pursuant to notice, at 2:35 p.m., in 
room SD-342, Dirksen Senate Office Building, Hon. Thomas R. 
Carper, Chairman of the Subcommittee, presiding.
    Present: Senator Carper.

              OPENING STATEMENT OF SENATOR CARPER

    Senator Carper. The Subcommittee will come to order.
    Governor, welcome. We are delighted that you are here. 
Thank you so much for taking time to join us. Senator Tom 
Coburn is the Ranking Republican of this Subcommittee. He will 
be here shortly. But given all that you have on your plate, 
Governor O'Malley, we are delighted that you are able to find 
time to be here and talk about something not only that you are 
interested in, but you know a whole lot about, and we thank you 
for your willingness to share your experiences, your 
perspectives, both as Mayor of Baltimore and now as Chief 
Executive of the State of Maryland.
    One of my favorite sayings is ``I would rather see a sermon 
than hear one,'' and I think we can look at what you have done, 
both as mayor and as governor of your State, to actually show 
us by your own behavior that this works and it is something 
that may just work for us, as well.
    In a little less than 6 months from now, we will have a new 
President standing on the West Portico of the Capitol, 
preparing to raise his right hand and take an oath to defend 
our country and Constitution. Our new Chief Executive will face 
a soaring Federal deficit at home, and security challenges 
abroad. Those are just a couple of the exceptional challenges 
that our Federal Government must be prepared to face. No matter 
who is elected President, whether it is Senator Obama or 
Senator McCain, we must make sure that person has the 
information and the tools that they need in order to keep our 
ship of state headed toward the horizon.
    And though our politics may differ here in Washington and 
across the country, I think we can all agree that the strength 
of our democracy hinges on the ability of our government to 
deliver its promises to the people. We have a responsibility to 
be judicious stewards of the resources the taxpayers invest in 
America and ensure that those resources are managed honestly 
and that they are managed effectively.
    Just recently, this Subcommittee heard from a number of 
witnesses, including our former Comptroller General, David 
Walker, about the dire fiscal situation this country faces, not 
just this year, not just in this decade, but for decades to 
come. Over the next two decades, 80 million baby boomers--I am 
one of them--will become eligible for Social Security and 
Medicare. Today, these two programs already make up over 40 
percent of our government's total expenses. And as boomers like 
me start to draw benefits, some experts we have heard from say 
that the share of these programs could equal within 40 years 
not 40 percent of our government's outlays, but all of our 
government's outlays.
    Without any changes, we will not have extra funds to 
prepare a world-class workforce in the 21st Century, funds to 
make us energy independent, meet our transportation needs, 
clean our air, or address our pressing national security needs. 
There is only so much pie to go around, so how we manage who is 
getting the next slice, and how big it is going to be, becomes 
increasingly important.
    In 1993, the year that I became Governor of Delaware, the 
Government Performance and Results Act (GPRA), was enacted to 
help us better manage our Nation's finite resources and improve 
the effectiveness and the delivery of Federal programs. Since 
that time, agencies have collected, and provided to Congress, a 
tremendous amount of performance data. However, Senator Coburn 
and I wondered if this information was being translated into 
results, or at least into improved results. Producing the 
information does not by itself improve performance, and we knew 
that there had been little increase in the actual use of 
performance data by agency managers.
    A lot of work has been done by GAO and others on how this 
data could be used, but nobody has really looked into whether 
agencies were putting theory into practice. A year or so ago, 
Senator Coburn and I asked the GAO to examine how performance 
information is being used to better manage Federal agencies, 
and if managers could be, and should be, employing it more 
often in their decisionmaking processes. We also asked the GAO 
to consider the recent efforts by the Administration to improve 
the usefulness of agency performance information and through 
initiatives such as the Program Assessment Rating Tool (PART), 
a key component of President Bush's Management Agenda.
    I am eager to hear GAO's findings and ask that we focus our 
discussion on the following crucial questions today. To what 
extent are Federal agencies using performance information to 
perform key management tasks, such as identifying performance 
problems and taking corrective actions or identifying and 
sharing best practices?
    Second, how can Federal agencies make better use of 
performance information to improve our results, and our 
outcomes?
    And finally, what lessons can be learned and how can we 
build on this Administration's efforts to improve the 
usefulness and employment of performance information?
    Today, with these questions in mind, I want us to do the 
following. One is to accurately assess how well and how often 
Federal agencies are using performance information to correct 
problems and promote best practices.
    Two, I would like for us to discuss how the use of 
performance information can be increased and improved in the 
next Administration.
    And three, I want us to solicit ideas about how Congress 
can play an effective role in the path forward.
    Today, it is my hope that we can begin to develop a 
blueprint for the next Administration. We must do all that we 
can to ensure that the transition is streamlined and that our 
next President has a clear picture of the strengths and 
weaknesses of agencies under his direction.
    We face, in this country, unparalleled challenges both here 
and abroad and these require a knowledgeable and nimble Federal 
Government that can respond effectively. With concerns growing 
about our mounting Federal deficit and national debt, the 
American people deserve to know that every dollar we send to 
Washington is being used to its utmost potential. Performance 
information is an invaluable tool that can do just that, if we 
use it. If used effectively, it can identify problems, help us 
find solutions, and develop approaches that improve results.
    We thank, again, not only our lead-off witness, but all of 
our witnesses for taking this opportunity, setting aside time 
in their own lives to be with us today to share with us some of 
their ideas about the challenges before us and how best to 
address those challenges.
    Our lead-off witness is Governor Martin O'Malley. He was 
elected Governor of Maryland in 2006, coming on the heels of an 
unusually successful tenure as Mayor of the City of Baltimore. 
Early in his life, as a young troubadour, he led his Irish 
group into the City of Wilmington and left the fans at O'Friels 
Irish Pub standing and cheering in his wake at an age when he 
was actually too young to get in legally to O'Friels, but came 
in and did a great job performing for us.
    In 2005, Time Magazine named him one of America's Top Five 
Big City Mayors. As Mayor, he pioneered the CitiStat, a 
statistics-based tracking system that focuses in on areas of 
under-performance and demands a results-driven government 
model. CitiStat has saved Baltimore residents more than $350 
million and was awarded Harvard University's prestigious 
Innovations in American Government Award in 2004.
    As governor, he has brought the program State-wide, 
implementing StateStat across all Maryland's government 
services, and provided a model not just for mayors, I think not 
just for governors, but I think maybe for Presidents and for 
those of us who serve here in the Legislative Branch of our 
government.
    We are honored by your presence and we are especially 
honored to be able to serve here on a daily basis with your 
mother, who works, as some of the people here know, for Senator 
Mikulski. I am not sure who works for whom, and I don't know if 
Senator Mikulski is actually here. She was going to try to stop 
by today. She may pop in. But if your mother is around and if 
she shows up, please introduce her to us. We are delighted that 
you are here and that your mom raised you so well. [Laughter.]
    Governor O'Malley, you are recognized to speak for as long 
as you wish.

   TESTIMONY OF HON. MARTIN O'MALLEY,\1\ GOVERNOR OF MARYLAND

    Governor O'Malley. Chairman Carper, thank you very much.
---------------------------------------------------------------------------
    \1\ The prepared statement of Governor O'Malley appears in the 
Appendix on page 49.
---------------------------------------------------------------------------
    Senator Carper. Thank you.
    Governor O'Malley. Thank you for the opportunity to be able 
to join you at this important Subcommittee and as part of the 
discussion of this day.
    I am also joined by a number of members of my staff. 
Apropos to today's discussion, Matt Gallagher is here who runs 
our StateStat office and is the Deputy Chief of Staff for 
Operations in our State government.
    Senator Carper. Would you ask him to raise his hand?
    Governor O'Malley. There he is.
    Senator Carper. All right.
    Governor O'Malley. He also before that ran the CitiStat 
office in the City of Baltimore before that.
    It is an honor to be here today before you to talk about an 
issue that, frankly, I believe is changing, for the better, the 
way that many of us look at the operations of our government, 
and it is our government. It is our belief that the same 
performance-based governing strategies that were so very 
valuable in igniting Baltimore's come-back and have been so 
valuable to us at the State level in Maryland this past year-
and-a-half can also work not only in the Federal Government, 
but in any large human organization.
    In our public life, we tend to be very good at measuring 
inputs. We typically refer to those inputs as the budget, and 
it is typically done on an annual basis. But we have often 
neglected to pay enough attention to outputs, to the product of 
government. We are constantly asking, what is our funding 
level. For example, for something like interoperable 
communications without asking whether or not the purchases that 
have been made by local, municipal, or State governments this 
year have been in accordance with the standard that moves us 
down the road to a point in time when all first responders 
throughout our Nation can actually talk with one another when 
responding to an emergency.
    Performance-based government, in its essence, is about 
measuring, tracking, and improving outputs. Inputs play a role, 
but only in the pursuit of outputs.
    Mr. Chairman, I was first introduced to this model of 
governing about 8\1/2\ years ago when I began my first term as 
Mayor of the City of Baltimore. And when we were handed the 
keys to that 16,000 person, $2 billion a year operation known 
as city government, we inherited our fair share of challenges, 
some of them very big challenges. More than 300 of our fellow 
citizens every year were being murdered in our city. The 
streets were too often littered with trash. Our schools were 
too often failing and people were abandoning our city, in 
essence, voting with their feet, and leaving behind them 
buildings and homes that were vacant and themselves becoming 
nuisances.
    Quite understandably, the public at the time was demanding 
immediate results and immediate turn-around, and we rolled up 
our sleeves and got to work and found in beginning that work 
that there was very little in city government that was actually 
being measured in a consistent and real-time fashion. Now, 
don't get me wrong. Oftentimes, information was being collected 
at the ground level, sometimes very faithfully, sometimes very 
dutifully, not always the right information, but rarely, if 
ever, was it being used in a timely manner by the appropriators 
and the policy makers and the administrators at the highest 
level in order to deliver better outputs and better outcomes 
for citizens on the ground.
    So we were very blessed to have met a gentleman by the name 
of Jack Maple, who was one of the great minds behind the 
performance-based strategies employed in the New York City 
Police Department for its turnaround. Jack was the Deputy 
Commissioner of Police under Commissioner Bratton and we felt, 
having observed ComStat in action, that if the NYPD could so 
successfully use simple off-the-shelf software, computer pin 
mapping, deploying police resources to where the crime was 
actually happening, that data collection and mapping technology 
could also work for the other things that government does, 
whether it is garbage collection or repairing streetlights or 
addressing complaints about potholes.
    And from this approach was expropriated the four main 
tenets of CitiStat, which were the main tenets of ComStat. One, 
timely, accurate information shared by all; two, rapid 
deployment of resources; three, effective tactics and 
strategies; and four, relentless follow-up, not on an annual 
basis, not on a biannual basis, but on a daily, weekly, 
biweekly, monthly basis in order to improve performance.
    So we started setting goals. We started measuring results. 
And we did so weekly. We began tracking outputs instead of just 
tracking inputs and we started geomapping every conceivable 
service. And in short time, we turned around the city where 
many neighborhoods were considered ungovernable and we started 
making our city government function again in order to improve 
the quality of life in every neighborhood.
    I go into some greater detail about some of the results we 
achieved in the written statement I submitted to the 
Subcommittee, but just a few examples. Most important of all is 
the primary responsibility of all governments and that is 
public safety. We were able to achieve a 40 percent reduction 
in violent crime, its lowest level, actually, in four decades. 
We were able to back up with 98 percent success a 48-hour 
guarantee to address complaints from citizens about potholes.
    We reduced the number of children exposed to dangerously 
high levels of lead from lead dust, lead paint poisoning from 
old homes and deteriorating homes by 65 percent in a relatively 
short period of time.\1\ We were able to identify and reclaim 
by clearing title more than 5,900 vacant homes and buildings 
which then enables them to be redeveloped and put back on the 
tax rolls.
---------------------------------------------------------------------------
    \1\ The chart referred to appears in the Appendix on page 138.
---------------------------------------------------------------------------
    We had a boarding and cleaning backlog of about some 8 
months, which is how long it took when we began to address a 
boarding or cleaning complaint from any citizen. By the time we 
left, that was down to 14 days. Now, that didn't happen 
overnight. It didn't happen by measuring things annually. It 
happened by measuring them every single day and every week.
    And probably the most important outcome of all is that we 
were able by improving our quality of life with better-
performing government to be able to reverse four decades of 
what had been seemingly insurmountable population loss, and the 
city started growing again.
    I brought two charts with us today. One of them on the far 
right is the combination of homicides, shootings,\1\ and you 
will see three kinds of death, and you can see them over time. 
Again, these are just measured annually, but we measured them 
every 2 weeks and, by golly, when you look over your shoulder, 
you see you are actually making progress shrinking those danger 
zones in our city.
---------------------------------------------------------------------------
    \1\ Chart referred to appears in the Appendix on page 136.
---------------------------------------------------------------------------
    This next map is the cleaning and boarding measured not in 
terms of the reduction of the wait time but in terms of the 
improvement of the productivity.\2\ Again, these are annual 
outcomes, but the only way we were able to achieve the annual 
outcomes is because we developed systems so that we could 
measure every day, every week, and that information was then 
able to get back up to the policymakers, the administrators, 
the appropriators.
---------------------------------------------------------------------------
    \2\ Chart referred to appears in the Appendix on page 137.
---------------------------------------------------------------------------
    In 2007, when we were given the keys to an even larger 
organization, namely the State government of the great people 
of Maryland, we took this model with us. We created a program 
called StateStat, which has allowed us to track and measure 
progress on a level that we have never before been able to 
achieve in our State government. And through our StateStat 
program, while only in its first year and a half, we have been 
able to identify problems in information sharing among various 
law enforcement agencies and across several jurisdictions and 
we work every day to improve them.
    Second, we were able to identify right off the bat the 
imperative, the need to close the House of Corrections, an old, 
dilapidated, and very dangerous prison in our State. We were 
able to close that without incident within 50 days, saving 
probably lives and also saving $3 million in overhead and 
overtime.
    We were also able to identify and fix more than 100 
problems within our juvenile detention facilities, many of 
which had been under various consent decrees and court orders 
for a long time without ever improving those conditions.
    We are now using GPS technology and performance measures to 
target our resources that are geared at restoring the national 
treasure, which is the Chesapeake Bay, through a program that 
we call BayStat. You may notice that virtually anything, if you 
measure it, you can slap a ``stat'' on the end of it and you 
have a new way of saying whether what you are doing is making 
any impact on the problem you are trying to solve.
    We go through a few more accomplishments in the written 
statement I have submitted to you, but I wanted to close by 
just saying a couple of words about CitiStat, StateStat, and 
why this model, I believe, can be and should be applied at the 
Federal level, particularly in the realm of homeland security.
    We believe the same approach can be relevant to governments 
and organizations anywhere and of any size. Recently, I had 
occasion to meet and to listen to Sir Michael Barbour from Tony 
Blair's government who applied many of these principles to the 
administration of national government in the United Kingdom and 
also implemented a new innovation called the delivery unit to 
make sure that all of those along the chain of delivery, from 
policy maker to appropriator to administrator to implementor 
down to the level of citizen, were actually held accountable 
for their piece of delivering improved performance.
    The Environmental Protection Agency, I understand, has 
launched an EPAStat program, and there are governments 
worldwide that are working to implement this model. In fact, we 
have had several delegations from governments all over the 
world who have come to visit us in Maryland to learn about the 
workings, the early workings, of our program of statewide 
performance measurement.
    Government performance management, I believe, is really a 
nonpartisan issue. There is no Democratic or Republican way to 
fill a pothole, to make sure that you improve the outcomes, 
whether it is at a municipal, State, or National Government. 
The beauty of a map is that a map doesn't know whether a 
neighborhood is black or white, or whether a neighborhood is 
rich or poor, or whether a neighborhood is Democratic or 
Republican.
    Most of us in government would say that it is the 
responsibility of every public official to provide the most 
effective government possible and to provide for continued 
progress and improvement. That is what CitiStat and StateStat 
are really about.
    Mr. Chairman, I want to close today with the words of 
Robert Kennedy, who once said, ``There is no basic 
inconsistency between ideals and realistic possibilities, no 
separation between the deepest desires of the heart and of the 
mind, and the rational application of human effort to human 
problems.'' The rational application, well and timely measured, 
of human effort to human problems.
    That is what performance-based government is about, and I 
thank you so very much for your interest and your leadership in 
bringing this to our Federal Government. Thanks very much.
    Senator Carper. Governor, thank you very much.
    One of my finest memories of being governor was going to 
New Governors School every 2 years, right after the election. 
About the middle of November, current governors, would host the 
newly-elected governors and spouses and teach the new governors 
and spouses by virtue of our own experience the things we had 
done wrong, in many cases, and some of the things we had done 
right.
    We had a Center for Best Practices within the National 
Governors Association. I suspect we still do. I am just 
wondering, does your experience with CitiStat and what you are 
doing with StateStat, does any of that show up within the 
National Governors Association, either at a New Governors 
School like a forum, or through the Center for Best Practices 
so that other States can learn from what you have done, both in 
Baltimore and in Maryland?
    Governor O'Malley. Yes. To some degree, other States have 
begun this in one way, shape, or form, and some of them started 
doing it years ago on the heels of seeing the success of the 
NYPD. In fact, Christine Gregoire, who began her first term as 
Governor of Washington State, visited us, actually before all 
the votes were totally counted in that close race----
    Senator Carper. I said to Christine, who had been Attorney 
General, I said, keep counting the votes and recounting the 
votes until you win. Then stop. [Laughter.]
    That is what they did.
    Governor O'Malley. Well, during the period of pre-
transition, she sent a group of her people to Baltimore and has 
actually begun and does have a performance measurement program 
in Washington State. I know I have had conversations with 
Kathleen Sebelius in Kansas back at the time, and I believe she 
applied it to some things, as well.
    It is, I believe, part of the National Governors 
Association, part of their Office of Best Practices, and we 
were able to recruit from there not only Malcolm Wolf, our 
Energy Administrator with whom you had some discussions, but 
also our policy directors from that area. The best ideas are 
the ones that you find from other practitioners. That is what 
we believe.
    One of the exciting things about beginning this program in 
Baltimore years ago is that other cities have taken it up, and 
I have found myself learning from things that Mayor Cicilline 
in Providence was doing to apply this to juvenile justice, to 
be able to learn from Mayor Slay in St. Louis the things he was 
doing on lead paint abatement and the like.
    So I hope, and I believe, that there will be more of this 
going on at the State level that we can learn from.
    Senator Carper. In Baltimore, I presume this initiative was 
something that you promoted as mayor, and as governor, I 
presume that this is an approach that you have been championing 
in your State.
    Governor O'Malley. Absolutely. One of the things I have 
noticed, Senator, is that newly-elected executives tend to have 
an easier time embracing this than those of us who have been in 
office for some time. The very uncomfortable part of this 
process is that when you begin measuring things that have never 
been measured before and start sharing that information widely 
and broadly, everyone comes to understand how poorly many 
things had been functioning in the past. The fresh start of a 
new executive is, I think, something that enables an 
organization to make that sort of culture shift in ways that 
others find more difficult.
    It has also been my experience that however uncomfortable 
that initial period is of the openness and the transparency, 
which are the hallmarks, I think, of performance measurement 
and a republican form of government, certainly, that openness 
and transparency over time pays tremendous dividends, however 
embarrassing the up-front moments are of, oh my goodness, I 
didn't know that we were that bad at that particular service 
delivery. Over time, people come to respect and appreciate it.
    Another innovation that we borrowed from Mayor Richard 
Daley in Chicago was the use of the 311 center for all city 
services. In the past, we would go around knocking on doors at 
campaign time and giving out the ``where to call for help'' 
card. The ``where to call for help'' card had some 300 various 
services listed alphabetically. Look under V for vacant houses. 
Look under R for rat abatement. And then we would have 300 
different phone numbers.
    The 311 system on the front end and being able to have a 
call courteously answered and give every citizen, regardless of 
what neighborhood that the call came from, being able to give 
them a customer service number that was common to all citizens 
and a time frame within which to expect that service, whether 
it was a broken curb, a pothole, dead tree, or what have you, a 
time frame within which to expect that service to be delivered 
was something that gave our citizens a lot of optimism about 
the future and the fact that they still do, in fact, have the 
reigns of controlling and holding accountable that government 
into which they pay their hard-earned dollars.
    Senator Carper. Well, you are going to have a whole crop of 
new governors being elected this November. A couple of weeks 
after that, they are going to show up at the NGA School for New 
Governors and Spouses and they are going to be looking for 
things, ways to provide good services, working within the 
constraints of tight budgets, as you know. I suspect that you 
can provide a real service for them as you are here today in 
sharing with them your successes in Baltimore and in Maryland.
    Actually, talking about taking good ideas, we used to steal 
the best ideas from one another. Sometimes you would attribute, 
sometimes not. But we have taken the ideas of New Governors 
School and we have actually incorporated it here in Washington. 
In the middle of November, 2 weeks after the election, we will 
have, I call it New Senators School, but it is actually this 
orientation for new Senators and their spouses, 3 days, very 
much like NGA. Faculty are current Senators and spouses, and 
basically learn from one another.
    Governor O'Malley. And I trust that they put former 
governors at the front of the class?
    Senator Carper. Actually, three of the people who started 
it, Lamar Alexander, former governor, former NGA Chair from 
Tennessee, George Voinovich, former governor, former NGA Chair, 
Mark Pryor has very active as Attorney General from the State 
of Arkansas, and yours truly. So three out of the four are all 
governors.
    In Baltimore, you obviously have a city council, 
Legislative Branch. As governor, you have a General Assembly, 
Legislative Branch. Here, we have the Legislative Branch in 
which I am privileged to serve. What role in city, State, and 
maybe in the Federal Government, the National Government, can a 
Legislative Branch play? How is this relevant in the lives of 
the Legislative Branch in the city, State, and in Washington?
    Governor O'Malley. Initially, like many new ideas, it was 
greeted with a tremendous amount of skepticism and----
    Senator Carper. By your legislators?
    Governor O'Malley. Yes. Let me talk first on the city 
council and then I will talk at the State level. In the city 
council, the concern was that this was going to undermine or 
somehow diminish the ability of council members to deliver for 
the constituents who called them for a variety of city 
services, and so over time, we were able to overcome that, 
primarily because of the openness and the transparency of the 
process and also the ability of city council staff to be able 
to log into the same system that our 311 operators were able to 
log into in order to give out those citizen service complaints 
and the time frame within which to expect that those services 
would be delivered.
    And once the council staff were trained in it and they went 
back and worked on their council members, a lot of the fears 
dissipated because everybody wants to be able to produce. 
Everybody wants to know that when their constituent calls, that 
they will be able to pick up the phone and deliver.
    On the State level, we recently created a new fund for the 
restoration and health of the Chesapeake Bay, work that has 
been going on for some time, work that needs to be accelerated 
and improved and--the tendency, I think, in most legislative 
bodies is to specifically designate any new dollars that are 
appropriated for a given purpose, namely the cleanup of the 
Chesapeake Bay. In a big public health challenge like the 
Chesapeake Bay, there are probably at least 100 different 
activities that could be funded that contribute to the health 
of the Chesapeake Bay, everything from stormwater upgrades to 
cover crops to expanding the forested buffer along streams.
    The legislature, because of the openness, because of the 
transparency, and because of the performance measurement that 
they saw in BayStat and their belief, their well-founded hope, 
I think, that the deployment of those dollars will be guided by 
the best science and the most effective use of those dollars, 
they chose after some deliberation not to specifically 
designate this first $2 million shall go to this. This next $3 
million shall go here to that.
    I think that legislators that have seen the beginning of 
this process are encouraged that they will be able to get more 
timely, accurate reports on the things that are working, the 
things that are not, which then will make them much more 
effective in exercising their oversight of the things that we 
do fund, the things that we maybe should fund more of, and the 
things that perhaps we should not be funding as much of.
    Senator Carper. In our State, and certainly in your State 
and the other 48 States, we have spent a lot of time and energy 
and money on raising student achievement. Most of the States 
have established academic standards of what we expect our kids 
to know and be able to do in math, science, English, social 
studies, and other curriculum. We designed these tests to 
measure student performance to try to figure out how to hold 
schools accountable, students accountable, educators 
accountable, even parents accountable. We have created enormous 
amounts of information from the results from our tests. Some 
States give tests on an annual basis to selective grades. Now 
more and more, we are giving at least annual tests to a broader 
cross-section of students and grades of students.
    In my own State, we have a program called Principal for a 
Day. The State Chamber of Commerce does it in conjunction with 
our school districts. We have a little over 200 public schools. 
It is a program I participate in almost every year, so last 
year, I was Principal for a Day at a school called Stanton 
Middle School, about 10 miles outside of the City of 
Wilmington. They had been taken under the wing of Selbyville 
Middle School. Selbyville is in southernmost Delaware, almost 
in Ocean City.
    Selbyville Middle School had figured out how to raise 
student achievement in ways most middle schools have not, and 
the results were dramatic and impressive and sustained, and the 
folks at Stanton had been struggling to figure out, what are we 
doing? What are they doing right? How can we learn from them?
    What they found out is that the folks at Selbyville Middle 
School in Southern Delaware had figured out how to, not to 
teach to the test, but to say these are our standards. We want 
our teachers to teach to the standards. Then we will have a 
test and the test will reflect the standards.
    But the school--Selbyville was able to figure out how to 
use the information gained from the tests to change the way 
they taught, to change the way they allocate resources, to 
change the way they work with individual students. Stanton 
Middle School last year--in fact, this year, was selected as 
one of the best, most improved schools in our State, and what 
they learned to do was to take the information and to use that 
to change behavior, not just of the students, but of the 
educators and the administrators, as well.
    Have you had the opportunity in your State to take this 
approach of performance information and using that to manage 
for better results and to put it to work in your schools?
    Governor O'Malley. To a degree. We attempted with some 
success to do this in the City of Baltimore and our schools 
there and it was embraced to a degree, especially when it came 
to some of the management functions of the school district. The 
school district in our State that has embraced the notion of 
performance management, performance measurement most 
wholeheartedly is Montgomery County under Dr. Weast, and I have 
had occasion to observe his SchoolStat meetings where he brings 
in principals, and oftentimes it is, as you would suggest, a 
principal from one of the higher-performing schools and a 
principal from one of the more struggling schools, and just as 
governors and mayors learn from one another, principals learn 
from one another the things they are doing and the things that 
work.
    Montgomery County has done it the best and the most 
wholeheartedly. Other school districts, I hope will follow on. 
I was reminded in listening to your story about the two school 
districts in Delaware, one of the leading governors in America 
on education reform is Mike Easley of North Carolina----
    Senator Carper. And he followed another great governor, 
Governor Hunt.
    Governor O'Malley. A good man.
    Senator Carper. Oh, a terrific education governor.
    Governor O'Malley. I once heard him say, and it is true of 
all performance measurement, not just in schools but whether 
you talk about the potholes or the boarding or the other 
environmental things, he said, ``You are not going to improve 
the weight of the pig just by constantly weighing it.''
    Applying the standards and the performance measurement is 
important. Setting the goals is important. But along with that 
go the other tenets of the rapid deployment of resources, 
having real serious dialogue, conversation, and some 
experimentation to determine which tactics are effective, which 
strategies are effective, and having the guts to jettison some 
and embrace others, and doing so in a relentless follow-up way.
    There is always a temptation that if we wave the magic wand 
and simply weigh the pig more constantly, that somehow that is 
going to improve the weight of the pig, but the inputs matter. 
How you manage those inputs, where you deploy those inputs do 
matter. But the only way to determine that is by being wide 
awake and aware and having that dashboard that measures the 
performance over time----
    Senator Carper. All right. Thank you.
    I think there is a great opportunity in my State to build 
on what has happened at the two schools that I have mentioned. 
There are others, as well. But we are thinking about changing 
the way we actually administer our tests. Instead of being a 
test that is given out, pen and pencil test given once a year 
to a third grade, to a fourth grade, to a fifth grade, to have 
a computer-assisted test so the students could actually do the 
test from a computer. They would get the results almost 
immediately.
    And the test is something that if it is happening in the 
third grade and they are performing at a second or a first 
grade level, or a fourth or fifth grade level, that would be 
garnered and it would be a better tool to help the teachers and 
educators in that school to work with an individual student, 
and it is a test that could be given not just once a year, it 
could be given several times during the course of a year, and 
as you watch somebody who is an under-performer move from high 
first grade performance to low second grade, to middle second 
grade toward lower third grade during the course of a year, you 
actually see that progress.
    So we think about changing No Child Left Behind so there is 
more--it is there not to discredit or embarrass school 
districts, but to help and to really be a way to help move 
student performance. There is real potential here for applying 
what you have done in your State and city, I think.
    Governor O'Malley. May I say a word about the testing and 
the inputs? One of the real benefits that we have seen from the 
standardized tests over the last 7 years is the positive 
outcomes in achievement that followed almost immediately on the 
heels of a tremendous investment in full-day kindergarten for 
all children throughout our State.
    When I was first elected Mayor of Baltimore, not one of our 
grades scored majority proficient city-wide in reading or math, 
not one. Within the very first year after the first cohort of 
children in our city, which has a lot of challenges with 
poverty and adult literacy and other things, we saw the first 
graders score a majority proficient in reading and math, and 
some said, well, that was just a blip and once they are longer 
in the system, they will go down.
    Well, quite to the contrary. As we look over our shoulders 
now from that point in time, that cohort has not only 
maintained its level of achievement, but every class and cohort 
of kids after them have come up to that rising level and it 
showed us that the investment that we made--we went from about, 
I think, $3.2 billion as a State to roughly $5 billion in State 
aid to local education, a big part of which was that full-day 
kindergarten, and we have seen that investment has brought 
about these results, including a narrowing of the achievement 
gap between minority--so-called minority students and non-
minority students.
    High expectations are very important and I do believe that 
expectations become behavior if together we make the right 
choices to make it so. But the testing alone, I think the 
biggest part of the improvement of test scores in our State and 
certainly in our city was a result of the big investment, not 
necessarily a result of more tests.
    Senator Carper. By having the testing results, when the 
testing results--actually, you trying to find out why are these 
kids doing better----
    Governor O'Malley. Right.
    Senator Carper [continuing]. In a particular school 
district, and then you look at the number of kids that maybe 
have the opportunity for full-day kindergarten, or you look at 
the kids in the school districts where they actually have an 
opportunity to get decent pre-K training, you are right. In my 
State and your State and across the country, one of the best 
indicators of better performance is quality----
    Governor O'Malley. We are also about to borrow another idea 
from Governor Easley, the teacher surveys and learning from 
what the teachers are telling us----
    Senator Carper. Excellent.
    Governor O'Malley [continuing]. Almost in a sort of 
corporate 360 approach.
    Senator Carper. That is great. A couple questions and we 
will let you escape. Let me just ask, how do you, as chief 
executive of your State, convey your own personal commitment to 
addressing specific performance issues to the agency staff, the 
folks that are responsible for bringing about improvement? How 
do you do that?
    Governor O'Malley. We do it through--you mean actual means 
of keeping engaged as the executive in this process, Senator?
    Senator Carper. Is this something you talk about in your 
State of the Union? Is it something that is reflected in your 
budget? How do you convey to the people that work in your 
agencies, not just the cabinet secretaries, not just the 
division directors, but also to rank-and-file employees why 
this is important, why you think it is working, why they should 
do it?
    Governor O'Malley. The relentlessness of our meetings--I 
was in a meeting today, for example, with the folks from our 
Public Safety Department, and in our State, Public Safety 
includes not only the prisons, but also parole and probation 
and all the agents that go into the sort of community 
supervision of people on the street. And at that meeting, there 
were probably some 30 or 40 people present. Now, not everybody 
talks at the meetings, but we like to think everybody listens 
at the meetings.
    The fact that at the meetings we have not only an executive 
summary that is distributed to all parties beforehand, but we 
also have minutes that are taken and sort of an after-action 
report as to the things that we need to prepare for by the next 
meeting, the questions that need to be answered by the next 
meeting. That is sort of our method, and the fact that we are 
relentless about it and that it is not subject to my schedule. 
I don't go to all of them, but they always happen. And the fact 
that they are relentless and that the follow-up always happens, 
and fortunately, I have dedicated people like Mr. Gallagher and 
Michael Enright that continue to drive that train regardless of 
whether I am pulled out of town or to important committee 
hearings in Washington.
    Senator Carper. I understand. Well, good. How important do 
you think it is to recognize good performance when it does 
occur?
    Governor O'Malley. Oh, critically important. A lot of times 
when people write about this and sometimes when journalists 
come to observe a session, they focus in on how uncomfortable 
these sessions are for the under-performers. The more 
significant change, I think, comes from the executive and 
leadership recognizing leadership at the department level, at 
the implementor level.
    Jack Maple had this theory of progress in a big 
organization. He said 80 percent of the people are in the 
middle, and then on one side are the leaders, and on the other 
10 percent of the bell curve are the slackers. And if 
leadership and the executive recognize the leaders and the 
achievers, the whole organization tilts towards the leaders and 
the achievers. It improves morale. It improves outputs. It 
improves progress. It improves effectiveness. And that is what 
we try to do.
    So in our city, the methods we used for such things were, I 
don't know, something as simple as a thank you note when the 
crews hit the 48-hour pothole guarantee. It seems like a little 
thing. It was probably the first time they were ever able to go 
home to their spouses and say, ``Hey, look what the Mayor sent 
to me.'' Sometimes we would give tickets to Oriole games or the 
Mayor's Box at the Ravens game or other things, and we would do 
it in front of people at these events.
    And the people that are the leaders at one level of 
government were also the people that we promote in the course 
of time that hopefully imbues a sense of meritocracy to 
governments that quite honestly in the past, decisions are 
often made for reasons having little to do with the level of 
achievement, the leadership, or the industry or hard work of 
the individuals.
    Senator Carper. You mentioned the Ravens games. The top 
draft pick for the Ravens at the NFL draft this year, they 
picked a quarterback from the University of Delaware, Joe 
Flacco. In Delaware, people tend to follow the Ravens or we 
tend to follow the Eagles. We will see how it turns out.
    One last thing. Six months from, it will be January 20. I 
believe that is the day we swear in a new President and Vice 
President. The Congress will be sworn in on January 3. So we 
are going to have in the space of a couple of weeks 435 new 
members, just sworn, newly minted House members and about a 
third of the Senate will be newly sworn in.
    If you were going to give that new President and new Vice 
President, and maybe even the folks who serve in the 
Legislative Branch, just some friendly advice on how we can 
take your experience in Baltimore and your experience in 
Maryland and apply it to provide better service, better 
results, and some things that are important for all of our 
States and for our country, what advice might that be?
    Governor O'Malley. I think the most important advice is 
that beyond the guts and the courage that it takes to set goals 
and beyond the trust in the public that it takes to create a 
system that is open and transparent and performance measured, 
the third really important part of this sort of performance 
management is executive commitment.
    This cannot be something that an executive at any level--
municipal, county, State, national--does for the sake of a one-
day press conference or even for the sake of rolling into a 
budget address or a State of the Union. It requires a 
tremendous amount of executive commitment and relentless 
follow-up by that man or woman that is in charge of that 
government, because if there is not the commitment at the 
executive, it is very hard for it to transform that pyramid of 
command and control that is a big Federal Government.
    So those are the three things, Mr. Chairman. Have the guts 
to set goals. Have enough faith in the people we serve to 
create a system that is open, transparent, and measured for 
results. And third, stay committed as an executive to driving 
it every day and to never waver from that commitment to 
openness, transparency, and performance measurement.
    Senator Carper. Good. Among the new governors who will be 
elected this year will be the Governor of Delaware, my State 
and your neighbor. The thought of coming in with a new chief 
executive, and as good as we think we are, we always know we 
can do better. But my hope is that our new governor will reach 
out to you and to see what he can learn from the experiences 
that you have had and the progress that you have made in your 
own city and your own State.
    One of the things that we are doing--we talked about this 
last week and again today--one of the things that we are trying 
to do in Delaware is endeavor to become the first State to 
deploy an offshore wind farm about 12 miles off the coast of 
Rehoboth Beach, as you know. Our hope is that we can find some 
partners to our south and to our west and maybe to our east and 
to our north who might like to partner with us. We talked a 
little bit about this, but my hope is that you and that 
Maryland might like to be a partner, at least explore that 
possibility with us.
    Governor O'Malley. Well, we are very interested in that. We 
have the renewable portfolio standard that was just updated, 
upgraded, increased by our legislature in, I believe, the most 
recent General Assembly session. We have a number of entities 
that are working on renewable sources and hats off to Delaware 
for leading the way on that offshore wind. I hope once we work 
through our request for proposals process, we will be able to 
join in that. We still have a long way to go, but I think it 
makes a whole lot of sense.
    It is certainly needed as we move ourselves into more 
sustainable energy future, and we do so as a region. In fact, 
that might be the only way that this is able to be 
accomplished, or at least that could well be the leading edge 
of that progress, is when States join together and create the 
critical mass necessary to prime new endeavors like the wind 
project off Delaware's coast.
    Senator Carper. A couple of years from now, people will 
look at Rehoboth Beach on a clear day like today, and if they 
can look out 12 miles to the east, they will see sticking above 
the horizon some small objects about the size of a thumbnail 
that will be 60 or 70 windmills that collectively will provide 
about 15 percent of our electricity needs in Delaware. We 
believe, and I think you do, too, that to the extent that we 
can have more turbines and look for economies of scale, it is 
not just advantageous to us, but to our neighbors, as well. So 
I am encouraged with what you have said.
    Governor O'Malley. I am very interested.
    Senator Carper. Last, going back to football again for just 
a moment, I will close with this.
    Governor O'Malley. I reserve the right to turn to staff. 
[Laughter.]
    Senator Carper. Vince Lombardi, that legendary head coach 
of the Green Bay Packers, used to say that unless you are 
keeping score, you are just practicing. And it occurs to me 
that they didn't just do more than keep score at Green Bay all 
those years that they played so well, but what they did is they 
kept not just score, but they kept the results from how each of 
the players did, the plays that worked, the plays that didn't, 
and then they used that to change their game plans and really 
to change their defense and their offense. They actually 
applied the information that they collected.
    We are pretty good in government in keeping score, using 
the football analogy, how are we using the score that we are 
keeping, how are we using those scores and the information that 
we gather to actually change the plays that we call at the line 
of scrimmage, the plays that we call in the huddle, the way we 
deploy our defense or our offense, the kind of players that we 
draft? It actually does apply. It applies in the world of 
football and it applies very much in city government, State 
government, and I think in the Federal Government.
    You have been just a stand-out on this front, and I know 
the people of Baltimore have recognized that and rewarded you 
for that and the people of Maryland have done so, as well. 
Thank you so much for being here and helping to show us the way 
as a Nation, and we look forward to hearing our second and 
third panels, as well, but you have gotten us off on the right 
start.
    Governor O'Malley. Mr. Chairman, thank you very much.
    Senator Carper. Thank you.
    Governor O'Malley. To the extent that we have been 
successful, it is because of the people we serve and the good 
people that I have been blessed to be able to work with in 
public service. Thank you very much.
    Senator Carper. Thank you.
    Governor O'Malley. Thanks for your leadership.
    [Pause.]
    Senator Carper. Before I introduce our witnesses on our 
second panel, I want to take just a moment and thank all of our 
witnesses, from certainly the governor and this panel and our 
third panel, as well, for doing something that doesn't happen 
every day, and that is for providing all of your testimony on 
time. You are to be commended for doing that and we are 
grateful. It is a notable achievement.
    Our three witnesses today, I will start off by providing 
just a little bit of background. Bernice Steinhardt is Director 
for Strategic Issues at the Government Accountability Office, 
where she is responsible for examining government-wide 
management issues. For over 9 years, she has led the GAO's 
efforts in strategic planning and helped to develop the 
organization's first strategic plan. She has held a number of 
positions at GAO, including Director of Public Health Issues 
Group and Associate Director for Energy, Natural Resources, and 
Science Issues, as well as Environmental Protection Issues.
    Before joining GAO, she served at the Department of the 
Interior at the President's Council on Environmental Quality. 
Who was the President at that time when you were on the Council 
on Environmental Quality?
    Ms. Steinhardt. The first time I was there, the Chairman 
was Russ Peterson and the President was Gerald Ford.
    Senator Carper. You were just a child. Were you an intern 
then?
    Ms. Steinhardt. You are very kind.
    Senator Carper. Russ Peterson is still alive and well.
    Ms. Steinhardt. Is he?
    Senator Carper. He is 91 years old and he was a Republican 
then, now he is a Democrat, but I think he was always a 
Democrat if you want to know the truth.
    Ms. Steinhardt. Well, he was an environmentalist, I can say 
that.
    Senator Carper. He is a strong environmentalist. But he is 
still just doing incredible things at the age of 91. What an 
inspiration.
    Ms. Steinhardt. He was a wonderful Chairman.
    Senator Carper. He is one of my mentors.
    Ms. Steinhardt. Mine, too.
    Senator Carper. Good. Thank you.
    Marcus Peacock is the Deputy Administrator of the U.S. 
Environmental Protection Agency, where he currently is focused 
on improving EPA management systems. Previously, he served as 
an Associate Director in the U.S. Office of Management and 
Budget, where he was responsible for making budget decisions 
encompassing $160 billion in spending at various Federal 
agencies. While at OMB, he created the Performance Assessment 
Rating Tool (PART), which is used to rate the effectiveness of 
Federal programs.
    We want to thank Mr. Peacock for really filling two roles, 
wearing two hats here today, one speaking on behalf of EPA and 
also providing some insight into his former work at OMB. We had 
invited Clay Johnson of OMB to be with us today, but 
unfortunately, he is out of the country and OMB has kindly 
provided testimony for the record from Robert Shea and we look 
forward to including it as a part of our hearing today.\1\
---------------------------------------------------------------------------
    \1\ The prepared statement of Mr. Shea appears in the Appendix on 
page 131.
---------------------------------------------------------------------------
    We are really grateful that you are here, Mr. Peacock.
    Dr. Don Kettl is Director of the Fels Institute of 
Government and the Robert A. Fox Leadership Professor at the 
University of Pennsylvania, where he specializes in management 
of public organizations. We consider him a neighbor, too, 
because he lives only about 30 miles up the road from us in 
Philadelphia.
    In 2008, he was awarded the John Gaus Award for a lifetime 
of exemplary scholarship in political science and public 
administration. He is the author or editor of a dozen or so 
books and has won twice the Louis Brownlow Book Award for the 
best book published in public administration. His newest book, 
due out in November, is entitled ``The Next Government of the 
United States: Why Our Institutions Fail Us and How to Fix 
Them.'' He has consulted for governments at all levels, both 
here in the United States and abroad. Dr. Kettl is currently a 
nonresident Senior Fellow at The Brookings Institute and is a 
regular columnist for Governing Magazine.
    We are delighted that all of you are here. Your entire 
statement will be made part of our record. You are welcome to 
summarize that, and I would ask you to stick close to 5 
minutes, but we won't quarrel if you go a little bit beyond 
that.
    Ms. Steinhardt, I would welcome your leading this parade. 
Thank you so much for coming.

  TESTIMONY OF BERNICE STEINHARDT,\2\ DIRECTOR FOR STRATEGIC 
         ISSUES, U.S. GOVERNMENT ACCOUNTABILITY OFFICE

    Ms. Steinhardt. Thanks very much for having me. I am 
pleased, in turn, to be here today, especially in such 
distinguished company, and to share with you some of the 
lessons that we have learned in examining the progress that the 
Federal Government has made in managing for results.
---------------------------------------------------------------------------
    \2\ The prepared statement of Ms. Steinhardt appears in the 
Appendix on page 55.
---------------------------------------------------------------------------
    Starting at least 15 years ago with the Government 
Performance and Results Act, the Congress and each 
Administration have put into place a framework of strategic 
plans, performance measures, and reports that together have 
heightened focus on performance and accountability for results. 
So today, I would like to talk about where we now stand and to 
suggest some actions that the next Administration can take to 
sustain and build on what we have accomplished so far.
    Since 1997, GAO has conducted four surveys of Federal 
managers across the government in order to gauge the extent to 
which a performance culture has taken hold. Clearly, there has 
been progress compared to what we saw in our first survey, and 
I would call your attention to the chart up top there.\3\ 
Significantly more Federal managers that we surveyed in 2007 
reported having performance measures for their programs, which 
is to say the basic information that they need to judge how 
well their programs are working. The blue bars are the 2007 
results and the white bars, which you can see outlined there, 
are for 1997. So there is quite a bit of difference.
---------------------------------------------------------------------------
    \3\ Chart referred to appears in the Appendix on page 137.
---------------------------------------------------------------------------
    But in order for this information to make a difference, it 
has to be useful and used. To borrow Governor Easley's term, 
this is about weighing the pig. How are we making changes in 
the pig's weight? So the second chart shows that there are 
still large percentages of managers who aren't using 
performance information to a great extent in making decisions, 
like setting priorities or allocating resources.\1\
---------------------------------------------------------------------------
    \1\ Chart referred to appears in the Appendix on page 137.
---------------------------------------------------------------------------
    In fact, there has been significant change in only two of 
these nine areas, the ones that are highlighted there. One is 
an increase--and this is a good thing--in the use of 
performance information to reward employees. But at the same 
time, we also see a decrease in the use of performance 
information for making decisions about new program approaches 
or work processes, which does give us some concern, this 
decline, because this is where you would want to see managers 
using information to change direction when performance is not 
up to the standards.
    From our earlier work, and these latest survey results 
confirm this, we know that there are a number of practices that 
could spur some movement in these relatively static trend lines 
and lead to more widespread use of performance information. 
First and perhaps most importantly, and Governor O'Malley, I 
thought, underscored this very clearly, the agency leadership 
needs to demonstrate its commitment to achieving results. If 
the leaders don't care or if they don't show they care, then 
managers are not going to pay much attention, either. In fact, 
our survey results only underscore this report because those 
managers who reported using performance information to a 
greater extent also reported that their leadership showed a 
commitment to using the results of that information.
    Second, agencies need to link organizational results to 
individual performance. That is a kind of line of sight from 
the organization's results to individual performance. And here, 
performance management systems can be used to let employees 
know how they can contribute to results and then to hold them 
accountable for doing so.
    Third, agencies need to make sure that they have the 
capacity to collect and to use performance information. 
Managers aren't going to use the information if it is not 
timely, if it is not relevant, if it is not accurate, if it is 
difficult to use, or if they simply don't know how to use it.
    Promoting these practices within agencies ought to be a 
first step for the next Administration. Beyond this, though, we 
would advise a few others, again based on insights that we have 
gained from our work over the years.
    First, we would urge a more strategic and cross-cutting 
approach to overseeing performance across government. Many of 
the challenges that government faces today, whether it is 
homeland security, emergency response, climate change, all of 
them involve multiple agencies and programs. While this 
Administration's major performance improvement initiative, the 
PART process, has brought heightened attention to the 
performance of individual programs, we also need an approach 
that encompasses multiple programs and provides a more 
integrated view of what government is accomplishing.
    Second, performance information has to be more useful to 
the Congress. Whatever performance initiatives the next 
Administration adopts, the Congress should be engaged in 
helping to identify meaningful measures of success as well as 
the form in which the performance information will be useful to 
Congress in its oversight, legislative, and appropriations 
role, each of which could require different types of 
performance information.
    Finally, OMB can do more to build agency confidence in its 
performance assessments. OMB deserves full credit for its 
leadership in fostering a performance culture across 
government, but as our survey data suggests, many Federal 
managers still question the quality of the PART assessments. 
The more confidence that managers have in the quality of the 
assessments, the less they perceive them as paperwork 
exercises, the more likely they are to use the results.
    I want to close by noting that our study for you that is 
continuing, and we hope to be reporting back here early next 
year on additional actions that some agencies can take to 
increase their use of performance information. But in the 
meantime, we welcome the opportunity to work with you and your 
staff, who have been very helpful already in organizing this 
hearing, and we hope we can continue to work with you on these 
critical issues.
    With that, I will conclude my remarks and look forward to 
your questions. Thank you.
    Senator Carper. Thank you so much for your coming. Thank 
you very much to you and your colleagues at GAO for your work 
in this arena and for helping us as we approach the change in 
administrations and the new Congress. Thank you.
    Ms. Steinhardt. Our pleasure.
    Senator Carper. Mr. Peacock, you have done good work, both 
at OMB and clearly at EPA on this front, in this and other 
endeavors, too. We thank you for that and we welcome your 
testimony today. Thanks for joining us.

 TESTIMONY OF MARCUS C. PEACOCK,\1\ DEPUTY ADMINISTRATOR, U.S. 
 ENVIRONMENTAL PROTECTION AGENCY; AND FORMER OFFICIAL AT U.S. 
                OFFICE OF MANAGEMENT AND BUDGET

    Mr. Peacock. Well, thank you, Mr. Chairman. Thank you for 
having this hearing. As you have noted, my name is Marcus 
Peacock and I serve as the Deputy Administrator at the 
Environmental Protection Agency. I do sincerely thank the 
Subcommittee for having this hearing. Congress has lots of 
hearings every year, but they tend to focus on how a particular 
government program can do a particular job better. I think this 
Subcommittee understands that you can come up with similar 
methods for not just improving one, two, or three Federal 
programs, but for improving all Federal programs.
---------------------------------------------------------------------------
    \1\ The prepared statement of Mr. Peacock appears in the Appendix 
on page 84.
---------------------------------------------------------------------------
    The President certainly gets that idea. That is why in 
2001, he directed that agencies integrate performance 
information into their budgeting decisions. This meant agencies 
had to identify performance measures for each Federal program 
and factor that when they did their budgets, the ultimate goal 
was to improve results across the government, and at that time 
the President directed me to lead this cross-agency effort.
    The result of the initial work we did was the Program 
Assessment Rating Tool (PART). PART is essentially a method for 
assessing the effectiveness of Federal programs, and more 
importantly, making recommendations regarding how those 
programs can be improved. And since its inception, the PART has 
won a number of awards. It has been copied by other 
governments.
    I am often given credit for creation of the PART, but I 
will tell you, I was very fortunate to have three things on my 
side. The first was support from the top, and that was from the 
President. The second was to have a very talented team of 
people at OMB, the Office of Management and Budget, who did the 
hard work of creating and fleshing out the PART. And finally, 
third, we had a very talented and smart advisory committee, 
which included Dr. Kettl, who kept looking at our work and 
reviewing it.
    In 2005, I moved from OMB to run the day-to-day operations 
of EPA, and in the last 3 years, I have spent a large amount of 
my time improving EPA's performance management systems. A 
number of my predecessors, including Al Alm and Hank Habicht 
also spent--emphasized improving management systems, and 
hundreds of EPA staff over the last 20 years have spent time on 
this effort. So I know better than anyone else that I really 
stand on their shoulders.
    EPA has come a long way. As the governor mentioned, we just 
rolled out EPAStat. It is the first Federal agency to have a 
Stat program. But I would say the biggest problem EPA currently 
faces is to make sure our performance measures are actually 
used. You can treat a measurement system like a thermometer or 
a thermostat, and, of course, a thermometer measures the 
temperatures, but a thermostat not only measures the 
temperature, but allows you a way of changing it. And 
performance management systems should be thermostats, not 
thermometers. They should be used to produce change so that we 
become more effective at serving the public.
    Metrics for reporting don't mean much, but metrics for 
management, I think, are vital. There are five barriers that I 
have found in trying to get performance information used more, 
and the first is a lack of fresh and frequent data. The Federal 
Government has lots of annual measures, but annual measures 
don't work too well in running the day-to-day operations of an 
organization or program.
    There is this Enterprise Rent a Car commercial you may be 
familiar with where the car is all wrapped up in brown paper 
and I have always wondered how that car stays on the road. You 
can't look out the windshield or the windows. And I think 
trying to use annual performance measures to manage the day-to-
day operations of a program is similar to trying to drive that 
car around. With annual measures, you can't really see where 
you are going.
    The second barrier to using performance measures that I 
have run into is a focus on money and not results. Washington, 
DC still operates in a culture that asks how much did a program 
get rather than what did the program produce, and that is not a 
healthy way, I think, of looking at things. Just think about 
the incentives inherent in declaring an organization more 
successful simply by its spending more money.
    A third barrier is too many meaningless measures. What 
matters gets measured, but if everything is being measured, it 
is hard to tell what matters. So measures need to be meaningful 
and you need to be able to objectively evaluate where the 
measures came from.
    Fourth, too little access. The fewer people that have 
access to performance measures, the fewer people are able to 
actually use them. That is why I believe performance 
information should be made available to the widest possible 
audience.
    And then finally, the fifth barrier is resistance. Many 
people naturally worry about the consequences of not meeting 
performance targets. They think performance data may be used as 
a cover to either cut funding or punish people. And the fact is 
that these systems can be abused, but that is a very poor 
reason to avoid them. An organization cannot become excellent 
without measuring its performance. If you can't see what you 
are doing right, then you won't learn from that.
    In conclusion, I am fortunate to work at a place like EPA. 
The employees love the mission of the agency. They are very 
results-driven, so that if you can show them that you are going 
to be able to improve the results, they are willing to change 
what they are doing in order to get there. They understand that 
when EPA works better, public health and the environment 
improve faster. And EPA management initiatives that aren't 
linked to results are just gobbledy-gook. You need to 
demonstrate that they will lead to cleaner air, water, and 
land.
    We need to get to the point where all Federal agencies 
practice good performance management. Whether the latest and 
hottest issue is homeland security or securities regulation or 
climate change, a functioning performance management system is 
invaluable to helping any Federal agency, no matter what its 
work, do its job better. We need to get to the point where 
there is always an answer to any Federal employee who asks the 
question, ``How can I do my job better?''
    Thank you.
    Senator Carper. Thank you very much. I want us to come back 
during the questions and answers. Mr. Peacock has just run 
through five reasons why we don't always use information to 
help us improve our performance and I am going to ask our other 
witnesses to comment on those, if you would.
    Dr. Kettl, you were good enough to spend some time with 
Wendy Anderson and myself in our office earlier this week. We 
are grateful for that and for your years of work in this area 
and for all that you have written and lectured. We appreciate 
your being here today and helping us to do better, to do our 
job better. Thank you. You are recognized.

 TESTIMONY OF DONALD F. KETTL,\1\ DIRECTOR, FELS INSTITUTE OF 
    GOVERNMENT, AND ROBERT A. FOX PROFESSOR OF LEADERSHIP, 
                   UNIVERSITY OF PENNSYLVANIA

    Mr. Kettl. Thank you, Mr. Chairman. It is a great pleasure 
to be here today. I want to thank the Subcommittee for this 
opportunity to explore what in many ways is the most important 
issue facing this government, which is how to make sure that it 
works, how to make sure it delivers results and value to 
citizens. It may very well be the most important question we 
never stop to ask until it is too late, and smart government 
means getting out ahead of that to make sure that we are 
prepared for the challenges that we know that we need to find a 
way to be able to master.
---------------------------------------------------------------------------
    \1\ The prepared statement of Mr. Kettl appears in the Appendix on 
page 86.
---------------------------------------------------------------------------
    I want to make several points here today as we try to 
explore that question, the first of which is that if you look 
from a 30,000-foot level at the question of government reform, 
we are, in all likelihood, at the end of a natural life cycle 
of Presidential management reforms. For the first time since 
the Eisenhower Administration, we are now in a place where the 
next thing that the Federal Government might do, the next thing 
a new President might do is obvious.
    President Eisenhower had the Hoover Commission. President 
Kennedy had his whiz kids. President Johnson had planning, 
programming, budgeting systems and budget reform. President 
Nixon had management by objectives. President Carter brought in 
zero-based budgeting from Georgia. The Reagan Administration 
brought in a whole wide privatization initiative. At this point 
in the campaign back in 1992, it was very clear that President 
Clinton was going to reinvent government based on the best 
seller. And by this point in the campaign in 2000, President 
Bush had already delivered a major speech on the management of 
the Federal Government.
    It is not clear what it is that is going to happen next, 
but it is clear that what we have done so far has accomplished 
a lot, but that we are now at the point where we are going to 
need to do something different, that more of the same kinds of 
reforms are not likely to get us to where we need to go.
    So the first point is that we are at a crucial turning 
point because we are at the end of a series of natural life 
cycles in the reform movement.
    The second thing is that we have to find a way to get very 
smart very fast about what kind of reform we need because we 
face a set of wicked problems that will prove especially 
punishing if we do not find a way to get ahead of these 
problems. We face problems whether we are talking about 
tomatoes in the supermarkets or hurricanes like Katrina that 
come up very quickly, that are very large, that provide little 
time to react, and that impose large consequences for failure.
    And so we are at a point where we need to find a way to do 
smart things fast to be able to deal with these issues that we 
know we are going to be facing, and we know that more of what 
we have been doing is simply not going to do the job.
    The third point is I want to look in particular at what the 
reforms in the Clinton and Bush Administrations have produced. 
Both of these are tremendously importantly initiatives that in 
many ways, although they may not seem like it from a distance, 
are part of the same broad fabric of looking at results and 
trying to motivate managers to do better.
    The Clinton Administration brought a strong focus on 
results for the first time to the entire Federal Government. It 
encouraged Federal managers to innovate. It brought an expanded 
customer service initiative to the government, ranging from 
Social Security to the IRS, in ways that were truly 
breathtaking and important. But at the same time, one of the 
things the Administration did was to engage in substantial 
downsizing of the workforce without engaging in a right-sizing 
of what that workforce ought to look like.
    As a result of that, the Government Accountability Office 
has now named human resource management one of the high-risk 
areas that the Federal Government faces, the issues that are 
most likely to cause the biggest problems in management, and 
GAO has identified this and in many ways it is the consequence 
of trying to figure out now what kind of government do we need. 
We clearly can't downsize any further without thinking about 
how to right-size what it is that we do, and that was a major 
question that the Clinton Administration left unanswered.
    In the Bush Administration, the PART process we have heard 
discussed here this afternoon was a tremendous breakthrough. 
For the first time, all Federal agencies had to try to assess 
goals for all their programs. They had to measure what it is 
that they were trying to accomplish and focus on activities, 
and it was government-wide in the scale that was anticipated in 
some ways by the Clinton Administration but never fully 
realized. And two other things happened tied to the Office of 
Management and Budget and tied to the budgetary process. Both 
of these were tremendously important and major breakthroughs.
    The piece that we have left unanswered here is trying to 
find ways of separating out the ideological commitments to some 
parts of the management agenda, for instance, increasing 
contracting out, from the question of what it is that we want 
the contracting out process and other management processes to 
accomplish, focusing on these goals and objectives, and 
ensuring that we have a focus on activities that cut across 
government agencies.
    The basic fact, for better or worse, is that there is no 
program that can adequately encompass any problem that matters. 
Or put differently, there is no problem that matters to anybody 
that any one Federal agency, any one level of government, any 
one unit can possibly control. That was a lesson from Hurricane 
Katrina and it strikes through every single important thing 
that the Federal Government does.
    So the problem is trying to figure out how to get leverage 
over these long and complex chains that typically require 
action across many agencies, and the piece that lies next after 
the President's management agenda in this Administration is 
finding a way to get to that cross-cutting piece. What the PART 
process was terrific at was the vertical integration of 
knowledge. The next step has to be the horizontal management 
coordination to try to ensure that kind of collaboration.
    What we need, in short, is my fourth point, actionable 
intelligence, trying to find ways of finding out the things 
that we can do something with, and once we figure out what it 
is that we need to know, getting action to happen. We need 
people--this is a lesson from the Clinton Administration--we 
need Federal managers who can lead. The difficulty is that we 
have a lot of very smart people who are doing very good things, 
but often out there feeling that they need to work against the 
odds. And what we have to do is find a way to help them to do 
what it is that they know they need to know and know what they 
need to do in a way that makes it easy.
    We need information that focuses government on results 
instead of collecting information within silos. We need to 
drive that results to try to improve the way the government 
works instead of simply producing, as we have been discussing 
here this afternoon, more weights on the pig without making the 
pig any better.
    We need, finally, a focus on collaboration that builds 
cooperation along a horizontal chain against all those people, 
all those organizations, all those agencies, to be part of a 
solution to ensure that, in the end, the problem gets solved 
instead of just the agency gets managed.
    We, in short, need to try to find ways of developing a new 
generation of reforms. More of what we have been doing is 
likely to produce diminishing returns because, as GAO is 
pointing out, we are collecting more and more information, but 
it is not like people are doing anything much with it. What we 
need to do is figure out how to make sure that the information 
that we have is information that improves the way that 
government works.
    We need results that matter to people, which gets to my 
fifth point about trying to figure out how to make that work. 
First, we need to talk about what it is that we are trying to 
do. Often, a focus on results within individual agencies has to 
do with how many checks have been mailed, how many grants have 
been processed, how many forms have been filed, but in the end, 
none of that matters unless the problems get solved.
    The fact is that most citizens don't care about what it is 
that the FAA does, but what they want is when they go to the 
airport, they want to be able to take off to where they are 
going and land at the destination safely. They don't care about 
the FDA, but they would like to be able to eat tomatoes safely 
without having to worry about it. Citizens care about results 
and we need a government that is driven toward those results.
    The second point within this, though, is that the chain of 
producing accountability toward those outcomes, which is what 
people care most about, is often long and complex, where 
individual agencies, individual units contribute pieces, but 
nobody controls the whole thing. That means we need to find a 
way to hold each individual player accountable for their 
contribution to the outcome as a whole. We need to know what we 
are trying to accomplish and understanding what each 
individual's contribution to that is.
    This means ultimately, as my third point here, that the 
government needs to get leverage over the outcomes that it 
wants to try to accomplish. One of the great findings and 
discoveries about EPA is that EPA is most effective not when it 
does it itself, but when it enlists its partners toward the 
common pursuit of making the environment clean.
    What is it this ought to look like in the next 
Administration? For the President, there are several things 
that could happen. The first is a new generation of reforms 
that understands the importance of what has happened before but 
the need primarily first to focus on outcomes and results that 
matter.
    Second, a kind of geographic information and performance 
stat for those parts of the Federal Government that make sense 
so that we can look at what it is that matters and measure in 
real time today how well we are doing it because if we wait 
until the end of the year for a GPRA report to produce some 
kind of results, by that point, it will be too late to change 
what it is that needs to be fixed.
    We need an Office of Personnel Management that is more 
robust and develops leadership in the Federal Government, and 
we need a focus in the White House in particular to have 
somebody in the White House who can focus on the issues of 
making the government work and making that important for the 
President.
    For Congress, we need a system of the Government 
Performance and Results Act that focuses on results that 
matter, hearings that on every occasion ask people what is it 
that we are trying to accomplish and how well are we doing it, 
and a focus on programs and agencies that put outcomes that 
matter for people at the forefront of things.
    We have plenty of information. We need to make it 
actionable and find ways of producing results for the citizens 
in the end, because that is what it is that the people expect.
    Senator Carper. Excellent. Thank you very much for really 
three terrific testimonies.
    I want to go back to, as I said I would, to Mr. Peacock. 
You mentioned five barriers to agencies taking this approach 
and improving their outcomes. Would you briefly mention those 
again, and then I am going to ask our other two witnesses to 
comment on them.
    Mr. Peacock. Sure. The first one is lack of fresh and 
frequent data. The second is a focus on money, not results, and 
you could broaden that to inputs, not results or outcomes. 
Third is too many meaningless measures. Let me know if I am 
going too fast. Fourth is too little access. The governor 
talked about transparency. It is the same issue there, too 
little access to the measures. And then finally, just 
resistance, organizational resistance to change.
    Senator Carper. Let me just ask either Ms. Steinhardt or 
Dr. Kettl, I want to ask both of you, to react to those five 
points, if you would.
    Ms. Steinhardt. I think our work supports every single one 
of those points. In fact, where we have seen greater use, real 
live use of performance information is where agencies have been 
able to overcome some of those kinds of barriers.
    It is very important in agencies or organizations to be 
able to communicate the kind of performance information 
throughout the organization, making it transparent to everyone, 
and we have seen some good examples of how agencies do that, 
communicating frequently to all of its employees where they 
stand. It helps spur competition. It provides good benchmarks 
for units across the organization. It has real benefits and a 
number of agencies have been able to do that.
    Let me consult my list again here.
    Senator Carper. Go right ahead.
    Ms. Steinhardt. Real-time information, absolutely. This was 
a point that I was making, I think, earlier, where the 
information isn't timely, credible, accurate, it is not going 
to be used, and organizations that figure out how to make that 
information available on a daily or real-time basis are ones 
that are going to see that it is used, and Governor O'Malley 
talked about how that concept is applied and was applied in 
Baltimore and now in StateStat. We have seen some very good 
examples of that, as well.
    Too many measures, that is sort of a basic precept of 
performance measurement. The vital few. You can't expect people 
to focus and really align themselves with what the organization 
is trying to accomplish if it is diffuse and scattered.
    Senator Carper. Good. Thank you. Dr. Kettl.
    Mr. Kettl. I simply couldn't agree more. That is exactly 
the core point here. The real issue here is that any time the 
process of performance descends into questions of measurement, 
we have in a sense lost the game because we get away from what 
it is that performance is all about. It is about communication. 
It is about talking about what it is that matters and figuring 
out how that communication gets people to act. And so as we 
tend to slice and dice things into ever smaller pieces and 
measure more things, we are likely to get to a point where we 
are unlikely to get people to act. Performance is about 
communication. It is not primarily about measurement.
    What is it that communication ought to be about? The key to 
overcoming resistance is primarily one of getting people to 
agree on what problem they want to try to solve and how to go 
about trying to do that.
    One of the examples that while mayor, Governor O'Malley 
worked on was something called the rat rub-out program in 
Baltimore. They had rats that they wanted to rub out. They had 
maps that showed where citizens were complaining about problems 
of rats. Which department in Baltimore is in charge of rats? 
The answer is, none of them are. On the other hand, it is a 
function of public health, of housing, of redevelopment, of 
transportation, of the people in charge of water and sewer. It 
is a cross-cutting collection of agencies which, put together, 
have to be responsible for solving the problem.
    If you ask everybody, do you want to collaborate? 
Everybody, of course, always says yes. But getting it to happen 
in real time and in real life is very hard to do. You put a map 
up on the board that shows, here is a problem with rats. Is 
anybody happy with that? No. What we are going to do about 
that? We want to get together to solve it. Who is going to 
solve it? Everybody understands their particular contribution 
to solving that problem, and that is what it is that drives it. 
It is communication about results that matter and getting 
people to put themselves to work to solve the problems that 
matter.
    So the reason why so often performance management doesn't 
work is it becomes issues of measurement and that conversation 
never happens.
    Senator Carper. Ms. Steinhardt, and then Mr. Peacock, you 
may want to say something here when she is finished.
    Ms. Steinhardt. I just wanted to add to this concept of 
visibility of information, because I think that is really also 
extremely key here. I came across this article a couple of days 
ago in the Wall Street Journal about how U.S. Airways has now 
gone from the bottom of the list of airlines in on-time 
performance to the top so far in 2008. What is really striking 
to me--the article is actually very interesting in discussing 
how it did that--is the fact that everybody knows about on-time 
performance. It is a very widely-used statistic. And in the 
airline industry, it is one of the key indicators of their 
viability as a company. It really matters to the CEO and 
everybody in the company, how their on-time performance is 
ranked. It is the visibility of the information, I think, that 
really makes such a big difference.
    Senator Carper. Excellent. Mr. Peacock, do you want to add 
anything before we move to another question?
    Mr. Peacock. The governor mentioned that EPA had rolled out 
an EPAStat program and he also mentioned two other things. He 
said it is easy to take any measurement program and put the 
``stat'' label on it, which is true. But he also mentioned that 
if you are going to do this seriously, you really have to be 
committed to it and have some core elements that you are 
committed to to make it work.
    Across these five barriers we have been talking about, EPA, 
through its EPAStat program, I think, is probably tackling--has 
tackled about three-and-a-half of them. But when I got to the 
agency, regarding lack of fresh and frequent information, I 
went in hoping to get monthly information. The most we could 
tolerate and are tolerating right now is quarterly information, 
which still took a lot of work to get. If it was any less 
frequent than that, it wouldn't work.
    The number of measures we had was too many. We have been 
able to cut it by 15 percent. We will continue to decrease it 
so that it becomes more manageable.
    In terms of access, the relentless meetings, EPAStat 
meetings we have, and they do have to be relentless--this gets 
back to Dr. Kettl's point, which I think is critical. This is 
all about communication. We have meetings--I have meetings 
personally with each assistant administrator and regional 
administrator every quarter on these statistics, but the 
richness of this is in discussing them and having a 
conversation about what is going right, what is going wrong, 
and what we are going to do about it.
    We now broadcast those meetings internally to any EPA 
employee who wants to view them, and our quarterly management 
report, as far as I know, we are the only Federal agency that 
on a quarterly basis puts out our performance measures to the 
public for anybody to look at. So we are very strong on the 
access.
    And then in terms of resistance, once again, I just happen 
to work in a culture where people are willing to see if changes 
results in us meeting our mission. There is just not a lot of 
resistance.
    But we still have a problem, I think, with focus on money 
and inputs rather than results. It is the sort of culture we 
run into, not just at EPA but throughout the Federal Government 
and outside the Federal Government, where people want to know, 
well, how much money are you asking for, when really the first 
question should be, what are you planning on delivering next 
year?
    Senator Carper. So true. Why do you suppose that is the 
case?
    Mr. Peacock. If I knew that, I wouldn't be at three-and-a-
half of these solved. I would be at four-and-a-half of them 
solved. I mean, I would be interested to hear why other people 
think that is the case.
    I will tell you, my experience in the OMB, where we 
realized this was a big problem, I mean, when the budget comes 
out every year, still, the first thing that is talked about is 
the dollars that are being proposed, not what we are going to 
get for it. So it is just ingrained in the way we think about 
things and I think it is going to be incremental, but it is a 
change that has started to take place, but it is going to have 
to keep taking place.
    Senator Carper. Dr. Kettl, do you want to respond to that 
same question that I just asked?
    Mr. Kettl. I think your question is why is it we focus so 
much on inputs----
    Senator Carper. Yes.
    Mr. Kettl [continuing]. And it is like the old joke about 
the drunk looking for the keys underneath the lamppost. Is that 
where you lost them? No, I lost them over here. Why are you 
looking under the lamppost? Because that is where the light is. 
People focus on the things that are easy to look at and easy to 
measure because OMB at least publishes a budget that has all 
the numbers in it and it is easy to say, how do we know we care 
about something? Because we are spending a lot of money on it. 
Then we wonder why it is that we are not producing good 
results. Well, because that is not the function of what it is 
that we are driving the system to.
    But what we need is, in a sense, an alternative way of 
thinking about accountability. What citizens care about--what 
citizens know about is that they want government to work, and 
in the end, quick headline, increases, decreases, symbols that 
are valuable. But if it doesn't make the government work 
better, citizens don't care, and in fact, it makes them even 
more cynical.
    I think the main reason why we do it is, first, because the 
numbers are there and they are easy and there is no clear 
alternative otherwise, and second, it has to do with the fact 
that we don't have an obvious alternative accountability 
mechanism that focuses on outcomes. Why? Because we have lots 
of measures, but we haven't figured out what to do with them 
yet. That is the next generation of reform that I think we 
need.
    Senator Carper. OK. Let us talk about the importance of 
leadership. I don't think the City of Baltimore would have gone 
to CitiStat without the leadership of their mayor. My guess is 
that the State of Maryland wouldn't have gone to StateStat 
without the leadership of Governor O'Malley.
    In Federal Government, we elect a President every 4 years, 
we elect or reelect. We have six-year terms here in the Senate 
and two-year terms, as you know, in the House. Folks who serve 
in the Executive Branch as cabinet secretaries are--they can 
serve for 4 years, occasionally 8 years, but more often than 
not they serve less than 4 years. Folks that are deputy 
secretaries and associate secretaries and under secretaries, 
they tend to come and go.
    How important is leadership and the continuity of 
leadership at the top and then down to different levels of 
government here in Washington? How important is the consistent 
message, that this is important, we have got to do this, and to 
hear that not just from the very top of the government, but 
also the top of each department and with the agencies 
themselves? How important is that? Please.
    Ms. Steinhardt. I would say it is absolutely the first 
step. It is the precursor. Nothing else--none of the other 
recommendations, none of the other practices that we talk about 
would be successful if the leader of the organization doesn't 
care about this because nobody else will if the leader of the 
organization does not. It doesn't end there, but it starts 
there.
    Senator Carper. All right.
    Mr. Peacock. I agree. It is a prerequisite, but there are 
two ways to institutionalize these systems and this way of 
thinking. One is through standard operating procedures and 
organizational structure, so for instance, at EPA, we now have 
put in place as part of a regular annual process we go through 
scrubbing our measures to make sure that they are meaningful, 
and that is actually the process by which we are reducing our 
measures.
    And second, in terms of an organization, for instance, we 
have now created a Program Analysis Division at EPA which 
essentially, with the Office of the Chief Financial Officer, 
makes the system run, produces the statistics. It is 
essentially like the EPAStat Office or the analogy which these 
cities and the State of Maryland, for instance, have.
    And then the second way to institutionalize this is the 
career people. Obviously, the policy officials, the political 
appointees aren't there very long, but we have career officials 
and managers who, at least at EPA, have been there a very long 
time. Particularly as the value of these systems are 
demonstrated to them, they become believers and advocates for 
them, and whether or not you have what the political appointees 
may be doing, or even if the standard operating procedures are 
in place, if this is a value to good managers in terms of 
managing their programs, then these systems will be used.
    Mr. Kettl. Mr. Chairman, the people question is absolutely 
essential. There is a paradox in the way in which the Federal 
Government has evolved, I think. As it has gotten more complex, 
the role of people in leadership has become more important. We 
had this model for a long time that said we will create a very 
complex system and we will slide people in and out in standard 
operating procedures because they will be pretty much 
interchangeable. The idea of being able to create a civil 
service system and all kinds of standard operating procedures 
were not to make people unimportant, but to make it less 
important who was serving in those positions.
    It is increasingly important now, the more complex the 
system has gotten, to have the right people, smart people in 
the right places to drive things forward or otherwise things 
just don't happen, and the way in which it does happen, and, 
for example, the Coast Guard has proven this, is making that 
part of the organization's culture. When the standard operating 
procedure becomes a culture that reinforces the pursuit of 
strong outcomes, that is what it is that produces this piece.
    Unfortunately, too often in the Federal Government, it is 
something that happens either by accident or by people having 
to do it despite the fact that the game somehow seems rigged 
against them. We have got to find a way to institutionalize 
strong performance-oriented cultures into the very core of the 
way government operates.
    Senator Carper. In looking at our third panel and thinking 
about the agencies that we are holding up here for acclaim, for 
applause, because they have gotten it and they are really 
setting an example for the rest of us, EPA is one of those and 
I think it is fair to say that, Mr. Peacock, he probably 
wouldn't admit this, but he is a big reason why EPA has moved 
down this road and has good results to show for it.
    But among the other agencies that are here, the Nuclear 
Regulatory Commission, NASA, the VA, and I wonder if there is 
any common thread with all of those or if you can think if 
maybe there is. I am not sure, but I think there might be, but 
any thoughts?
    Mr. Kettl. Mr. Chairman, I think it is in some ways a very 
simple answer to a very complex problem. They know what they 
are trying to do. They focus relentlessly on doing it. They 
reinforce people for doing it. They keep track of how well they 
are doing it. And it is a simple process that focuses on 
results and having a culture that reinforces what they are 
doing and they focus on what their outcome is.
    Their job is not simply to, at the VA, for example, taking 
a stack of papers on veterans' benefits from Part A and moving 
them over to Part B. They focus on the veterans. They have had 
a system, for example, of veteran-centered health care for a 
long time that is what drives them and drives what it is that 
they do. They know what they are there for and they know who 
they are trying to serve and they focus the system on serving 
them. And developing standard operating procedures and 
leadership to try to do that is the key to making a high-
performing organization.
    Senator Carper. Anyone else?
    Mr. Peacock. I think Dr. Kettl is exactly right. A clear 
mission that everybody can see and believes in has got to be a 
prerequisite for this. I don't remember who mentioned it, but 
this clear line of sight idea that everybody in the 
organization can see not only the mission of the agency, but 
how it connects to what they do on a day-to-day basis, and that 
is a prerequisite for this.
    There are other agencies that have a clear mission and I 
think everybody sees it but aren't necessarily doing a great 
job at this. So while it is a prerequisite, I don't think it is 
sufficient.
    Senator Carper. Ms. Steinhardt.
    Ms. Steinhardt. I agree completely with Dr. Kettl and Mr. 
Peacock's comments. I would add, though, and I don't 
necessarily apply this to VA and NRC, perhaps to NASA, but 
sometimes it is a crisis that focuses an agency on what it is 
about and what its mission and what its desired goals ought to 
be.
    This happened to the IRS, actually, in the mid-1990s. There 
were oversight hearings, a lot of press attention, 
Congressional attention on allegations of taxpayer abuse and 
dismal customer-taxpayer service and IRS had no information, 
performance information, to refute all of these horror stories 
that were coming out. And the Congress itself, I think, played 
a very key role in setting some very specific performance 
targets for the agency, and the agency began to collect the 
information, became very performance focused and it really 
turned itself around.
    In the period certainly from the 2000 survey, when we 
started, when we were asking about managers' use of information 
there to the most recent survey, they have showed a really 
positive improvement there, and more significantly in actually 
how the agency is performing, they have really made great 
strides.
    Senator Carper. All right. Well, good. There is still 
plenty of work to be done. Every year, we hear the size of the 
tax gap, and we learned last year it is over $300 billion, 
monies that are owed, not being collected. We deal with this on 
this panel. We deal with the improper payments. Not everyone, 
not all of the Federal agencies are reporting their improper 
payments, but of those who do, we know that our improper 
payments are, I think, roughly $50 billion a year, mostly 
overpayments. And it occurs to me that if somehow there were no 
tax gap and if we weren't making these improper payments, we 
would basically have a balanced budget.
    Ms. Steinhardt. But today, we know the size of the tax gap, 
or at least now IRS is able to estimate the size of the tax 
gap.
    Senator Carper. We weighed the pig. [Laughter.]
    Ms. Steinhardt. Well, first step.
    Senator Carper. There you go. A couple more questions and 
then we will excuse this panel and invite our last panel to the 
table.
    Ms. Steinhardt, in light of your survey results showing 
improvements in the amount and the types of performance 
measurements that agencies have at their disposal, why do you 
think so little change in managers' use of that information is 
being displayed in their decisionmaking?
    Ms. Steinhardt. Well, just to comment on the survey 
results, a lot of the results--the average, rather, is affected 
by the fact that we have a couple of large agencies like DOD 
and the Interior Department that are low, relatively low users 
of performance information. The smaller agencies like NASA, 
NRC, EPA, tend to be higher, but they don't color the average 
so much.
    In terms of why we are not seeing so much progress or use 
in those other agencies like Interior, for example, that is the 
subject of the second part of our study, so we hope to be back 
here. So we hope to be back here with some more concrete 
information on what those agencies can do to improve.
    Senator Carper. All right. Thank you.
    Mr. Peacock, how do you suppose the next Administration can 
get agency leaders to buy into such assessments and use a 
review process in resulting performance information to improve 
their management decisions?
    Mr. Peacock. I mean, there are so many directions one could 
go in that would be worthwhile, although I think that Dr. Kettl 
hit on some of them. We now have a wealth of information that 
wasn't there before, and looking at this information but from a 
cross-agency perspective, looking at it from the sense of the 
public who is more interested in protecting wetlands rather 
than whether it is the Corps of Engineers or the Department of 
Interior or EPA that is protecting the wetlands and being able 
to compare across agencies which programs are doing this the 
best, doing it the most efficiently, and then using that 
information in terms of making budget decisions or policy 
decisions.
    I mean, there would be any number of ways to do that, but 
there probably needs to be less emphasis now on trying to 
collect this information and more emphasis on actually bringing 
it in-house and someone sifting through it to actually use it 
and make decisions using it.
    Senator Carper. OK. Thank you.
    And finally, one more for Dr. Kettl, if I could. In your 
work on how government responded to Hurricane Katrina, you have 
described the importance, and I am going to quote here, ``the 
importance of working in horizontal networks instead of 
struggling over a vertical chain of command.'' As GAO has 
testified, there are many programs that need to work together 
to accomplish common outcomes. I think Mr. Peacock has said as 
much, too. What are some approaches the next Administration 
could take to bring about a more cross-cutting focus on 
performance issues?
    Mr. Kettl. The first thing, Mr. Chairman, is to underline 
the importance of these cross-cutting approaches to begin with 
because increasingly, what it is that government does has to 
happen through those relationships.
    The second thing is to focus on a smaller number of more 
important outcome-based measures, less on how many applications 
for processing, more on what kind of results we are producing, 
getting the information in real time, getting the information 
displayed graphically. One of the things about Mayor and then 
Governor O'Malley's work is that if you look at the maps, you 
just look at it and it talks to you. It tells you what the 
problem is and where to go about doing it.
    For example, he had a series of charts about the decline in 
the murder rate in the three kidney-shaped areas that he talked 
about. You look at that and you see it, you can see the change, 
you can see the progress, you see where the problem is. It 
tells you what it is that you need to work on and where you 
need to go. There is something about information that is 
focused on what matters, that is real time, that focuses on 
where it is that it happens that drives the collaborations that 
are required and then makes it possible to have the 
conversations about who it is who is going to contribute to the 
solutions to the problems that in the end matter.
    That is the secret to the successes in EPA's work, of 
trying to figure out what are we trying to do. How can we make 
the air cleaner? Who has got a piece of the action? How can we 
measure whether or not we are doing it? And it is that kind of 
relentless cycle of follow-through, not once a year, once every 
5 years collecting some interesting reports and putting it on a 
chart. It has got to be real-time information that drives 
action and drives action the citizens care about.
    I can say that with confidence not because I know it to be 
true, but because I have watched others do it. The successful 
managers around the Federal Government are doing that and more 
of that is what it is that we need.
    Senator Carper. I will just close with an observation. When 
I was privileged to serve as the chief executive of our State, 
there are a number of goals that we set. We wanted to raise 
student achievement, and we didn't want to just put it all on 
the schools or on the Department of Education. We wanted to 
reduce the incidence of teenage pregnancy. We just didn't want 
to put it all on the kids department or on the Department of 
Health and Social Services. We wanted to reduce the incidence 
of recidivism in our prisons, but we didn't want to put it all 
on our prisons. We wanted to reduce the crime rate, but we 
didn't want to put it all on our police.
    And we ended up establishing a Family Services Cabinet 
Council which included about half of the cabinet, and I would 
meet with them every month and we would set measurable goals 
and then try to hold one another accountable for progress. But 
I think the fact that I sat with the cabinet secretaries every 
month and then they would meet once a month without me with 
members of my governor's office staff just to follow up had 
some pretty good effect.
    There is something to be said for trying to hold a 
particular agency responsible for getting, say, ``A'' done, but 
sometimes ``A'' needs to be done in collaboration with a bunch 
of other folks. So you need somebody saying, you guys and gals 
have to work together. We do have stovepipes in the Federal 
Government just like we did in State Government and they are 
hard to get rid of.
    I want to thank you so much for your testimony today, 
really for your terrific work. Ms. Steinhardt, to you and your 
colleagues at GAO who are just enormously helpful to this 
Subcommittee and really to our Executive Branch, too. To Mr. 
Peacock, who has worked in OMB and done great work there and 
now gone to EPA and helped to show as a practitioner what we 
can do and always is a source of inspiration, actually. And Dr. 
Kettl, we have a lot of people who testify before us. Few are 
as clear and concise, particularly academics---- [Laughter.]
    Who are as concise and able to put in terms that everybody, 
even I, can understand. It is a real gift. So we thank you. It 
is easy to see why you have had real success and are highly 
sought after.
    I want to leave the record open for a week or two so that 
folks who are not here can have a chance to submit some 
questions, and if you get those and would respond to them in a 
timely way, we would be most grateful.
    But thank you so much for joining us today. Thank you.
    Ms. Steinhardt. Thank you.
    Mr. Peacock. Thank you.
    Mr. Kettl. Thank you, Mr. Chairman.
    Senator Carper. To our third panel, welcome. Thank you for 
hanging in here until almost 4:30. We promise not to keep you 
much longer, but thanks a lot. Have you all been here for the 
presentations of our first two panels?
    Mr. Tucker. Yes, sir.
    Senator Carper. Good. Well, a special thanks.
    One of the questions that I asked is, what are the common 
threads between the agencies that we are really holding up for 
acclaim here, who seem to get it and have been achieving better 
results because of that fact, in terms of measuring outcomes 
and actually using the information to change performance. That 
is one of the questions that I am going to be asking each of 
you, so be thinking about that, if you will.
    Jim Dyer is the Chief Financial Officer for the Nuclear 
Regulatory Commission. We were privileged in another committee 
that I serve on and chair, a subcommittee with Senator 
Voinovich, to have a chance to work with your agency a lot and 
have a very high regard for you. What is it, the best agency in 
the Federal Government in which to work?
    Mr. Dyer. The best place to work.
    Senator Carper. The best place to work, yes. But there, you 
serve as the person who is responsible for planning, for 
budgeting, and financial management of agency resources. You 
are also, I understand, currently the agency's Performance 
Improvement Officer, responsible for leading the agency's 
performance management activities.
    Prior to your position as Chief Financial Officer, I 
understand you served as Director of the Office of Nuclear 
Reactor Regulation, where you were responsible for agency 
safety programs for the commercial power, research, and test 
reactors of the United States. A couple of people suggested to 
me you also may have at one time served in the U.S. Navy and 
that you were a submarine officer from 1973 to 1977. I was on 
active duty from 1968 to 1973 looking for submarines and had a 
pretty good naval flight officer, P-3s, Orions, and our job was 
to hunt for Red October. We found plenty of them, but whenever 
we were looking for you guys, for our guys, we could almost 
never find you. You are mighty good at what you did. You served 
in the Naval Reserve until your retirement in 1995, and I 
served in the Reserves until my wife made me quit in 1991. But 
thank you for your service then and for your service today.
    Scott Pace is the Associate Administrator for Program 
Analysis and Evaluation at NASA. He is responsible for 
providing objective studies and analyses in support of policy, 
program, and budget decisions by the NASA Administrator.
    He previously served as Chief Technologist for Space 
Communications, where he participated in the negotiations that 
resulted in the 2004 GPS-Galileo agreement between the U.S. and 
the European Commission, and Mr. Pace also previously served as 
the Deputy Chief of Staff to NASA Administrator Sean O'Keefe. 
What is he doing these days?
    Mr. Pace. I believe he is working in industry these days.
    Senator Carper. All right. Tell him the Junior Senator from 
Delaware was asking for him if you run into him, please.
    Mr. Pace. Certainly.
    Senator Carper. Daniel Tucker, Deputy Assistant Secretary 
for Budget for the Department of Veterans Affairs. In this 
capacity, Mr. Tucker helps to oversee the $90 billion budget 
for the Department. In addition, Mr. Tucker serves as 
Performance Improvement Officer for the Department and is 
responsible for overseeing the development of Veterans Affairs' 
annual performance plan and managing Program Assessment Rating 
Tool evaluations. Prior to his current position at the VA, he 
served as the Chief Financial Officer for the National Cemetery 
Administration.
    With those introductions behind us, let me just ask Mr. 
Dyer if you would go ahead and lead us off. Summarize, if you 
will, in roughly 5 minutes and then we will have some questions 
and call it a day. But thank you. Please proceed.

   TESTIMONY OF JAMES DYER,\1\ CHIEF FINANCIAL OFFICER, AND 
   PERFORMANCE IMPROVEMENT OFFICER, U.S. NUCLEAR REGULATORY 
                           COMMISSION

    Mr. Dyer. Thank you, Mr. Chairman. It is an honor to appear 
before you today to share our approach for using performance 
information to improve management of U.S. Nuclear Regulatory 
Commission programs.
---------------------------------------------------------------------------
    \1\ The prepared statement of Mr. Dyer appears in the Appendix on 
page 92.
---------------------------------------------------------------------------
    The mission of the NRC is to license and regulate the 
Nation's civilian use of byproducts, source, and special 
nuclear materials to ensure adequate protection of public 
health and safety, promote the common defense and security, and 
protect the environment.
    The NRC is pleased that the Government Accountability 
Office recognized our improvements in the use of performance 
information and we believe it is the result of three factors: 
One, a strong commitment by the NRC commissioners and senior 
management to continuous improvement; two, the effective 
implementation of our planning, budgeting, and performance 
management process; and three, NRC openness with employees and 
our external stakeholders.
    We also believe that the recent government activities to 
create the Performance Improvement Officers and Performance 
Improvement Council can further improve the NRC performance 
management.
    We created the NRC planning, budgeting, and performance 
management process in response to the Government Performance 
and Results Act in 1997 and still use the four simple 
integrated components of the process to manage our program 
performance.
    First, our current strategic plan concisely identifies two 
strategic goals for safety and security to accomplish our 
mission and an organizational excellence objective which 
characterizes the manner in which we intend to achieve these 
goals.
    Second, our budget process involves multiple levels within 
the NRC organization, so we obtain staff's commitment to 
complete the planned activities on a schedule and within 
budget.
    Third, NRC executives monitor performance using office 
operating plans that track budget expenditures and performance 
targets well beyond the detail in the NRC performance budget 
delivered to Congress to improve our accountability.
    And fourth, our program assessments integrate the inputs 
from several sources, including Congressional hearings and the 
OMB PART tool results to develop program improvements. 
Additionally, we have recently started using the Lean Six Sigma 
evaluation process to more systematically assess program 
performance. These program assessments provide a significant 
input for NRC's senior executive and manager appraisals and 
awards.
    We also strive to effectively communicate NRC performance 
expectations and results to our staff and external stakeholders 
to promote openness and increased accountability. The NRC uses 
a strategic plan, the performance budget, Congressional 
reports, and our performance and accountability report to 
communicate with our stakeholders and receive their feedback. 
Commission meetings concerning NRC program performance reviews 
are some of our best attended public meetings. NRC offices 
publish quarterly operating plan results on their internal 
websites, routinely discuss the performance results during 
periodic staff meetings, and in newsletters, and recognize 
staff contributions to agency mission through the awards 
process.
    However, we do face challenges with effective performance 
management. As you are well aware, the NRC has had significant 
growth in the past few years, and this has increased the 
complexity of performance monitoring, with new programs, new 
organizations, and an increased level of work within the 
agency. Better performance management through expanded use of 
techniques such as the Lean Six Sigma and evaluation is needed 
for improved regulatory program consistency, efficiency, and 
effectiveness.
    Additionally, our current system of performance monitoring 
and assessments is very labor intensive. We need to modernize 
methods for processing performance information using the latest 
technology. Improving our budget structure and integrating cost 
and performance information in a more timely manner will 
facilitate improved performance management.
    Also, our performance metrics focus on the quantity and 
timeliness of our products over the quality because of the ease 
of measurement. Most often, quality issues are identified by 
schedule delays because of required rework. We need to develop 
methods for more timely measurement of the quality performance 
of our programs.
    The Office of Management and Budget-led Performance 
Improvement Council provides a forum to discuss these 
challenges through sharing agency best practices for improving 
the use of performance information. Presentations by other 
agencies on their experience with implementing Lean Six Sigma 
evaluations have been valuable to the NRC, and the Council's 
planned agenda items offer the potential to further improve our 
performance management.
    In conclusion, we appreciate the Government Accountability 
Office's recognition, but realize that we can further improve 
our use of performance information for managing our programs. 
We intend to continue to improve our performance through 
effective use of internal assessments, external oversight 
inputs, public feedback, and sharing experience with other 
Federal agencies.
    Thank you. I look forward to answering your questions.
    Senator Carper. Mr. Dyer, thank you very much.
    Mr. Pace, your entire statement will be made part of the 
record for each of you, so if you want to summarize, feel free.

TESTIMONY OF SCOTT PACE,\1\ ASSOCIATE ADMINISTRATOR FOR PROGRAM 
 ANALYSIS AND EVALUATION, AND PERFORMANCE IMPROVEMENT OFFICER, 
         NATIONAL AERONAUTICS AND SPACE ADMINISTRATION

    Mr. Pace. Just the highlights. Good afternoon, Mr. 
Chairman. Thank you, and it is a pleasure to be here.
---------------------------------------------------------------------------
    \1\ The prepared statement of Mr. Pace appears in the Appendix on 
page 97.
---------------------------------------------------------------------------
    As you mentioned my background at NASA, probably the most 
relevant point for today's hearing is I am also the agency's 
Performance Improvement Officer under Executive Order 13-450 
and the lead for various program improvement initiatives under 
the President's Management Agenda, which, of course, as you 
know, Sean O'Keefe had a lot to do with in writing. So I have 
had maybe a bit of an advantage.
    I have been able to observe the full spectrum of NASA's 
activities, its performance, where we have excelled and where 
we can improve, and I would like to share some observations, I 
think, on why NASA generally receives good marks and continues 
to receive strong bipartisan support from Congress, for which 
we are very grateful.
    First of all, as some of the other speakers, I think, have 
mentioned, NASA is really fortunate in being charged by the 
Administration and Congress with missions that are well suited 
to performance management. We have a performance-based culture 
that values mission success as a central tenet, and through 
hard experiences, both good and bad, NASA has sought to put the 
right tools and governance processes in place to better ensure 
accountability, transparency, and oversight. These processes 
are not ends in and of themselves but means for accomplishing 
the missions that are assigned to us by the Administration and 
Congress.
    In our experience, the best tools for creating 
accountability and transparency are those that provide 
consistent external reporting requirements, that provide 
flexibility internally in the design of measurement techniques 
tailored to our unique missions. Consistent external reporting 
helps focus the agency on the most important measures. As was 
mentioned, if you measure too many things, you are probably 
deluding yourself. And flexibility is important to help avoid 
the trap of imposing simple one-size-fits-all performance 
measures that can really mask more than they can reveal.
    While our system is generally working well, we are not 
without challenges and we need the support of Congress in 
maintaining our commitment to the efficient and effective 
execution of agency missions. Our mission is very concrete and 
our goals readily flow down to every level of the agency. Fly 
each Space Shuttle mission safely until retirement. Complete 
the International Space Station. Develop the next generation of 
launch vehicles that will return us to the moon, Mars, and 
beyond. And the performance against concrete goals is, 
therefore, measurable, traceable, and thus actionable.
    Our budget is aligned to our goals. We have well-
established program and project management policies driven by 
the need to deliver a wide portfolio of missions of many 
different sizes, many different destinations.
    Our programs are typically milestone-driven and we face 
hard deadlines, such as planetary windows, which we just have 
to hit. Our schedules are complex and must be integrated. Many 
organizations have to come together to fly each mission. And as 
an agency, we are comparatively small and compact, and 
therefore the key strategic informed conversations can be held 
at the highest levels of the agency. So our internal decision 
loops can be fairly tight.
    We are currently focused on aligning all of our external 
reporting to a single set of measures. We are linking internal 
performance indicators directly to these measures and 
commitments, in particular GPRA and the PART tools, as has been 
discussed earlier.
    I could describe our processes in a lot of detail, but the 
real demonstration, I think, of performance management is 
mission success. We currently have 56 robotic spacecraft 
operating throughout, and in some cases beyond, the solar 
system. We continue to conduct groundbreaking scientific and 
aeronautics research. The Space Station is nearing completion. 
We have seven more assembly flights to go plus two logistics 
flights, which we hope we will be able to fly before Shuttle 
retirement in 2010. This October, we are looking toward a final 
Shuttle mission to the Hubble space telescope to finish its 
repair.
    In the risky business we are in, we are going to continue 
to face challenges. Nine projects have breached cost or 
schedule thresholds in fiscal year 2008 against performance-
based lines that we established with the Congress in our major 
program annual reports. We are continuing aggressive effort to 
improve the fidelity of our up-front estimates of cost and 
applying more rigor to our life cycle cost estimates than 
really any other time in the agency. There are many sources of 
these cost growths, some we can control, some we can't, and we 
are working on each one.
    We accept the need to improve performance and transparency, 
but at the same time, we are struggling with the issue of 
reporting complexity. Of course, if a project is in trouble, it 
is in trouble, but under current reporting requirements from 
the multiple stakeholders, such as the OMB, Congress, and the 
Government Accountability Office, we have a variety of 
different trigger points and thresholds that include life cycle 
costs, development costs, schedule growth, key milestone slips, 
and each one of them can be slightly different.
    So we are looking for some greater consolidation in how 
these various breaches are defined, greater consolidation of 
requirements for what is in and out of that reporting, not only 
to reduce our reporting transaction costs, but also to ensure 
that agency attention and effort is focused on mitigating the 
most significant project performance issues so we stay focused.
    I really want to thank you very much for your time this 
afternoon and really appreciate you drawing the Committee's 
attention to this important topic. It is something that is near 
and dear to us and we appreciate your interest. Thank you.
    Senator Carper. Thank you, and thank you for coming and for 
the example that you provide. Mr. Tucker.

TESTIMONY OF DANIEL TUCKER,\1\ PERFORMANCE IMPROVEMENT OFFICER, 
AND DEPUTY ASSISTANT SECRETARY FOR THE BUDGET, U.S. DEPARTMENT 
                      OF VETERANS' AFFAIRS

    Mr. Tucker. Thank you, Mr. Chairman. I appreciate the 
opportunity to be here today to discuss how the Department of 
Veterans Affairs is using performance information to improve 
service delivery to veterans and accountability for results.
---------------------------------------------------------------------------
    \1\ The prepared statement of Mr. Tucker appears in the Appendix on 
page 100.
---------------------------------------------------------------------------
    Since the passage of GPRA 15 years ago, we have seen a 
major transformation in the manner in which performance 
information is used in budgeting and performance management 
throughout the VA. We have made noteworthy progress in using 
performance information to better justify our requests for 
resources, monitor our programs throughout the year, and 
document our accomplishments and challenges in a manner that is 
transparent to veterans, our stakeholders, and the general 
public.
    Every year, we reevaluate the performance measures included 
in our Congressional budget justifications and operating plans. 
We analyze each measure to ensure it is still appropriate and 
meaningful and we develop new or improved metrics that present 
a better gauge of program results. We use customer feedback to 
ensure that the performance metrics we employ are measuring 
those things that are most important to veterans and their 
families.
    Today, our budget request for every program contains a 
wealth of information on our strategic goals and objectives, 
our historical and projected performance levels, and the means 
and strategies we will use to achieve our goals. In other 
words, our budget is justified not only by the kinds of 
activities we will conduct, but more importantly, by the 
results we expect to achieve.
    VA employs a variety of mechanisms to monitor program 
performance. The most important of these tools is our monthly 
performance review. Chaired by the Deputy Secretary, these 
reviews involve senior leaders throughout the VA and focus on 
financial management and program performance as well as the 
execution of major construction and information technology 
projects. These meetings play a vital role in keeping the 
Department focused on its highest priorities, achieving key 
performance goals, and resolving operational challenges.
    On November 15 each year, we publish our annual Performance 
and Accountability Report. This report presents a detailed 
description of how well VA performed relative to the 
performance goals we established at the beginning of the year.
    Our commitment to transparency and reporting has been 
highlighted by the Mercatus Center of George Mason University. 
This independent research organization conducts an extensive 
evaluation of all agency performance reports. They have ranked 
VA among the top-rated reports all 9 years they have conducted 
their analysis. For the last 2 years, the Mercatus Center has 
presented an award to VA for achieving the highest score in the 
Federal Government in transparency in reporting and the 
Department is extremely proud of this recognition.
    Another one of our major accomplishments has been our 
success in establishing a performance culture in VA. This has 
largely involved a gradual movement away from a focus on inputs 
toward a more meaningful discussion of program outputs and 
outcomes. And education and training within the Department for 
staff and managers is critical to this successful shift in 
emphasis.
    Our monthly performance reviews have also been instrumental 
in institutionalizing a performance culture at VA. Most 
importantly, these monthly meetings are a clear demonstration 
of the interest and support of VA's top policy issues in using 
performance measurement to oversee departmental programs and 
operations.
    Another key strategy we employed to institutionalize a 
performance culture was to develop a set of key performance 
measures. Several years ago, we realized that we have been 
pretty successful in developing improved performance metrics, 
but we had created so many that it is hard to figure out what 
was most important. So to correct this problem, we identified a 
set of about 25 key measures that we considered critical to the 
success of the Department. These key measures formed the 
foundation of our budget request, our monthly performance 
reviews, and our annual performance report.
    While we have successfully tackled many of these 
challenges, we still have more that need further attention. We 
have a sound set of outcome measures for some of our programs, 
particularly medical care, but there are other program areas 
for which we still need better indicators of the extent to 
which VA programs improve quality of life for veterans and 
their families.
    An additional challenge all agencies face is how to more 
tightly link cost data to program performance. In particular, 
we need to strengthen our ability to demonstrate how 
performance could change if resource levels vary. As with all 
performance metrics, this will require the necessary 
information systems and analytical tools to produce valid and 
reliable data.
    That concludes my statement, Mr. Chairman. I will be happy 
to answer any questions.
    Senator Carper. All right, Mr. Tucker. You are right on the 
money. That was exactly 5 minutes. Thank you.
    Thank you all for wonderful testimony. Why do you suppose 
your agencies have risen to the top, at least with respect to 
your use of information to be able to improve performance? What 
common threads are there with your agencies in terms of your 
experience and improvement? Mr. Dyer, when you retired from the 
Navy, what was your rank?
    Mr. Dyer. I was a captain, sir.
    Senator Carper. Captain Dyer, OK. I was a captain, too.
    Mr. Dyer. Yes, sir.
    Senator Carper. A recovering captain. A recovering 
governor.
    Mr. Dyer. Senator, as I heard the other two presentations, 
that was the first time I had heard or seen their text, too, 
the thing that I heard--the three reasons I thought that we had 
improved in the past few years, I heard that come out in 
theirs, and that is a commitment to doing it--and we heard it 
from the earlier panels, too--a commitment to doing it, a 
mission statement that is simplified and that allows you to go 
to outcomes, which drive outputs, which drive the inputs to the 
budget, and that kind of a focus on executing to achieve the 
outcomes intended.
    Senator Carper. Mr. Pace.
    Mr. Pace. I would agree. I think the most important 
advantage, I remember the prior panel talking about as things 
become more and more complex, people become more and more 
important because they are not interchangeable and I think that 
the performance culture that we have at NASA, which is 
reinforced in so many different ways, has been actually really 
crucial, so that when you come in with a performance 
measurement system of some sort, initially, it is looked at 
askance. What is this, a new unfunded mandate dropping down on 
us?
    But when people get into it and see that you are going to 
use that information to make better decisions that then have 
some linkage to mission success, then there is a greater 
acceptance of it. It is something that actually reinforces the 
performance culture of the agency because it is not seen as 
alien to the agency but, in fact, is integral and consistent 
with the values of the agency, and that in turn is reflected by 
statements from the top management down through the system.
    So it is a very interwoven problem, but as you hear, it is 
a very consistent one that repeats in many organizations.
    Senator Carper. All right. Mr. Tucker.
    Mr. Tucker. I think from what the panel members here have 
stated as well as the members of the previous panels, the thing 
that you hear consistently is top management interest and top 
management involvement. You have to have the leadership 
engaged. If the leadership is engaged and focused on using 
performance information, that will drive change throughout the 
organization.
    Senator Carper. Let me just interrupt for a second. I have 
been privileged to know three VA Secretaries during my time 
here in the Senate, and that was in 7 years, so you have had a 
number of Secretaries. Is it Tom Bowman? Who is the Chief of 
Staff----
    Mr. Tucker. Tom Bowman is.
    Senator Carper. I think Mr. Bowman has been around for a 
while----
    Mr. Tucker. Yes, sir.
    Senator Carper [continuing]. And maybe he provides the 
continuity. An old Marine, as I recall.
    Mr. Tucker. Yes, sir.
    Senator Carper. But you have had a fair amount of turnover 
at the top in the VA, but a continued commitment to 
improvement.
    Mr. Tucker. I think the commitment from those that remain, 
like Tom Bowman is a good example as the Chief of Staff, he 
sits in on our monthly performance reviews every month. He is 
certainly a very active member of the management team. I think 
also what is important, especially at the VA, even though we 
are a very large agency and we have a great diversity of 
programs, we are all there to serve veterans and we are all 
there to serve their families and those that are eligible for 
the different benefits that we provide, and I think that is 
part of our performance and the success of our performance 
culture at the VA, is a real clear mission statement and we 
know who we are there to serve.
    Senator Carper. If you think of it, let Tom Bowman know we 
just opened our second VA outpatient clinic in Delaware. We 
only have three counties. We have two of these clinics now. We 
just opened two of them in the last 2 or 3 months. But he was a 
big help in getting that done, so tell him we are grateful for 
that and for his service.
    Mr. Tucker. I will do that.
    Senator Carper. You have mentioned performance culture. Mr. 
Pace has mentioned it. I think Mr. Dyer has mentioned it. One 
of my other hats is chairman of the subcommittee that has 
jurisdiction over nuclear safety. We talk a fair amount about 
safety culture, and during the time in my 23 years in Naval 
aviation, oftentimes our commanding officers would say to us, 
and again, our job was to hunt for Red October, track Soviet 
nuclear submarines, and do all kinds of surface surveillance 
and fly missions off the coast of Vietnam and Cambodia and 
other places.
    But our skippers would always say the most important thing 
you are doing in your mission today isn't whether you are 
trying to mine some harbors or whether you are doing certain 
service missions where you are tracking the bad guys, the bad 
submarines, the Ruskies, whatever it was. He said your most 
important thing that you are doing is taking off safely, flying 
safely, coming back and landing safely, and walking away from 
it and going home. So that is the most important thing that you 
are doing.
    Almost everything that we did, whether it was in the 
aircraft, the folks who were maintaining our aircraft, training 
us, everything pointed toward safety. It was really a culture 
of safety and we tried to do it in a way so it wasn't a 
``gotcha'' system, so that if people observed things that were 
wrong, they would actually step forward and say, ``This isn't 
really a very safe practice,'' and not be penalized or punished 
for having said that.
    So we try to take the same thing, the same approach with 
respect to our nuclear power plants and to encourage them, and 
I know that the NRC are very much involved in doing that for 
us. The safety culture, we have 104 nuclear power plants. We 
want every one of them to be as committed to safety as we were 
in my squadron, our squadrons in the Navy.
    Talk to me, if you will, about performance culture. How do 
we establish a kind of--obviously, you have a performance 
culture at your agencies and we have a bunch of agencies who 
don't have it. You are the exception, not the rule. How do we 
establish the kind of performance culture at more agencies, 
much as we are trying to establish safety cultures at 104 
nuclear power plants? And that would be a question for each of 
you. I don't know who cares to go first. Mr. Dyer, feel free.
    Mr. Dyer. Yes, sir, Mr. Chairman. Just like you talked 
about the safety culture at the nuclear plants and the 
guidelines that go along with them, I think starting with the 
performance culture at the NRC, it begins actually with the 
formulation of the budget. When we get in to recognizing the 
roles and responsibilities and of the individuals within the 
organization and get their commitment that we are going to 
expend this amount of resources to achieve this kind of 
results, we get that kind of commitment of them that they have 
bought in, that they sign up for it, and it is not something 
that we pass down from on top and just say, you are going to 
get X number of resources and get it done. It is they have an 
input to it that starts, that contractor, that bond, if you 
will, to actually perform the work and get the results that are 
intended.
    Senator Carper. All right. Thank you. Mr. Pace.
    Mr. Pace. Really, I want to also agree with that. One of 
the crucial issues we have is aligning the resources that 
content, that we make program commitment agreements and we say 
we want to do this vehicle, do this kind of mission, this is 
what it is going to cost, and if we change the resources, we 
change the content that we ask of people. And if we change what 
we ask of people, we also change the resources. The two in the 
budget process have to be very closely linked. So if the 
schedule slips or something else slips or new problems crop up, 
that we account for it.
    And I think that is part of having really almost an ethic, 
a moral duty, really, to the people we ask to do the work, that 
we are going to back them up and they, in turn, have an 
obligation to bring forward their best estimates about what it 
is going to take because we are both trying to achieve that 
mission success. So the budget process is at the center of it.
    The second thing I would add is that along with having a 
safety culture, it is important that our independent reviews 
and our assessments are done with the mindset of assisting the 
project to be successful, not of a mindset of auditing the 
project or trying to come up with a certain number of gigs. The 
mentality we try to bring to it is we are kind of like the 
graduate TAs. We are there to get the other students to pass. 
We may beat on them pretty hard, may review them pretty hard, 
but our goal is we want them to pass, we want them to succeed, 
and we are there to help them succeed, and having that 
mentality of an assist function also gets people to buy in, 
bring their problems forward because we are not out there to 
have them fail. We are there for mission success.
    Senator Carper. Thank you. Mr. Tucker, anything you would 
like to add?
    Mr. Tucker Just adding on top of what the other witnesses 
have said, I think one of the big changes that we have seen in 
establishing a performance culture at the VA is the shift from 
the discussion on inputs into outcomes. A lot of folks have 
said that today.
    I can think back years ago and when we would have monthly 
budget reviews in the Department. It was always just simply a 
discussion of, well, what is the variance from plan? Did you 
spend more than your monthly plan this month? Did you spend 
less? How much more? How much less? People were either happy or 
sad about the results, but the discussion didn't really focus 
on program results.
    Those are the kinds of discussions that we have now. We 
have the data. We have the data systems in place to facilitate 
that discussion. And management from the very top all the way 
down to the medical center level use that information to 
improve performance. The measures are the same whether they are 
at the national level, the network level, or the local level.
    An example I can give you is on wait times for VA patients 
to get into our health care system. We have used performance 
information to really drive improvement in reducing the number 
of veterans that are waiting more than 30 days for an 
appointment from--it was over 250,000 in April 2006 and we have 
driven that down to about 51,000 in June, just last month. So 
that is one example of how leadership is focused at the top all 
the way down to the facility level to drive performance 
improvement.
    Senator Carper. OK. Thank you. Mr. Pace, maybe one last 
question for you. According to a GAO survey, I think it was 
done in 2007, a large majority of your managers said that they 
use performance information to identify and solve program 
problems. Do you think that looking at program performance 
information has helped improve NASA's programs, and if so, how? 
You have talked about this already, but do you want to add 
anything else?
    Mr. Pace. I think one of the major areas of differences 
where I have certainly seen a change in the time that I have 
been at NASA is when I first came there, we would go to program 
reviews and people would be talking about the status and in 
some cases how they felt about the program or project, as Mr. 
Tucker was just saying. What you see in the status reviews that 
we have monthly now and baseline performance reviews is much 
more toward quantitative. So programs are coming in with--here 
is where we are on the finances. Here we are on the human 
capital. This is what the state of the asset and facilities is 
looking like. This is where the program has hit various 
milestones. So monthly meetings, much more quantitative.
    That then sparked discussions. I mean, the most important 
thing that occurs in that quantitative data and where people 
see various disconnects is then a conversation occurs at the 
senior management level that says, why are those disconnects 
happening? What can we do about them? What is the resolution 
path forward?
    So I think that it is helping NASA quite a bit. We are 
doing things that are consistent with our culture, but we are 
doing them, I think, in a much more rigorous and consistent 
way. We are still learning. We still have plenty of other areas 
that are imperfect, but I think the intention on performance 
being quantitative and getting those discussions out in front 
of senior management has been very helpful.
    Senator Carper. Thank you. Mr. Dyer, according to the GAO's 
same 2007 survey results, the NRC's managers report a 
significant increase in the use of performance information 
since responding, I think to a survey 7 years earlier, the 2000 
survey. In fact, I am told that NRC managers are now among the 
most positive among their government-wide counterparts in their 
reported use of performance information. I just ask, in your 
view, how and why you think this change came about.
    Mr. Dyer. Mr. Chairman, I believe we have had this same 
process in place since 1997 and every year we do a little bit 
better. We revise our performance indicators. We reach out and 
get them to lower levels in the organization. And I think the 
people can trend progress and we report out on it and we 
provide feedback to the staff on it and they recognize that 
these things are important and that they now accept them and 
embrace them.
    Senator Carper. All right. One of the things that I find 
remarkable when we have testimony from the NRC and from the 
nuclear industry and others who are interested in the industry 
is the increase in the power plants that we have, their ability 
to provide power. If they are able to provide power 100 percent 
of the time, that would be terrific. But it wasn't that long 
ago when they were down around 60, 70 percent, and today they 
are up over 90, 91 percent. That is a remarkable achievement.
    Mr. Dyer. Yes, sir.
    Senator Carper. A lot of people deserve credit for that and 
I know among those people are the NRC.
    The last question would be really for all of you. Each of 
your agencies has had programs that were assessed by OMB 
through the PART process. What suggestions do you have for the 
next Administration, whoever might lead that, but the next 
Administration on how to design and structure a performance 
assessment process such as PART?
    Mr. Tucker. I will go first.
    Senator Carper. Yes, Mr. Tucker, feel free.
    Mr. Tucker. I think what will be helpful for OMB to do is 
to work collaboratively with the agencies to look at the 
criteria that is used and really to continue the process and 
refine the process of the evaluation and performance measures. 
I mean, I think that has been the key benefit that the VA has 
obtained from going through the PART process is having a really 
hard look at the performance metrics that we use, whether they 
are outcome-focused or efficiency focused, and developing 
action plans and requiring the action plans to be updated on a 
regular basis so you are continually improving the metrics that 
the organization is using to assess performance.
    I think one other thing--I will just put in a plug--is 
recently in response to the Executive Order that the President 
signed back in November, OMB did establish an interagency 
Performance Improvement Council. I think that council should 
continue into the next Administration. It is just getting 
started. It is growing. It is getting traction, starting to get 
some legs under it.
    Senator Carper. Who heads up that council?
    Mr. Dyer. Robert Shea is leading the Performance 
Improvement Council at OMB right now.
    Senator Carper. OK. Thank you.
    Mr. Dyer. But I hope that it will be as successful in the 
future as the Budget Officers Advisory Council has been that 
OMB had established a long time ago where the budget officers 
from the different departments come together once a month to 
share best practices, to raise issues and concerns, and just to 
have that face-to-face once-a-month meeting and discussion. And 
I think the Performance Improvement Council can do the same 
thing.
    Senator Carper. Good. I had not heard of the Performance 
Improvement Council. It sounds like a good idea. I hope it 
continues, as well. Thank you.
    Mr. Dyer. Thank you.
    Senator Carper. Mr. Pace.
    Mr. Pace. Well, I would also actually endorse some of the 
Performance Improvement Council meetings. I also go, as well. 
But I think one of the things I have found most valuable for it 
is actually the cross-agency discussions that have happened, so 
seeing what other agencies are doing, sharing notes with them, 
and comparing best practices. I thought that has been a useful 
forum. I know that other speakers have talked about cross-
agency issues.
    Regarding the PART, I think it obviously should continue. I 
think it has been helpful. We have been able to improve the 
measures of effectiveness for some of our programs. 
Particularly we have made some improvements, I think, in our 
education efforts that have been helpful and traceable to PART.
    But the other thing I would say in terms of maybe a 
particular parochial plug that we and some other agencies have 
is in measurements of R&D. We have a very project-oriented 
agency. Other agencies are very process-oriented. But some of 
our science agencies, NSF, maybe NOAA, ourselves, Geological 
Survey, when we are doing R&D activities that stretch over a 
longer period of time, it is often harder to see the results 
each year. You have to look over a longer time horizon to judge 
what the benefits of R&D are.
    So ourselves and NSF have had some good conversations about 
this. I know with DOE Science Office, we have also had some 
good conversations, and measuring effectiveness of R&D is one 
of those areas where a simple one-size-fits-all sometimes 
doesn't work. And the conversations we have had in the PIC, I 
think have been helpful to us, coming up with some ideas we are 
going to hope to pursue.
    Senator Carper. Thank you. Mr. Dyer, the last word?
    Mr. Dyer. Mr. Chairman, I think the PART program, the PART 
is fundamentally sound and it is a good program. At the NRC, we 
have had seven PART evaluations. Six of them have been rated 
high as effective and one was moderately effective, and I 
owned, designed that one that was moderately effective, and it 
had a good therapeutic effect, I think, on the NRC that when we 
went back and pulled the string on it, what we really looked at 
and came up with was that we had a problem with our budget 
structure and that cascaded down into problems, and that wasn't 
what we thought going in.
    So I have got a healthy respect for the PART. That being 
said, it is an extremely resource-intensive process. So as we 
go forward, I would hope that we streamline the process so we 
don't have to dedicate the amount of energy that goes in in 
conducting the PART evaluations.
    Senator Carper. All right. Thank you.
    We are at the end of our hearing. I was just reflecting on 
this. It is pretty clear I haven't had a lot of company up here 
on this side of the rostrum. We have a lot of the agencies, not 
all the agencies have really emulated your success and your 
approach by taking seriously our ability to use the information 
that it has generated to improve our performance. Not everyone 
does that.
    Having said that, we are going to have a new President in 6 
months. We will have a new Congress in less than that. And we 
are going to still have a lot of problems to face here at home 
and around the world. We don't have unlimited resources. We are 
finding out just how limited our resources are, financial 
resources are to deal with those challenges.
    And if we are smart, and I hope we will be, the more of us 
here in the Legislative Branch and the folks in the Executive 
Branch, including right at the top, will realize that if we 
want to provide better results, better outcomes for the folks 
who are paying the taxes in this country, there are a variety 
of ways to do it and one of them is to take all this massive 
information that we have, try to figure out what of it we can 
learn from in order to provide cleaner air, in order to provide 
better transportation, in order to reduce our threats from 
within and without, and act on that. If we do those things, 
taxpayers will be happy and we will be better off as a Nation, 
too.
    We chose in this hearing today not to focus on the agencies 
that, frankly, weren't doing a good job. We chose to hold up a 
handful that are doing a good job in this regard and to say by 
inference to other agencies that could learn from you, take a 
look at these folks.
    I said to Governor O'Malley, we would rather see a sermon 
than hear one, and with you, you have had a chance to give us a 
little bit of a sermonette, but we have also had an opportunity 
to watch your performance over a number of years and we like 
what we see. Hopefully, by holding you up for praise and 
acclaim, we will encourage some other agencies to emulate your 
good performance.
    With that having been said, we are going to leave the 
record open for 2 weeks so we will have the opportunity for 
some of my colleagues and to submit some questions, maybe some 
statements, as well, for the record. We would ask if you do 
receive questions that you just respond promptly and we will be 
better for it.
    Thank you so much for coming, for the good work that you 
and your colleagues back in your agencies are doing. Convey our 
congratulations and our thanks to them, if you will. Thank you 
so much.
    Mr. Dyer. Thank you.
    Mr. Pace. Thank you.
    Mr. Tucker. Thank you.
    Senator Carper. This hearing is adjourned.
    [Whereupon, at 5:10 p.m., the Subcommittee was adjourned.]




                            A P P E N D I X

                              ----------                              

[GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]




                                 




