    [Senate Hearing 110-817]
    [From the U.S. Government Publishing Office]

                                                            S. Hrg. 110-817

       IMPROVING FEDERAL PROGRAM MANAGEMENT USING PERFORMANCE INFORMATION

    =======================================================================

                                    HEARING

                                   before the

                    FEDERAL FINANCIAL MANAGEMENT, GOVERNMENT
                       INFORMATION, FEDERAL SERVICES, AND
                      INTERNATIONAL SECURITY SUBCOMMITTEE

                                     of the

                                  COMMITTEE ON
                             HOMELAND SECURITY AND
                              GOVERNMENTAL AFFAIRS
                              UNITED STATES SENATE

                           ONE HUNDRED TENTH CONGRESS

                                 SECOND SESSION

                                   __________

                                 JULY 24, 2008

                                   __________

           Available via http://www.gpoaccess.gov/congress/index.html

           Printed for the use of the Committee on Homeland Security
                            and Governmental Affairs

                      U.S. GOVERNMENT PRINTING OFFICE
    44-582                    WASHINGTON : 2009
    -----------------------------------------------------------------------
    For sale by the Superintendent of Documents, U.S. Government Printing
    Office Internet: bookstore.gpo.gov Phone: toll free (866) 512-1800; DC
    area (202) 512-1800 Fax: (202) 512-2104  Mail: Stop IDCC, Washington, DC
    20402-0001

            COMMITTEE ON HOMELAND SECURITY AND GOVERNMENTAL AFFAIRS

                   JOSEPH I. LIEBERMAN, Connecticut, Chairman
    CARL LEVIN, Michigan                 SUSAN M. COLLINS, Maine
    DANIEL K. AKAKA, Hawaii              TED STEVENS, Alaska
    THOMAS R. CARPER, Delaware           GEORGE V. VOINOVICH, Ohio
    MARK L. PRYOR, Arkansas              NORM COLEMAN, Minnesota
    MARY L. LANDRIEU, Louisiana          TOM COBURN, Oklahoma
    BARACK OBAMA, Illinois               PETE V. DOMENICI, New Mexico
    CLAIRE McCASKILL, Missouri           JOHN WARNER, Virginia
    JON TESTER, Montana                  JOHN E. SUNUNU, New Hampshire

                      Michael L. Alexander, Staff Director
         Brandon L. Milhorn, Minority Staff Director and Chief Counsel
                      Trina Driessnack Tyrer, Chief Clerk

    FEDERAL FINANCIAL MANAGEMENT, GOVERNMENT INFORMATION, FEDERAL SERVICES,
                    AND INTERNATIONAL SECURITY SUBCOMMITTEE

                      THOMAS R. CARPER, Delaware, Chairman
    CARL LEVIN, Michigan                 TOM COBURN, Oklahoma
    DANIEL K. AKAKA, Hawaii              TED STEVENS, Alaska
    BARACK OBAMA, Illinois               GEORGE V. VOINOVICH, Ohio
    CLAIRE McCASKILL, Missouri           PETE V. DOMENICI, New Mexico
    JON TESTER, Montana                  JOHN E. SUNUNU, New Hampshire

                        John Kilvington, Staff Director
                      Katy French, Minority Staff Director
                           Monisha Smith, Chief Clerk

                                C O N T E N T S

                                     ------
    Opening statement:
                                                                       Page
        Senator Carper...............................................     1

                                   WITNESSES
                            Thursday, July 24, 2008

    Hon. Martin O'Malley, Governor of Maryland.......................     4
    Bernice Steinhardt, Director for Strategic Issues, U.S.
      Government Accountability Office...............................    18
    Marcus C. Peacock, Deputy Administrator, U.S. Environmental
      Protection Agency, and Former Official at U.S. Office of
      Management and Budget..........................................    20
    Donald F. Kettl, Director, Fels Institute of Government, and
      Robert A. Fox Professor of Leadership, University of
      Pennsylvania...................................................    23
    James Dyer, Chief Financial Officer, and Performance Improvement
      Officer, U.S. Nuclear Regulatory Commission....................    36
    Scott Pace, Associate Administrator for Program Analysis and
      Evaluation, and Performance Improvement Officer, National
      Aeronautics and Space Administration...........................    38
    Daniel Tucker, Performance Improvement Officer, and Deputy
      Assistant Secretary for the Budget, U.S. Department of Veterans
      Affairs........................................................    40

                         Alphabetical List of Witnesses

    Dyer, James:
        Testimony....................................................    36
        Prepared statement...........................................    92
    Kettl, Donald F.:
        Testimony....................................................    23
        Prepared statement...........................................    86
    O'Malley, Hon. Martin:
        Testimony....................................................     4
        Prepared statement...........................................    49
    Pace, Scott:
        Testimony....................................................    38
        Prepared statement...........................................    97
    Peacock, Marcus C.:
        Testimony....................................................    20
        Prepared statement...........................................    84
    Steinhardt, Bernice:
        Testimony....................................................    18
        Prepared statement...........................................    55
    Tucker, Daniel:
        Testimony....................................................    40
        Prepared statement...........................................   100

                                    APPENDIX

    Questions and Responses for the Record from:
        Governor O'Malley............................................   107
        Ms. Steinhardt...............................................   112
        Mr. Peacock..................................................   117
        Mr. Tucker...................................................   128
    Prepared statement from Robert Shea, Associate Director for
      Government Performance, Office of Management and Budget........   131
    Charts submitted for the Record by Governor O'Malley.............   136

       IMPROVING FEDERAL PROGRAM MANAGEMENT USING PERFORMANCE INFORMATION

                                  ----------

                            THURSDAY, JULY 24, 2008

                                     U.S. Senate,
            Subcommittee on Federal Financial Management,
                   Government Information, Federal Service,
                                  and International Security,
                              of the Committee on Homeland Security
                                            and Governmental Affairs,
                                                        Washington, DC.
        The Subcommittee met, pursuant to notice, at 2:35 p.m., in
    room SD-342, Dirksen Senate Office Building, Hon. Thomas R.
    Carper, Chairman of the Subcommittee, presiding.
        Present: Senator Carper.

                  OPENING STATEMENT OF SENATOR CARPER

        Senator Carper. The Subcommittee will come to order.
        Governor, welcome. We are delighted that you are here.
    Thank you so much for taking time to join us. Senator Tom
    Coburn is the Ranking Republican of this Subcommittee. He will
    be here shortly. But given all that you have on your plate,
    Governor O'Malley, we are delighted that you are able to find
    time to be here and talk about something not only that you are
    interested in, but you know a whole lot about, and we thank you
    for your willingness to share your experiences, your
    perspectives, both as Mayor of Baltimore and now as Chief
    Executive of the State of Maryland.
        One of my favorite sayings is ``I would rather see a sermon
    than hear one,'' and I think we can look at what you have done,
    both as mayor and as governor of your State, to actually show
    us by your own behavior that this works and it is something
    that may just work for us, as well.
        In a little less than 6 months from now, we will have a new
    President standing on the West Portico of the Capitol,
    preparing to raise his right hand and take an oath to defend
    our country and Constitution. Our new Chief Executive will face
    a soaring Federal deficit at home, and security challenges
    abroad. Those are just a couple of the exceptional challenges
    that our Federal Government must be prepared to face. No matter
    who is elected President, whether it is Senator Obama or
    Senator McCain, we must make sure that person has the
    information and the tools that they need in order to keep our
    ship of state headed toward the horizon.
        And though our politics may differ here in Washington and
    across the country, I think we can all agree that the strength
    of our democracy hinges on the ability of our government to
    deliver its promises to the people. We have a responsibility to
    be judicious stewards of the resources the taxpayers invest in
    America and ensure that those resources are managed honestly
    and that they are managed effectively.
        Just recently, this Subcommittee heard from a number of
    witnesses, including our former Comptroller General, David
    Walker, about the dire fiscal situation this country faces, not
    just this year, not just in this decade, but for decades to
    come. Over the next two decades, 80 million baby boomers--I am
    one of them--will become eligible for Social Security and
    Medicare. Today, these two programs already make up over 40
    percent of our government's total expenses. And as boomers like
    me start to draw benefits, some experts we have heard from say
    that the share of these programs could equal within 40 years
    not 40 percent of our government's outlays, but all of our
    government's outlays.
        Without any changes, we will not have extra funds to
    prepare a world-class workforce in the 21st Century, funds to
    make us energy independent, meet our transportation needs,
    clean our air, or address our pressing national security needs.
    There is only so much pie to go around, so how we manage who is
    getting the next slice, and how big it is going to be, becomes
    increasingly important.
        In 1993, the year that I became Governor of Delaware, the
    Government Performance and Results Act (GPRA), was enacted to
    help us better manage our Nation's finite resources and improve
    the effectiveness and the delivery of Federal programs. Since
    that time, agencies have collected, and provided to Congress, a
    tremendous amount of performance data. However, Senator Coburn
    and I wondered if this information was being translated into
    results, or at least into improved results. Producing the
    information does not by itself improve performance, and we knew
    that there had been little increase in the actual use of
    performance data by agency managers.
        A lot of work has been done by GAO and others on how this
    data could be used, but nobody has really looked into whether
    agencies were putting theory into practice. A year or so ago,
    Senator Coburn and I asked the GAO to examine how performance
    information is being used to better manage Federal agencies,
    and if managers could be, and should be, employing it more
    often in their decisionmaking processes. We also asked the GAO
    to consider the recent efforts by the Administration to improve
    the usefulness of agency performance information and through
    initiatives such as the Program Assessment Rating Tool (PART),
    a key component of President Bush's Management Agenda.
        I am eager to hear GAO's findings and ask that we focus our
    discussion on the following crucial questions today. To what
    extent are Federal agencies using performance information to
    perform key management tasks, such as identifying performance
    problems and taking corrective actions or identifying and
    sharing best practices?
        Second, how can Federal agencies make better use of
    performance information to improve our results, and our
    outcomes?
        And finally, what lessons can be learned and how can we
    build on this Administration's efforts to improve the
    usefulness and employment of performance information?
        Today, with these questions in mind, I want us to do the
    following. One is to accurately assess how well and how often
    Federal agencies are using performance information to correct
    problems and promote best practices.
        Two, I would like for us to discuss how the use of
    performance information can be increased and improved in the
    next Administration.
        And three, I want us to solicit ideas about how Congress
    can play an effective role in the path forward.
        Today, it is my hope that we can begin to develop a
    blueprint for the next Administration. We must do all that we
    can to ensure that the transition is streamlined and that our
    next President has a clear picture of the strengths and
    weaknesses of agencies under his direction.
        We face, in this country, unparalleled challenges both here
    and abroad and these require a knowledgeable and nimble Federal
    Government that can respond effectively. With concerns growing
    about our mounting Federal deficit and national debt, the
    American people deserve to know that every dollar we send to
    Washington is being used to its utmost potential. Performance
    information is an invaluable tool that can do just that, if we
    use it. If used effectively, it can identify problems, help us
    find solutions, and develop approaches that improve results.
        We thank, again, not only our lead-off witness, but all of
    our witnesses for taking this opportunity, setting aside time
    in their own lives to be with us today to share with us some of
    their ideas about the challenges before us and how best to
    address those challenges.
        Our lead-off witness is Governor Martin O'Malley. He was
    elected Governor of Maryland in 2006, coming on the heels of an
    unusually successful tenure as Mayor of the City of Baltimore.
    Early in his life, as a young troubadour, he led his Irish
    group into the City of Wilmington and left the fans at O'Friels
    Irish Pub standing and cheering in his wake at an age when he
    was actually too young to get in legally to O'Friels, but came
    in and did a great job performing for us.
        In 2005, Time Magazine named him one of America's Top Five
    Big City Mayors. As Mayor, he pioneered the CitiStat, a
    statistics-based tracking system that focuses in on areas of
    under-performance and demands a results-driven government
    model. CitiStat has saved Baltimore residents more than $350
    million and was awarded Harvard University's prestigious
    Innovations in American Government Award in 2004.
        As governor, he has brought the program State-wide,
    implementing StateStat across all Maryland's government
    services, and provided a model not just for mayors, I think not
    just for governors, but I think maybe for Presidents and for
    those of us who serve here in the Legislative Branch of our
    government.
        We are honored by your presence and we are especially
    honored to be able to serve here on a daily basis with your
    mother, who works, as some of the people here know, for Senator
    Mikulski. I am not sure who works for whom, and I don't know if
    Senator Mikulski is actually here. She was going to try to stop
    by today. She may pop in. But if your mother is around and if
    she shows up, please introduce her to us. We are delighted that
    you are here and that your mom raised you so well. [Laughter.]
        Governor O'Malley, you are recognized to speak for as long
    as you wish.

       TESTIMONY OF HON. MARTIN O'MALLEY,\1\ GOVERNOR OF MARYLAND

        Governor O'Malley. Chairman Carper, thank you very much.
    ---------------------------------------------------------------------------
        \1\ The prepared statement of Governor O'Malley appears in the
    Appendix on page 49.
    ---------------------------------------------------------------------------
        Senator Carper. Thank you.
        Governor O'Malley. Thank you for the opportunity to be able
    to join you at this important Subcommittee and as part of the
    discussion of this day.
        I am also joined by a number of members of my staff.
    Apropos to today's discussion, Matt Gallagher is here who runs
    our StateStat office and is the Deputy Chief of Staff for
    Operations in our State government.
        Senator Carper. Would you ask him to raise his hand?
        Governor O'Malley. There he is.
        Senator Carper. All right.
        Governor O'Malley. He also before that ran the CitiStat
    office in the City of Baltimore before that.
        It is an honor to be here today before you to talk about an
    issue that, frankly, I believe is changing, for the better, the
    way that many of us look at the operations of our government,
    and it is our government. It is our belief that the same
    performance-based governing strategies that were so very
    valuable in igniting Baltimore's come-back and have been so
    valuable to us at the State level in Maryland this past year-
    and-a-half can also work not only in the Federal Government,
    but in any large human organization.
        In our public life, we tend to be very good at measuring
    inputs. We typically refer to those inputs as the budget, and
    it is typically done on an annual basis. But we have often
    neglected to pay enough attention to outputs, to the product of
    government. We are constantly asking, what is our funding
    level. For example, for something like interoperable
    communications without asking whether or not the purchases that
    have been made by local, municipal, or State governments this
    year have been in accordance with the standard that moves us
    down the road to a point in time when all first responders
    throughout our Nation can actually talk with one another when
    responding to an emergency.
        Performance-based government, in its essence, is about
    measuring, tracking, and improving outputs. Inputs play a role,
    but only in the pursuit of outputs.
        Mr. Chairman, I was first introduced to this model of
    governing about 8\1/2\ years ago when I began my first term as
    Mayor of the City of Baltimore. And when we were handed the
    keys to that 16,000 person, $2 billion a year operation known
    as city government, we inherited our fair share of challenges,
    some of them very big challenges. More than 300 of our fellow
    citizens every year were being murdered in our city. The
    streets were too often littered with trash. Our schools were
    too often failing and people were abandoning our city, in
    essence, voting with their feet, and leaving behind them
    buildings and homes that were vacant and themselves becoming
    nuisances.
        Quite understandably, the public at the time was demanding
    immediate results and immediate turn-around, and we rolled up
    our sleeves and got to work and found in beginning that work
    that there was very little in city government that was actually
    being measured in a consistent and real-time fashion. Now,
    don't get me wrong. Oftentimes, information was being collected
    at the ground level, sometimes very faithfully, sometimes very
    dutifully, not always the right information, but rarely, if
    ever, was it being used in a timely manner by the appropriators
    and the policy makers and the administrators at the highest
    level in order to deliver better outputs and better outcomes
    for citizens on the ground.
        So we were very blessed to have met a gentleman by the name
    of Jack Maple, who was one of the great minds behind the
    performance-based strategies employed in the New York City
    Police Department for its turnaround. Jack was the Deputy
    Commissioner of Police under Commissioner Bratton and we felt,
    having observed ComStat in action, that if the NYPD could so
    successfully use simple off-the-shelf software, computer pin
    mapping, deploying police resources to where the crime was
    actually happening, that data collection and mapping technology
    could also work for the other things that government does,
    whether it is garbage collection or repairing streetlights or
    addressing complaints about potholes.
        And from this approach was expropriated the four main
    tenets of CitiStat, which were the main tenets of ComStat. One,
    timely, accurate information shared by all; two, rapid
    deployment of resources; three, effective tactics and
    strategies; and four, relentless follow-up, not on an annual
    basis, not on a biannual basis, but on a daily, weekly,
    biweekly, monthly basis in order to improve performance.
        So we started setting goals. We started measuring results.
    And we did so weekly. We began tracking outputs instead of just
    tracking inputs and we started geomapping every conceivable
    service. And in short time, we turned around the city where
    many neighborhoods were considered ungovernable and we started
    making our city government function again in order to improve
    the quality of life in every neighborhood.
        I go into some greater detail about some of the results we
    achieved in the written statement I submitted to the
    Subcommittee, but just a few examples. Most important of all is
    the primary responsibility of all governments and that is
    public safety. We were able to achieve a 40 percent reduction
    in violent crime, its lowest level, actually, in four decades.
    We were able to back up with 98 percent success a 48-hour
    guarantee to address complaints from citizens about potholes.
        We reduced the number of children exposed to dangerously
    high levels of lead from lead dust, lead paint poisoning from
    old homes and deteriorating homes by 65 percent in a relatively
    short period of time.\1\ We were able to identify and reclaim
    by clearing title more than 5,900 vacant homes and buildings
    which then enables them to be redeveloped and put back on the
    tax rolls.
    ---------------------------------------------------------------------------
        \1\ The chart referred to appears in the Appendix on page 138.
    ---------------------------------------------------------------------------
        We had a boarding and cleaning backlog of about some 8
    months, which is how long it took when we began to address a
    boarding or cleaning complaint from any citizen. By the time we
    left, that was down to 14 days. Now, that didn't happen
    overnight. It didn't happen by measuring things annually. It
    happened by measuring them every single day and every week.
        And probably the most important outcome of all is that we
    were able by improving our quality of life with better-
    performing government to be able to reverse four decades of
    what had been seemingly insurmountable population loss, and the
    city started growing again.
        I brought two charts with us today. One of them on the far
    right is the combination of homicides, shootings,\1\ and you
    will see three kinds of death, and you can see them over time.
    Again, these are just measured annually, but we measured them
    every 2 weeks and, by golly, when you look over your shoulder,
    you see you are actually making progress shrinking those danger
    zones in our city.
    ---------------------------------------------------------------------------
        \1\ Chart referred to appears in the Appendix on page 136.
    ---------------------------------------------------------------------------
        This next map is the cleaning and boarding measured not in
    terms of the reduction of the wait time but in terms of the
    improvement of the productivity.\2\ Again, these are annual
    outcomes, but the only way we were able to achieve the annual
    outcomes is because we developed systems so that we could
    measure every day, every week, and that information was then
    able to get back up to the policymakers, the administrators,
    the appropriators.
    ---------------------------------------------------------------------------
        \2\ Chart referred to appears in the Appendix on page 137.
    ---------------------------------------------------------------------------
        In 2007, when we were given the keys to an even larger
    organization, namely the State government of the great people
    of Maryland, we took this model with us. We created a program
    called StateStat, which has allowed us to track and measure
    progress on a level that we have never before been able to
    achieve in our State government. And through our StateStat
    program, while only in its first year and a half, we have been
    able to identify problems in information sharing among various
    law enforcement agencies and across several jurisdictions and
    we work every day to improve them.
        Second, we were able to identify right off the bat the
    imperative, the need to close the House of Corrections, an old,
    dilapidated, and very dangerous prison in our State. We were
    able to close that without incident within 50 days, saving
    probably lives and also saving $3 million in overhead and
    overtime.
        We were also able to identify and fix more than 100
    problems within our juvenile detention facilities, many of
    which had been under various consent decrees and court orders
    for a long time without ever improving those conditions.
        We are now using GPS technology and performance measures to
    target our resources that are geared at restoring the national
    treasure, which is the Chesapeake Bay, through a program that
    we call BayStat. You may notice that virtually anything, if you
    measure it, you can slap a ``stat'' on the end of it and you
    have a new way of saying whether what you are doing is making
    any impact on the problem you are trying to solve.
        We go through a few more accomplishments in the written
    statement I have submitted to you, but I wanted to close by
    just saying a couple of words about CitiStat, StateStat, and
    why this model, I believe, can be and should be applied at the
    Federal level, particularly in the realm of homeland security.
        We believe the same approach can be relevant to governments
    and organizations anywhere and of any size. Recently, I had
    occasion to meet and to listen to Sir Michael Barbour from Tony
    Blair's government who applied many of these principles to the
    administration of national government in the United Kingdom and
    also implemented a new innovation called the delivery unit to
    make sure that all of those along the chain of delivery, from
    policy maker to appropriator to administrator to implementor
    down to the level of citizen, were actually held accountable
    for their piece of delivering improved performance.
        The Environmental Protection Agency, I understand, has
    launched an EPAStat program, and there are governments
    worldwide that are working to implement this model. In fact, we
    have had several delegations from governments all over the
    world who have come to visit us in Maryland to learn about the
    workings, the early workings, of our program of statewide
    performance measurement.
        Government performance management, I believe, is really a
    nonpartisan issue. There is no Democratic or Republican way to
    fill a pothole, to make sure that you improve the outcomes,
    whether it is at a municipal, State, or National Government.
    The beauty of a map is that a map doesn't know whether a
    neighborhood is black or white, or whether a neighborhood is
    rich or poor, or whether a neighborhood is Democratic or
    Republican.
        Most of us in government would say that it is the
    responsibility of every public official to provide the most
    effective government possible and to provide for continued
    progress and improvement. That is what CitiStat and StateStat
    are really about.
        Mr. Chairman, I want to close today with the words of
    Robert Kennedy, who once said, ``There is no basic
    inconsistency between ideals and realistic possibilities, no
    separation between the deepest desires of the heart and of the
    mind, and the rational application of human effort to human
    problems.'' The rational application, well and timely measured,
    of human effort to human problems.
        That is what performance-based government is about, and I
    thank you so very much for your interest and your leadership in
    bringing this to our Federal Government. Thanks very much.
        Senator Carper. Governor, thank you very much.
        One of my finest memories of being governor was going to
    New Governors School every 2 years, right after the election.
    About the middle of November, current governors, would host the
    newly-elected governors and spouses and teach the new governors
    and spouses by virtue of our own experience the things we had
    done wrong, in many cases, and some of the things we had done
    right.
        We had a Center for Best Practices within the National
    Governors Association. I suspect we still do. I am just
    wondering, does your experience with CitiStat and what you are
    doing with StateStat, does any of that show up within the
    National Governors Association, either at a New Governors
    School like a forum, or through the Center for Best Practices
    so that other States can learn from what you have done, both in
    Baltimore and in Maryland?
        Governor O'Malley. Yes. To some degree, other States have
    begun this in one way, shape, or form, and some of them started
    doing it years ago on the heels of seeing the success of the
    NYPD. In fact, Christine Gregoire, who began her first term as
    Governor of Washington State, visited us, actually before all
    the votes were totally counted in that close race----
        Senator Carper. I said to Christine, who had been Attorney
    General, I said, keep counting the votes and recounting the
    votes until you win. Then stop. [Laughter.]
        That is what they did.
        Governor O'Malley. Well, during the period of pre-
    transition, she sent a group of her people to Baltimore and has
    actually begun and does have a performance measurement program
    in Washington State. I know I have had conversations with
    Kathleen Sebelius in Kansas back at the time, and I believe she
    applied it to some things, as well.
        It is, I believe, part of the National Governors
    Association, part of their Office of Best Practices, and we
    were able to recruit from there not only Malcolm Wolf, our
    Energy Administrator with whom you had some discussions, but
    also our policy directors from that area. The best ideas are
    the ones that you find from other practitioners. That is what
    we believe.
        One of the exciting things about beginning this program in
    Baltimore years ago is that other cities have taken it up, and
    I have found myself learning from things that Mayor Cicilline
    in Providence was doing to apply this to juvenile justice, to
    be able to learn from Mayor Slay in St. Louis the things he was
    doing on lead paint abatement and the like.
        So I hope, and I believe, that there will be more of this
    going on at the State level that we can learn from.
        Senator Carper. In Baltimore, I presume this initiative was
    something that you promoted as mayor, and as governor, I
    presume that this is an approach that you have been championing
    in your State.
        Governor O'Malley. Absolutely. One of the things I have
    noticed, Senator, is that newly-elected executives tend to have
    an easier time embracing this than those of us who have been in
    office for some time. The very uncomfortable part of this
    process is that when you begin measuring things that have never
    been measured before and start sharing that information widely
    and broadly, everyone comes to understand how poorly many
    things had been functioning in the past. The fresh start of a
    new executive is, I think, something that enables an
    organization to make that sort of culture shift in ways that
    others find more difficult.
        It has also been my experience that however uncomfortable
    that initial period is of the openness and the transparency,
    which are the hallmarks, I think, of performance measurement
    and a republican form of government, certainly, that openness
    and transparency over time pays tremendous dividends, however
    embarrassing the up-front moments are of, oh my goodness, I
    didn't know that we were that bad at that particular service
    delivery. Over time, people come to respect and appreciate it.
        Another innovation that we borrowed from Mayor Richard
    Daley in Chicago was the use of the 311 center for all city
    services. In the past, we would go around knocking on doors at
    campaign time and giving out the ``where to call for help''
    card. The ``where to call for help'' card had some 300 various
    services listed alphabetically. Look under V for vacant houses.
    Look under R for rat abatement. And then we would have 300
    different phone numbers.
        The 311 system on the front end and being able to have a
    call courteously answered and give every citizen, regardless of
    what neighborhood that the call came from, being able to give
    them a customer service number that was common to all citizens
    and a time frame within which to expect that service, whether
    it was a broken curb, a pothole, dead tree, or what have you, a
    time frame within which to expect that service to be delivered
    was something that gave our citizens a lot of optimism about
    the future and the fact that they still do, in fact, have the
    reigns of controlling and holding accountable that government
    into which they pay their hard-earned dollars.
        Senator Carper. Well, you are going to have a whole crop of
    new governors being elected this November. A couple of weeks
    after that, they are going to show up at the NGA School for New
    Governors and Spouses and they are going to be looking for
    things, ways to provide good services, working within the
    constraints of tight budgets, as you know. I suspect that you
    can provide a real service for them as you are here today in
    sharing with them your successes in Baltimore and in Maryland.
        Actually, talking about taking good ideas, we used to steal
    the best ideas from one another. Sometimes you would attribute,
    sometimes not. But we have taken the ideas of New Governors
    School and we have actually incorporated it here in Washington.
    In the middle of November, 2 weeks after the election, we will
    have, I call it New Senators School, but it is actually this
    orientation for new Senators and their spouses, 3 days, very
    much like NGA. Faculty are current Senators and spouses, and
    basically learn from one another.
        Governor O'Malley. And I trust that they put former
    governors at the front of the class?
        Senator Carper. Actually, three of the people who started
    it, Lamar Alexander, former governor, former NGA Chair from
    Tennessee, George Voinovich, former governor, former NGA Chair,
    Mark Pryor has very active as Attorney General from the State
    of Arkansas, and yours truly. So three out of the four are all
    governors.
        In Baltimore, you obviously have a city council,
    Legislative Branch. As governor, you have a General Assembly,
    Legislative Branch. Here, we have the Legislative Branch in
    which I am privileged to serve. What role in city, State, and
    maybe in the Federal Government, the National Government, can a
    Legislative Branch play? How is this relevant in the lives of
    the Legislative Branch in the city, State, and in Washington?
        Governor O'Malley. Initially, like many new ideas, it was
    greeted with a tremendous amount of skepticism and----
        Senator Carper. By your legislators?
        Governor O'Malley. Yes. Let me talk first on the city
    council and then I will talk at the State level. In the city
    council, the concern was that this was going to undermine or
    somehow diminish the ability of council members to deliver for
    the constituents who called them for a variety of city
    services, and so over time, we were able to overcome that,
    primarily because of the openness and the transparency of the
    process and also the ability of city council staff to be able
    to log into the same system that our 311 operators were able to
    log into in order to give out those citizen service complaints
    and the time frame within which to expect that those services
    would be delivered.
        And once the council staff were trained in it and they went
    back and worked on their council members, a lot of the fears
    dissipated because everybody wants to be able to produce.
    Everybody wants to know that when their constituent calls, that
    they will be able to pick up the phone and deliver.
        On the State level, we recently created a new fund for the
    restoration and health of the Chesapeake Bay, work that has
    been going on for some time, work that needs to be accelerated
    and improved and--the tendency, I think, in most legislative
    bodies is to specifically designate any new dollars that are
    appropriated for a given purpose, namely the cleanup of the
    Chesapeake Bay. In a big public health challenge like the
    Chesapeake Bay, there are probably at least 100 different
    activities that could be funded that contribute to the health
    of the Chesapeake Bay, everything from stormwater upgrades to
    cover crops to expanding the forested buffer along streams.
        The legislature, because of the openness, because of the
    transparency, and because of the performance measurement that
    they saw in BayStat and their belief, their well-founded hope,
    I think, that the deployment of those dollars will be guided by
    the best science and the most effective use of those dollars,
    they chose after some deliberation not to specifically
    designate this first $2 million shall go to this. This next $3
    million shall go here to that.
        I think that legislators that have seen the beginning of
    this process are encouraged that they will be able to get more
    timely, accurate reports on the things that are working, the
    things that are not, which then will make them much more
    effective in exercising their oversight of the things that we
    do fund, the things that we maybe should fund more of, and the
    things that perhaps we should not be funding as much of.
        Senator Carper. In our State, and certainly in your State
    and the other 48 States, we have spent a lot of time and energy
    and money on raising student achievement. Most of the States
    have established academic standards of what we expect our kids
    to know and be able to do in math, science, English, social
    studies, and other curriculum. We designed these tests to
    measure student performance to try to figure out how to hold
    schools accountable, students accountable, educators
    accountable, even parents accountable. We have created enormous
    amounts of information from the results from our tests. Some
    States give tests on an annual basis to selective grades. Now
    more and more, we are giving at least annual tests to a broader
    cross-section of students and grades of students.
        In my own State, we have a program called Principal for a
    Day. The State Chamber of Commerce does it in conjunction with
    our school districts. We have a little over 200 public schools.
    It is a program I participate in almost every year, so last
    year, I was Principal for a Day at a school called Stanton
    Middle School, about 10 miles outside of the City of
    Wilmington. They had been taken under the wing of Selbyville
    Middle School. Selbyville is in southernmost Delaware, almost
    in Ocean City.
        Selbyville Middle School had figured out how to raise
    student achievement in ways most middle schools have not, and
    the results were dramatic and impressive and sustained, and the
    folks at Stanton had been struggling to figure out, what are we
    doing? What are they doing right? How can we learn from them?
        What they found out is that the folks at Selbyville Middle
    School in Southern Delaware had figured out how to, not to
    teach to the test, but to say these are our standards. We want
    our teachers to teach to the standards. Then we will have a
    test and the test will reflect the standards.
        But the school--Selbyville was able to figure out how to
    use the information gained from the tests to change the way
    they taught, to change the way they allocate resources, to
    change the way they work with individual students. Stanton
    Middle School last year--in fact, this year, was selected as
    one of the best, most improved schools in our State, and what
    they learned to do was to take the information and to use that
    to change behavior, not just of the students, but of the
    educators and the administrators, as well.
        Have you had the opportunity in your State to take this
    approach of performance information and using that to manage
    for better results and to put it to work in your schools?
        Governor O'Malley. To a degree. We attempted with some
    success to do this in the City of Baltimore and our schools
    there and it was embraced to a degree, especially when it came
    to some of the management functions of the school district. The
    school district in our State that has embraced the notion of
    performance management, performance measurement most
    wholeheartedly is Montgomery County under Dr. Weast, and I have
    had occasion to observe his SchoolStat meetings where he brings
    in principals, and oftentimes it is, as you would suggest, a
    principal from one of the higher-performing schools and a
    principal from one of the more struggling schools, and just as
    governors and mayors learn from one another, principals learn
    from one another the things they are doing and the things that
    work.
        Montgomery County has done it the best and the most
    wholeheartedly. Other school districts, I hope will follow on.
    I was reminded in listening to your story about the two school
    districts in Delaware, one of the leading governors in America
    on education reform is Mike Easley of North Carolina----
        Senator Carper. And he followed another great governor,
    Governor Hunt.
        Governor O'Malley. A good man.
        Senator Carper. Oh, a terrific education governor.
        Governor O'Malley. I once heard him say, and it is true of
    all performance measurement, not just in schools but whether
    you talk about the potholes or the boarding or the other
    environmental things, he said, ``You are not going to improve
    the weight of the pig just by constantly weighing it.''
        Applying the standards and the performance measurement is
    important. Setting the goals is important. But along with that
    go the other tenets of the rapid deployment of resources,
    having real serious dialogue, conversation, and some
    experimentation to determine which tactics are effective, which
    strategies are effective, and having the guts to jettison some
    and embrace others, and doing so in a relentless follow-up way.
        There is always a temptation that if we wave the magic wand
    and simply weigh the pig more constantly, that somehow that is
    going to improve the weight of the pig, but the inputs matter.
    How you manage those inputs, where you deploy those inputs do
    matter. But the only way to determine that is by being wide
    awake and aware and having that dashboard that measures the
    performance over time----
        Senator Carper. All right. Thank you.
        I think there is a great opportunity in my State to build
    on what has happened at the two schools that I have mentioned.
    There are others, as well. But we are thinking about changing
    the way we actually administer our tests. Instead of being a
    test that is given out, pen and pencil test given once a year
    to a third grade, to a fourth grade, to a fifth grade, to have
    a computer-assisted test so the students could actually do the
    test from a computer. They would get the results almost
    immediately.
        And the test is something that if it is happening in the
    third grade and they are performing at a second or a first
    grade level, or a fourth or fifth grade level, that would be
    garnered and it would be a better tool to help the teachers and
    educators in that school to work with an individual student,
    and it is a test that could be given not just once a year, it
    could be given several times during the course of a year, and
    as you watch somebody who is an under-performer move from high
    first grade performance to low second grade, to middle second
    grade toward lower third grade during the course of a year, you
    actually see that progress.
        So we think about changing No Child Left Behind so there is
    more--it is there not to discredit or embarrass school
    districts, but to help and to really be a way to help move
    student performance. There is real potential here for applying
    what you have done in your State and city, I think.
        Governor O'Malley. May I say a word about the testing and
    the inputs? One of the real benefits that we have seen from the
    standardized tests over the last 7 years is the positive
    outcomes in achievement that followed almost immediately on the
    heels of a tremendous investment in full-day kindergarten for
    all children throughout our State.
        When I was first elected Mayor of Baltimore, not one of our
    grades scored majority proficient city-wide in reading or math,
    not one. Within the very first year after the first cohort of
    children in our city, which has a lot of challenges with
    poverty and adult literacy and other things, we saw the first
    graders score a majority proficient in reading and math, and
    some said, well, that was just a blip and once they are longer
    in the system, they will go down.
        Well, quite to the contrary. As we look over our shoulders
    now from that point in time, that cohort has not only
    maintained its level of achievement, but every class and cohort
    of kids after them have come up to that rising level and it
    showed us that the investment that we made--we went from about,
    I think, $3.2 billion as a State to roughly $5 billion in State
    aid to local education, a big part of which was that full-day
    kindergarten, and we have seen that investment has brought
    about these results, including a narrowing of the achievement
    gap between minority--so-called minority students and non-
    minority students.
        High expectations are very important and I do believe that
    expectations become behavior if together we make the right
    choices to make it so. But the testing alone, I think the
    biggest part of the improvement of test scores in our State and
    certainly in our city was a result of the big investment, not
    necessarily a result of more tests.
        Senator Carper. By having the testing results, when the
    testing results--actually, you trying to find out why are these
    kids doing better----
        Governor O'Malley. Right.
        Senator Carper [continuing]. In a particular school
    district, and then you look at the number of kids that maybe
    have the opportunity for full-day kindergarten, or you look at
    the kids in the school districts where they actually have an
    opportunity to get decent pre-K training, you are right. In my
    State and your State and across the country, one of the best
    indicators of better performance is quality----
        Governor O'Malley. We are also about to borrow another idea
    from Governor Easley, the teacher surveys and learning from
    what the teachers are telling us----
        Senator Carper. Excellent.
        Governor O'Malley [continuing]. Almost in a sort of
    corporate 360 approach.
        Senator Carper. That is great. A couple questions and we
    will let you escape. Let me just ask, how do you, as chief
    executive of your State, convey your own personal commitment to
    addressing specific performance issues to the agency staff, the
    folks that are responsible for bringing about improvement? How
    do you do that?
        Governor O'Malley. We do it through--you mean actual means
    of keeping engaged as the executive in this process, Senator?
        Senator Carper. Is this something you talk about in your
    State of the Union? Is it something that is reflected in your
    budget? How do you convey to the people that work in your
    agencies, not just the cabinet secretaries, not just the
    division directors, but also to rank-and-file employees why
    this is important, why you think it is working, why they should
    do it?
        Governor O'Malley. The relentlessness of our meetings--I
    was in a meeting today, for example, with the folks from our
    Public Safety Department, and in our State, Public Safety
    includes not only the prisons, but also parole and probation
    and all the agents that go into the sort of community
    supervision of people on the street. And at that meeting, there
    were probably some 30 or 40 people present. Now, not everybody
    talks at the meetings, but we like to think everybody listens
    at the meetings.
        The fact that at the meetings we have not only an executive
    summary that is distributed to all parties beforehand, but we
    also have minutes that are taken and sort of an after-action
    report as to the things that we need to prepare for by the next
    meeting, the questions that need to be answered by the next
    meeting. That is sort of our method, and the fact that we are
    relentless about it and that it is not subject to my schedule.
    I don't go to all of them, but they always happen. And the fact
    that they are relentless and that the follow-up always happens,
    and fortunately, I have dedicated people like Mr. Gallagher and
    Michael Enright that continue to drive that train regardless of
    whether I am pulled out of town or to important committee
    hearings in Washington.
        Senator Carper. I understand. Well, good. How important do
    you think it is to recognize good performance when it does
    occur?
        Governor O'Malley. Oh, critically important. A lot of times
    when people write about this and sometimes when journalists
    come to observe a session, they focus in on how uncomfortable
    these sessions are for the under-performers. The more
    significant change, I think, comes from the executive and
    leadership recognizing leadership at the department level, at
    the implementor level.
        Jack Maple had this theory of progress in a big
    organization. He said 80 percent of the people are in the
    middle, and then on one side are the leaders, and on the other
    10 percent of the bell curve are the slackers. And if
    leadership and the executive recognize the leaders and the
    achievers, the whole organization tilts towards the leaders and
    the achievers. It improves morale. It improves outputs. It
    improves progress. It improves effectiveness. And that is what
    we try to do.
        So in our city, the methods we used for such things were, I
    don't know, something as simple as a thank you note when the
    crews hit the 48-hour pothole guarantee. It seems like a little
    thing. It was probably the first time they were ever able to go
    home to their spouses and say, ``Hey, look what the Mayor sent
    to me.'' Sometimes we would give tickets to Oriole games or the
    Mayor's Box at the Ravens game or other things, and we would do
    it in front of people at these events.
        And the people that are the leaders at one level of
    government were also the people that we promote in the course
    of time that hopefully imbues a sense of meritocracy to
    governments that quite honestly in the past, decisions are
    often made for reasons having little to do with the level of
    achievement, the leadership, or the industry or hard work of
    the individuals.
        Senator Carper. You mentioned the Ravens games. The top
    draft pick for the Ravens at the NFL draft this year, they
    picked a quarterback from the University of Delaware, Joe
    Flacco. In Delaware, people tend to follow the Ravens or we
    tend to follow the Eagles. We will see how it turns out.
        One last thing. Six months from, it will be January 20. I
    believe that is the day we swear in a new President and Vice
    President. The Congress will be sworn in on January 3. So we
    are going to have in the space of a couple of weeks 435 new
    members, just sworn, newly minted House members and about a
    third of the Senate will be newly sworn in.
        If you were going to give that new President and new Vice
    President, and maybe even the folks who serve in the
    Legislative Branch, just some friendly advice on how we can
    take your experience in Baltimore and your experience in
    Maryland and apply it to provide better service, better
    results, and some things that are important for all of our
    States and for our country, what advice might that be?
        Governor O'Malley. I think the most important advice is
    that beyond the guts and the courage that it takes to set goals
    and beyond the trust in the public that it takes to create a
    system that is open and transparent and performance measured,
    the third really important part of this sort of performance
    management is executive commitment.
        This cannot be something that an executive at any level--
    municipal, county, State, national--does for the sake of a one-
    day press conference or even for the sake of rolling into a
    budget address or a State of the Union. It requires a
    tremendous amount of executive commitment and relentless
    follow-up by that man or woman that is in charge of that
    government, because if there is not the commitment at the
    executive, it is very hard for it to transform that pyramid of
    command and control that is a big Federal Government.
        So those are the three things, Mr. Chairman. Have the guts
    to set goals. Have enough faith in the people we serve to
    create a system that is open, transparent, and measured for
    results. And third, stay committed as an executive to driving
    it every day and to never waver from that commitment to
    openness, transparency, and performance measurement.
        Senator Carper. Good. Among the new governors who will be
    elected this year will be the Governor of Delaware, my State
    and your neighbor. The thought of coming in with a new chief
    executive, and as good as we think we are, we always know we
    can do better. But my hope is that our new governor will reach
    out to you and to see what he can learn from the experiences
    that you have had and the progress that you have made in your
    own city and your own State.
        One of the things that we are doing--we talked about this
    last week and again today--one of the things that we are trying
    to do in Delaware is endeavor to become the first State to
    deploy an offshore wind farm about 12 miles off the coast of
    Rehoboth Beach, as you know. Our hope is that we can find some
    partners to our south and to our west and maybe to our east and
    to our north who might like to partner with us. We talked a
    little bit about this, but my hope is that you and that
    Maryland might like to be a partner, at least explore that
    possibility with us.
        Governor O'Malley. Well, we are very interested in that. We
    have the renewable portfolio standard that was just updated,
    upgraded, increased by our legislature in, I believe, the most
    recent General Assembly session. We have a number of entities
    that are working on renewable sources and hats off to Delaware
    for leading the way on that offshore wind. I hope once we work
    through our request for proposals process, we will be able to
    join in that. We still have a long way to go, but I think it
    makes a whole lot of sense.
        It is certainly needed as we move ourselves into more
    sustainable energy future, and we do so as a region. In fact,
    that might be the only way that this is able to be
    accomplished, or at least that could well be the leading edge
    of that progress, is when States join together and create the
    critical mass necessary to prime new endeavors like the wind
    project off Delaware's coast.
        Senator Carper. A couple of years from now, people will
    look at Rehoboth Beach on a clear day like today, and if they
    can look out 12 miles to the east, they will see sticking above
    the horizon some small objects about the size of a thumbnail
    that will be 60 or 70 windmills that collectively will provide
    about 15 percent of our electricity needs in Delaware. We
    believe, and I think you do, too, that to the extent that we
    can have more turbines and look for economies of scale, it is
    not just advantageous to us, but to our neighbors, as well. So
    I am encouraged with what you have said.
        Governor O'Malley. I am very interested.
        Senator Carper. Last, going back to football again for just
    a moment, I will close with this.
        Governor O'Malley. I reserve the right to turn to staff.
    [Laughter.]
        Senator Carper. Vince Lombardi, that legendary head coach
    of the Green Bay Packers, used to say that unless you are
    keeping score, you are just practicing. And it occurs to me
    that they didn't just do more than keep score at Green Bay all
    those years that they played so well, but what they did is they
    kept not just score, but they kept the results from how each of
    the players did, the plays that worked, the plays that didn't,
    and then they used that to change their game plans and really
    to change their defense and their offense. They actually
    applied the information that they collected.
        We are pretty good in government in keeping score, using
    the football analogy, how are we using the score that we are
    keeping, how are we using those scores and the information that
    we gather to actually change the plays that we call at the line
    of scrimmage, the plays that we call in the huddle, the way we
    deploy our defense or our offense, the kind of players that we
    draft? It actually does apply. It applies in the world of
    football and it applies very much in city government, State
    government, and I think in the Federal Government.
        You have been just a stand-out on this front, and I know
    the people of Baltimore have recognized that and rewarded you
    for that and the people of Maryland have done so, as well.
    Thank you so much for being here and helping to show us the way
    as a Nation, and we look forward to hearing our second and
    third panels, as well, but you have gotten us off on the right
    start.
        Governor O'Malley. Mr. Chairman, thank you very much.
        Senator Carper. Thank you.
        Governor O'Malley. To the extent that we have been
    successful, it is because of the people we serve and the good
    people that I have been blessed to be able to work with in
    public service. Thank you very much.
        Senator Carper. Thank you.
        Governor O'Malley. Thanks for your leadership.
        [Pause.]
        Senator Carper. Before I introduce our witnesses on our
    second panel, I want to take just a moment and thank all of our
    witnesses, from certainly the governor and this panel and our
    third panel, as well, for doing something that doesn't happen
    every day, and that is for providing all of your testimony on
    time. You are to be commended for doing that and we are
    grateful. It is a notable achievement.
        Our three witnesses today, I will start off by providing
    just a little bit of background. Bernice Steinhardt is Director
    for Strategic Issues at the Government Accountability Office,
    where she is responsible for examining government-wide
    management issues. For over 9 years, she has led the GAO's
    efforts in strategic planning and helped to develop the
    organization's first strategic plan. She has held a number of
    positions at GAO, including Director of Public Health Issues
    Group and Associate Director for Energy, Natural Resources, and
    Science Issues, as well as Environmental Protection Issues.
        Before joining GAO, she served at the Department of the
    Interior at the President's Council on Environmental Quality.
    Who was the President at that time when you were on the Council
    on Environmental Quality?
        Ms. Steinhardt. The first time I was there, the Chairman
    was Russ Peterson and the President was Gerald Ford.
        Senator Carper. You were just a child. Were you an intern
    then?
        Ms. Steinhardt. You are very kind.
        Senator Carper. Russ Peterson is still alive and well.
        Ms. Steinhardt. Is he?
        Senator Carper. He is 91 years old and he was a Republican
    then, now he is a Democrat, but I think he was always a
    Democrat if you want to know the truth.
        Ms. Steinhardt. Well, he was an environmentalist, I can say
    that.
        Senator Carper. He is a strong environmentalist. But he is
    still just doing incredible things at the age of 91. What an
    inspiration.
        Ms. Steinhardt. He was a wonderful Chairman.
        Senator Carper. He is one of my mentors.
        Ms. Steinhardt. Mine, too.
        Senator Carper. Good. Thank you.
        Marcus Peacock is the Deputy Administrator of the U.S.
    Environmental Protection Agency, where he currently is focused
    on improving EPA management systems. Previously, he served as
    an Associate Director in the U.S. Office of Management and
    Budget, where he was responsible for making budget decisions
    encompassing $160 billion in spending at various Federal
    agencies. While at OMB, he created the Performance Assessment
    Rating Tool (PART), which is used to rate the effectiveness of
    Federal programs.
        We want to thank Mr. Peacock for really filling two roles,
    wearing two hats here today, one speaking on behalf of EPA and
    also providing some insight into his former work at OMB. We had
    invited Clay Johnson of OMB to be with us today, but
    unfortunately, he is out of the country and OMB has kindly
    provided testimony for the record from Robert Shea and we look
    forward to including it as a part of our hearing today.\1\
    ---------------------------------------------------------------------------
        \1\ The prepared statement of Mr. Shea appears in the Appendix on
    page 131.
    ---------------------------------------------------------------------------
        We are really grateful that you are here, Mr. Peacock.
        Dr. Don Kettl is Director of the Fels Institute of
    Government and the Robert A. Fox Leadership Professor at the
    University of Pennsylvania, where he specializes in management
    of public organizations. We consider him a neighbor, too,
    because he lives only about 30 miles up the road from us in
    Philadelphia.
        In 2008, he was awarded the John Gaus Award for a lifetime
    of exemplary scholarship in political science and public
    administration. He is the author or editor of a dozen or so
    books and has won twice the Louis Brownlow Book Award for the
    best book published in public administration. His newest book,
    due out in November, is entitled ``The Next Government of the
    United States: Why Our Institutions Fail Us and How to Fix
    Them.'' He has consulted for governments at all levels, both
    here in the United States and abroad. Dr. Kettl is currently a
    nonresident Senior Fellow at The Brookings Institute and is a
    regular columnist for Governing Magazine.
        We are delighted that all of you are here. Your entire
    statement will be made part of our record. You are welcome to
    summarize that, and I would ask you to stick close to 5
    minutes, but we won't quarrel if you go a little bit beyond
    that.
        Ms. Steinhardt, I would welcome your leading this parade.
    Thank you so much for coming.

      TESTIMONY OF BERNICE STEINHARDT,\2\ DIRECTOR FOR STRATEGIC
             ISSUES, U.S. GOVERNMENT ACCOUNTABILITY OFFICE

        Ms. Steinhardt. Thanks very much for having me. I am
    pleased, in turn, to be here today, especially in such
    distinguished company, and to share with you some of the
    lessons that we have learned in examining the progress that the
    Federal Government has made in managing for results.
    ---------------------------------------------------------------------------
        \2\ The prepared statement of Ms. Steinhardt appears in the
    Appendix on page 55.
    ---------------------------------------------------------------------------
        Starting at least 15 years ago with the Government
    Performance and Results Act, the Congress and each
    Administration have put into place a framework of strategic
    plans, performance measures, and reports that together have
    heightened focus on performance and accountability for results.
    So today, I would like to talk about where we now stand and to
    suggest some actions that the next Administration can take to
    sustain and build on what we have accomplished so far.
        Since 1997, GAO has conducted four surveys of Federal
    managers across the government in order to gauge the extent to
    which a performance culture has taken hold. Clearly, there has
    been progress compared to what we saw in our first survey, and
    I would call your attention to the chart up top there.\3\
    Significantly more Federal managers that we surveyed in 2007
    reported having performance measures for their programs, which
    is to say the basic information that they need to judge how
    well their programs are working. The blue bars are the 2007
    results and the white bars, which you can see outlined there,
    are for 1997. So there is quite a bit of difference.
    ---------------------------------------------------------------------------
        \3\ Chart referred to appears in the Appendix on page 137.
    ---------------------------------------------------------------------------
        But in order for this information to make a difference, it
    has to be useful and used. To borrow Governor Easley's term,
    this is about weighing the pig. How are we making changes in
    the pig's weight? So the second chart shows that there are
    still large percentages of managers who aren't using
    performance information to a great extent in making decisions,
    like setting priorities or allocating resources.\1\
    ---------------------------------------------------------------------------
        \1\ Chart referred to appears in the Appendix on page 137.
    ---------------------------------------------------------------------------
        In fact, there has been significant change in only two of
    these nine areas, the ones that are highlighted there. One is
    an increase--and this is a good thing--in the use of
    performance information to reward employees. But at the same
    time, we also see a decrease in the use of performance
    information for making decisions about new program approaches
    or work processes, which does give us some concern, this
    decline, because this is where you would want to see managers
    using information to change direction when performance is not
    up to the standards.
        From our earlier work, and these latest survey results
    confirm this, we know that there are a number of practices that
    could spur some movement in these relatively static trend lines
    and lead to more widespread use of performance information.
    First and perhaps most importantly, and Governor O'Malley, I
    thought, underscored this very clearly, the agency leadership
    needs to demonstrate its commitment to achieving results. If
    the leaders don't care or if they don't show they care, then
    managers are not going to pay much attention, either. In fact,
    our survey results only underscore this report because those
    managers who reported using performance information to a
    greater extent also reported that their leadership showed a
    commitment to using the results of that information.
        Second, agencies need to link organizational results to
    individual performance. That is a kind of line of sight from
    the organization's results to individual performance. And here,
    performance management systems can be used to let employees
    know how they can contribute to results and then to hold them
    accountable for doing so.
        Third, agencies need to make sure that they have the
    capacity to collect and to use performance information.
    Managers aren't going to use the information if it is not
    timely, if it is not relevant, if it is not accurate, if it is
    difficult to use, or if they simply don't know how to use it.
        Promoting these practices within agencies ought to be a
    first step for the next Administration. Beyond this, though, we
    would advise a few others, again based on insights that we have
    gained from our work over the years.
        First, we would urge a more strategic and cross-cutting
    approach to overseeing performance across government. Many of
    the challenges that government faces today, whether it is
    homeland security, emergency response, climate change, all of
    them involve multiple agencies and programs. While this
    Administration's major performance improvement initiative, the
    PART process, has brought heightened attention to the
    performance of individual programs, we also need an approach
    that encompasses multiple programs and provides a more
    integrated view of what government is accomplishing.
        Second, performance information has to be more useful to
    the Congress. Whatever performance initiatives the next
    Administration adopts, the Congress should be engaged in
    helping to identify meaningful measures of success as well as
    the form in which the performance information will be useful to
    Congress in its oversight, legislative, and appropriations
    role, each of which could require different types of
    performance information.
        Finally, OMB can do more to build agency confidence in its
    performance assessments. OMB deserves full credit for its
    leadership in fostering a performance culture across
    government, but as our survey data suggests, many Federal
    managers still question the quality of the PART assessments.
    The more confidence that managers have in the quality of the
    assessments, the less they perceive them as paperwork
    exercises, the more likely they are to use the results.
        I want to close by noting that our study for you that is
    continuing, and we hope to be reporting back here early next
    year on additional actions that some agencies can take to
    increase their use of performance information. But in the
    meantime, we welcome the opportunity to work with you and your
    staff, who have been very helpful already in organizing this
    hearing, and we hope we can continue to work with you on these
    critical issues.
        With that, I will conclude my remarks and look forward to
    your questions. Thank you.
        Senator Carper. Thank you so much for your coming. Thank
    you very much to you and your colleagues at GAO for your work
    in this arena and for helping us as we approach the change in
    administrations and the new Congress. Thank you.
        Ms. Steinhardt. Our pleasure.
        Senator Carper. Mr. Peacock, you have done good work, both
    at OMB and clearly at EPA on this front, in this and other
    endeavors, too. We thank you for that and we welcome your
    testimony today. Thanks for joining us.

     TESTIMONY OF MARCUS C. PEACOCK,\1\ DEPUTY ADMINISTRATOR, U.S.
     ENVIRONMENTAL PROTECTION AGENCY; AND FORMER OFFICIAL AT U.S.
                    OFFICE OF MANAGEMENT AND BUDGET

        Mr. Peacock. Well, thank you, Mr. Chairman. Thank you for
    having this hearing. As you have noted, my name is Marcus
    Peacock and I serve as the Deputy Administrator at the
    Environmental Protection Agency. I do sincerely thank the
    Subcommittee for having this hearing. Congress has lots of
    hearings every year, but they tend to focus on how a particular
    government program can do a particular job better. I think this
    Subcommittee understands that you can come up with similar
    methods for not just improving one, two, or three Federal
    programs, but for improving all Federal programs.
    ---------------------------------------------------------------------------
        \1\ The prepared statement of Mr. Peacock appears in the Appendix
    on page 84.
    ---------------------------------------------------------------------------
        The President certainly gets that idea. That is why in
    2001, he directed that agencies integrate performance
    information into their budgeting decisions. This meant agencies
    had to identify performance measures for each Federal program
    and factor that when they did their budgets, the ultimate goal
    was to improve results across the government, and at that time
    the President directed me to lead this cross-agency effort.
        The result of the initial work we did was the Program
    Assessment Rating Tool (PART). PART is essentially a method for
    assessing the effectiveness of Federal programs, and more
    importantly, making recommendations regarding how those
    programs can be improved. And since its inception, the PART has
    won a number of awards. It has been copied by other
    governments.
        I am often given credit for creation of the PART, but I
    will tell you, I was very fortunate to have three things on my
    side. The first was support from the top, and that was from the
    President. The second was to have a very talented team of
    people at OMB, the Office of Management and Budget, who did the
    hard work of creating and fleshing out the PART. And finally,
    third, we had a very talented and smart advisory committee,
    which included Dr. Kettl, who kept looking at our work and
    reviewing it.
        In 2005, I moved from OMB to run the day-to-day operations
    of EPA, and in the last 3 years, I have spent a large amount of
    my time improving EPA's performance management systems. A
    number of my predecessors, including Al Alm and Hank Habicht
    also spent--emphasized improving management systems, and
    hundreds of EPA staff over the last 20 years have spent time on
    this effort. So I know better than anyone else that I really
    stand on their shoulders.
        EPA has come a long way. As the governor mentioned, we just
    rolled out EPAStat. It is the first Federal agency to have a
    Stat program. But I would say the biggest problem EPA currently
    faces is to make sure our performance measures are actually
    used. You can treat a measurement system like a thermometer or
    a thermostat, and, of course, a thermometer measures the
    temperatures, but a thermostat not only measures the
    temperature, but allows you a way of changing it. And
    performance management systems should be thermostats, not
    thermometers. They should be used to produce change so that we
    become more effective at serving the public.
        Metrics for reporting don't mean much, but metrics for
    management, I think, are vital. There are five barriers that I
    have found in trying to get performance information used more,
    and the first is a lack of fresh and frequent data. The Federal
    Government has lots of annual measures, but annual measures
    don't work too well in running the day-to-day operations of an
    organization or program.
        There is this Enterprise Rent a Car commercial you may be
    familiar with where the car is all wrapped up in brown paper
    and I have always wondered how that car stays on the road. You
    can't look out the windshield or the windows. And I think
    trying to use annual performance measures to manage the day-to-
    day operations of a program is similar to trying to drive that
    car around. With annual measures, you can't really see where
    you are going.
        The second barrier to using performance measures that I
    have run into is a focus on money and not results. Washington,
    DC still operates in a culture that asks how much did a program
    get rather than what did the program produce, and that is not a
    healthy way, I think, of looking at things. Just think about
    the incentives inherent in declaring an organization more
    successful simply by its spending more money.
        A third barrier is too many meaningless measures. What
    matters gets measured, but if everything is being measured, it
    is hard to tell what matters. So measures need to be meaningful
    and you need to be able to objectively evaluate where the
    measures came from.
        Fourth, too little access. The fewer people that have
    access to performance measures, the fewer people are able to
    actually use them. That is why I believe performance
    information should be made available to the widest possible
    audience.
        And then finally, the fifth barrier is resistance. Many
    people naturally worry about the consequences of not meeting
    performance targets. They think performance data may be used as
    a cover to either cut funding or punish people. And the fact is
    that these systems can be abused, but that is a very poor
    reason to avoid them. An organization cannot become excellent
    without measuring its performance. If you can't see what you
    are doing right, then you won't learn from that.
        In conclusion, I am fortunate to work at a place like EPA.
    The employees love the mission of the agency. They are very
    results-driven, so that if you can show them that you are going
    to be able to improve the results, they are willing to change
    what they are doing in order to get there. They understand that
    when EPA works better, public health and the environment
    improve faster. And EPA management initiatives that aren't
    linked to results are just gobbledy-gook. You need to
    demonstrate that they will lead to cleaner air, water, and
    land.
        We need to get to the point where all Federal agencies
    practice good performance management. Whether the latest and
    hottest issue is homeland security or securities regulation or
    climate change, a functioning performance management system is
    invaluable to helping any Federal agency, no matter what its
    work, do its job better. We need to get to the point where
    there is always an answer to any Federal employee who asks the
    question, ``How can I do my job better?''
        Thank you.
        Senator Carper. Thank you very much. I want us to come back
    during the questions and answers. Mr. Peacock has just run
    through five reasons why we don't always use information to
    help us improve our performance and I am going to ask our other
    witnesses to comment on those, if you would.
        Dr. Kettl, you were good enough to spend some time with
    Wendy Anderson and myself in our office earlier this week. We
    are grateful for that and for your years of work in this area
    and for all that you have written and lectured. We appreciate
    your being here today and helping us to do better, to do our
    job better. Thank you. You are recognized.

     TESTIMONY OF DONALD F. KETTL,\1\ DIRECTOR, FELS INSTITUTE OF
        GOVERNMENT, AND ROBERT A. FOX PROFESSOR OF LEADERSHIP,
                       UNIVERSITY OF PENNSYLVANIA

        Mr. Kettl. Thank you, Mr. Chairman. It is a great pleasure
    to be here today. I want to thank the Subcommittee for this
    opportunity to explore what in many ways is the most important
    issue facing this government, which is how to make sure that it
    works, how to make sure it delivers results and value to
    citizens. It may very well be the most important question we
    never stop to ask until it is too late, and smart government
    means getting out ahead of that to make sure that we are
    prepared for the challenges that we know that we need to find a
    way to be able to master.
    ---------------------------------------------------------------------------
        \1\ The prepared statement of Mr. Kettl appears in the Appendix on
    page 86.
    ---------------------------------------------------------------------------
        I want to make several points here today as we try to
    explore that question, the first of which is that if you look
    from a 30,000-foot level at the question of government reform,
    we are, in all likelihood, at the end of a natural life cycle
    of Presidential management reforms. For the first time since
    the Eisenhower Administration, we are now in a place where the
    next thing that the Federal Government might do, the next thing
    a new President might do is obvious.
        President Eisenhower had the Hoover Commission. President
    Kennedy had his whiz kids. President Johnson had planning,
    programming, budgeting systems and budget reform. President
    Nixon had management by objectives. President Carter brought in
    zero-based budgeting from Georgia. The Reagan Administration
    brought in a whole wide privatization initiative. At this point
    in the campaign back in 1992, it was very clear that President
    Clinton was going to reinvent government based on the best
    seller. And by this point in the campaign in 2000, President
    Bush had already delivered a major speech on the management of
    the Federal Government.
        It is not clear what it is that is going to happen next,
    but it is clear that what we have done so far has accomplished
    a lot, but that we are now at the point where we are going to
    need to do something different, that more of the same kinds of
    reforms are not likely to get us to where we need to go.
        So the first point is that we are at a crucial turning
    point because we are at the end of a series of natural life
    cycles in the reform movement.
        The second thing is that we have to find a way to get very
    smart very fast about what kind of reform we need because we
    face a set of wicked problems that will prove especially
    punishing if we do not find a way to get ahead of these
    problems. We face problems whether we are talking about
    tomatoes in the supermarkets or hurricanes like Katrina that
    come up very quickly, that are very large, that provide little
    time to react, and that impose large consequences for failure.
        And so we are at a point where we need to find a way to do
    smart things fast to be able to deal with these issues that we
    know we are going to be facing, and we know that more of what
    we have been doing is simply not going to do the job.
        The third point is I want to look in particular at what the
    reforms in the Clinton and Bush Administrations have produced.
    Both of these are tremendously importantly initiatives that in
    many ways, although they may not seem like it from a distance,
    are part of the same broad fabric of looking at results and
    trying to motivate managers to do better.
        The Clinton Administration brought a strong focus on
    results for the first time to the entire Federal Government. It
    encouraged Federal managers to innovate. It brought an expanded
    customer service initiative to the government, ranging from
    Social Security to the IRS, in ways that were truly
    breathtaking and important. But at the same time, one of the
    things the Administration did was to engage in substantial
    downsizing of the workforce without engaging in a right-sizing
    of what that workforce ought to look like.
        As a result of that, the Government Accountability Office
    has now named human resource management one of the high-risk
    areas that the Federal Government faces, the issues that are
    most likely to cause the biggest problems in management, and
    GAO has identified this and in many ways it is the consequence
    of trying to figure out now what kind of government do we need.
    We clearly can't downsize any further without thinking about
    how to right-size what it is that we do, and that was a major
    question that the Clinton Administration left unanswered.
        In the Bush Administration, the PART process we have heard
    discussed here this afternoon was a tremendous breakthrough.
    For the first time, all Federal agencies had to try to assess
    goals for all their programs. They had to measure what it is
    that they were trying to accomplish and focus on activities,
    and it was government-wide in the scale that was anticipated in
    some ways by the Clinton Administration but never fully
    realized. And two other things happened tied to the Office of
    Management and Budget and tied to the budgetary process. Both
    of these were tremendously important and major breakthroughs.
        The piece that we have left unanswered here is trying to
    find ways of separating out the ideological commitments to some
    parts of the management agenda, for instance, increasing
    contracting out, from the question of what it is that we want
    the contracting out process and other management processes to
    accomplish, focusing on these goals and objectives, and
    ensuring that we have a focus on activities that cut across
    government agencies.
        The basic fact, for better or worse, is that there is no
    program that can adequately encompass any problem that matters.
    Or put differently, there is no problem that matters to anybody
    that any one Federal agency, any one level of government, any
    one unit can possibly control. That was a lesson from Hurricane
    Katrina and it strikes through every single important thing
    that the Federal Government does.
        So the problem is trying to figure out how to get leverage
    over these long and complex chains that typically require
    action across many agencies, and the piece that lies next after
    the President's management agenda in this Administration is
    finding a way to get to that cross-cutting piece. What the PART
    process was terrific at was the vertical integration of
    knowledge. The next step has to be the horizontal management
    coordination to try to ensure that kind of collaboration.
        What we need, in short, is my fourth point, actionable
    intelligence, trying to find ways of finding out the things
    that we can do something with, and once we figure out what it
    is that we need to know, getting action to happen. We need
    people--this is a lesson from the Clinton Administration--we
    need Federal managers who can lead. The difficulty is that we
    have a lot of very smart people who are doing very good things,
    but often out there feeling that they need to work against the
    odds. And what we have to do is find a way to help them to do
    what it is that they know they need to know and know what they
    need to do in a way that makes it easy.
        We need information that focuses government on results
    instead of collecting information within silos. We need to
    drive that results to try to improve the way the government
    works instead of simply producing, as we have been discussing
    here this afternoon, more weights on the pig without making the
    pig any better.
        We need, finally, a focus on collaboration that builds
    cooperation along a horizontal chain against all those people,
    all those organizations, all those agencies, to be part of a
    solution to ensure that, in the end, the problem gets solved
    instead of just the agency gets managed.
        We, in short, need to try to find ways of developing a new
    generation of reforms. More of what we have been doing is
    likely to produce diminishing returns because, as GAO is
    pointing out, we are collecting more and more information, but
    it is not like people are doing anything much with it. What we
    need to do is figure out how to make sure that the information
    that we have is information that improves the way that
    government works.
        We need results that matter to people, which gets to my
    fifth point about trying to figure out how to make that work.
    First, we need to talk about what it is that we are trying to
    do. Often, a focus on results within individual agencies has to
    do with how many checks have been mailed, how many grants have
    been processed, how many forms have been filed, but in the end,
    none of that matters unless the problems get solved.
        The fact is that most citizens don't care about what it is
    that the FAA does, but what they want is when they go to the
    airport, they want to be able to take off to where they are
    going and land at the destination safely. They don't care about
    the FDA, but they would like to be able to eat tomatoes safely
    without having to worry about it. Citizens care about results
    and we need a government that is driven toward those results.
        The second point within this, though, is that the chain of
    producing accountability toward those outcomes, which is what
    people care most about, is often long and complex, where
    individual agencies, individual units contribute pieces, but
    nobody controls the whole thing. That means we need to find a
    way to hold each individual player accountable for their
    contribution to the outcome as a whole. We need to know what we
    are trying to accomplish and understanding what each
    individual's contribution to that is.
        This means ultimately, as my third point here, that the
    government needs to get leverage over the outcomes that it
    wants to try to accomplish. One of the great findings and
    discoveries about EPA is that EPA is most effective not when it
    does it itself, but when it enlists its partners toward the
    common pursuit of making the environment clean.
        What is it this ought to look like in the next
    Administration? For the President, there are several things
    that could happen. The first is a new generation of reforms
    that understands the importance of what has happened before but
    the need primarily first to focus on outcomes and results that
    matter.
        Second, a kind of geographic information and performance
    stat for those parts of the Federal Government that make sense
    so that we can look at what it is that matters and measure in
    real time today how well we are doing it because if we wait
    until the end of the year for a GPRA report to produce some
    kind of results, by that point, it will be too late to change
    what it is that needs to be fixed.
        We need an Office of Personnel Management that is more
    robust and develops leadership in the Federal Government, and
    we need a focus in the White House in particular to have
    somebody in the White House who can focus on the issues of
    making the government work and making that important for the
    President.
        For Congress, we need a system of the Government
    Performance and Results Act that focuses on results that
    matter, hearings that on every occasion ask people what is it
    that we are trying to accomplish and how well are we doing it,
    and a focus on programs and agencies that put outcomes that
    matter for people at the forefront of things.
        We have plenty of information. We need to make it
    actionable and find ways of producing results for the citizens
    in the end, because that is what it is that the people expect.
        Senator Carper. Excellent. Thank you very much for really
    three terrific testimonies.
        I want to go back to, as I said I would, to Mr. Peacock.
    You mentioned five barriers to agencies taking this approach
    and improving their outcomes. Would you briefly mention those
    again, and then I am going to ask our other two witnesses to
    comment on them.
        Mr. Peacock. Sure. The first one is lack of fresh and
    frequent data. The second is a focus on money, not results, and
    you could broaden that to inputs, not results or outcomes.
    Third is too many meaningless measures. Let me know if I am
    going too fast. Fourth is too little access. The governor
    talked about transparency. It is the same issue there, too
    little access to the measures. And then finally, just
    resistance, organizational resistance to change.
        Senator Carper. Let me just ask either Ms. Steinhardt or
    Dr. Kettl, I want to ask both of you, to react to those five
    points, if you would.
        Ms. Steinhardt. I think our work supports every single one
    of those points. In fact, where we have seen greater use, real
    live use of performance information is where agencies have been
    able to overcome some of those kinds of barriers.
        It is very important in agencies or organizations to be
    able to communicate the kind of performance information
    throughout the organization, making it transparent to everyone,
    and we have seen some good examples of how agencies do that,
    communicating frequently to all of its employees where they
    stand. It helps spur competition. It provides good benchmarks
    for units across the organization. It has real benefits and a
    number of agencies have been able to do that.
        Let me consult my list again here.
        Senator Carper. Go right ahead.
        Ms. Steinhardt. Real-time information, absolutely. This was
    a point that I was making, I think, earlier, where the
    information isn't timely, credible, accurate, it is not going
    to be used, and organizations that figure out how to make that
    information available on a daily or real-time basis are ones
    that are going to see that it is used, and Governor O'Malley
    talked about how that concept is applied and was applied in
    Baltimore and now in StateStat. We have seen some very good
    examples of that, as well.
        Too many measures, that is sort of a basic precept of
    performance measurement. The vital few. You can't expect people
    to focus and really align themselves with what the organization
    is trying to accomplish if it is diffuse and scattered.
        Senator Carper. Good. Thank you. Dr. Kettl.
        Mr. Kettl. I simply couldn't agree more. That is exactly
    the core point here. The real issue here is that any time the
    process of performance descends into questions of measurement,
    we have in a sense lost the game because we get away from what
    it is that performance is all about. It is about communication.
    It is about talking about what it is that matters and figuring
    out how that communication gets people to act. And so as we
    tend to slice and dice things into ever smaller pieces and
    measure more things, we are likely to get to a point where we
    are unlikely to get people to act. Performance is about
    communication. It is not primarily about measurement.
        What is it that communication ought to be about? The key to
    overcoming resistance is primarily one of getting people to
    agree on what problem they want to try to solve and how to go
    about trying to do that.
        One of the examples that while mayor, Governor O'Malley
    worked on was something called the rat rub-out program in
    Baltimore. They had rats that they wanted to rub out. They had
    maps that showed where citizens were complaining about problems
    of rats. Which department in Baltimore is in charge of rats?
    The answer is, none of them are. On the other hand, it is a
    function of public health, of housing, of redevelopment, of
    transportation, of the people in charge of water and sewer. It
    is a cross-cutting collection of agencies which, put together,
    have to be responsible for solving the problem.
        If you ask everybody, do you want to collaborate?
    Everybody, of course, always says yes. But getting it to happen
    in real time and in real life is very hard to do. You put a map
    up on the board that shows, here is a problem with rats. Is
    anybody happy with that? No. What we are going to do about
    that? We want to get together to solve it. Who is going to
    solve it? Everybody understands their particular contribution
    to solving that problem, and that is what it is that drives it.
    It is communication about results that matter and getting
    people to put themselves to work to solve the problems that
    matter.
        So the reason why so often performance management doesn't
    work is it becomes issues of measurement and that conversation
    never happens.
        Senator Carper. Ms. Steinhardt, and then Mr. Peacock, you
    may want to say something here when she is finished.
        Ms. Steinhardt. I just wanted to add to this concept of
    visibility of information, because I think that is really also
    extremely key here. I came across this article a couple of days
    ago in the Wall Street Journal about how U.S. Airways has now
    gone from the bottom of the list of airlines in on-time
    performance to the top so far in 2008. What is really striking
    to me--the article is actually very interesting in discussing
    how it did that--is the fact that everybody knows about on-time
    performance. It is a very widely-used statistic. And in the
    airline industry, it is one of the key indicators of their
    viability as a company. It really matters to the CEO and
    everybody in the company, how their on-time performance is
    ranked. It is the visibility of the information, I think, that
    really makes such a big difference.
        Senator Carper. Excellent. Mr. Peacock, do you want to add
    anything before we move to another question?
        Mr. Peacock. The governor mentioned that EPA had rolled out
    an EPAStat program and he also mentioned two other things. He
    said it is easy to take any measurement program and put the
    ``stat'' label on it, which is true. But he also mentioned that
    if you are going to do this seriously, you really have to be
    committed to it and have some core elements that you are
    committed to to make it work.
        Across these five barriers we have been talking about, EPA,
    through its EPAStat program, I think, is probably tackling--has
    tackled about three-and-a-half of them. But when I got to the
    agency, regarding lack of fresh and frequent information, I
    went in hoping to get monthly information. The most we could
    tolerate and are tolerating right now is quarterly information,
    which still took a lot of work to get. If it was any less
    frequent than that, it wouldn't work.
        The number of measures we had was too many. We have been
    able to cut it by 15 percent. We will continue to decrease it
    so that it becomes more manageable.
        In terms of access, the relentless meetings, EPAStat
    meetings we have, and they do have to be relentless--this gets
    back to Dr. Kettl's point, which I think is critical. This is
    all about communication. We have meetings--I have meetings
    personally with each assistant administrator and regional
    administrator every quarter on these statistics, but the
    richness of this is in discussing them and having a
    conversation about what is going right, what is going wrong,
    and what we are going to do about it.
        We now broadcast those meetings internally to any EPA
    employee who wants to view them, and our quarterly management
    report, as far as I know, we are the only Federal agency that
    on a quarterly basis puts out our performance measures to the
    public for anybody to look at. So we are very strong on the
    access.
        And then in terms of resistance, once again, I just happen
    to work in a culture where people are willing to see if changes
    results in us meeting our mission. There is just not a lot of
    resistance.
        But we still have a problem, I think, with focus on money
    and inputs rather than results. It is the sort of culture we
    run into, not just at EPA but throughout the Federal Government
    and outside the Federal Government, where people want to know,
    well, how much money are you asking for, when really the first
    question should be, what are you planning on delivering next
    year?
        Senator Carper. So true. Why do you suppose that is the
    case?
        Mr. Peacock. If I knew that, I wouldn't be at three-and-a-
    half of these solved. I would be at four-and-a-half of them
    solved. I mean, I would be interested to hear why other people
    think that is the case.
        I will tell you, my experience in the OMB, where we
    realized this was a big problem, I mean, when the budget comes
    out every year, still, the first thing that is talked about is
    the dollars that are being proposed, not what we are going to
    get for it. So it is just ingrained in the way we think about
    things and I think it is going to be incremental, but it is a
    change that has started to take place, but it is going to have
    to keep taking place.
        Senator Carper. Dr. Kettl, do you want to respond to that
    same question that I just asked?
        Mr. Kettl. I think your question is why is it we focus so
    much on inputs----
        Senator Carper. Yes.
        Mr. Kettl [continuing]. And it is like the old joke about
    the drunk looking for the keys underneath the lamppost. Is that
    where you lost them? No, I lost them over here. Why are you
    looking under the lamppost? Because that is where the light is.
    People focus on the things that are easy to look at and easy to
    measure because OMB at least publishes a budget that has all
    the numbers in it and it is easy to say, how do we know we care
    about something? Because we are spending a lot of money on it.
    Then we wonder why it is that we are not producing good
    results. Well, because that is not the function of what it is
    that we are driving the system to.
        But what we need is, in a sense, an alternative way of
    thinking about accountability. What citizens care about--what
    citizens know about is that they want government to work, and
    in the end, quick headline, increases, decreases, symbols that
    are valuable. But if it doesn't make the government work
    better, citizens don't care, and in fact, it makes them even
    more cynical.
        I think the main reason why we do it is, first, because the
    numbers are there and they are easy and there is no clear
    alternative otherwise, and second, it has to do with the fact
    that we don't have an obvious alternative accountability
    mechanism that focuses on outcomes. Why? Because we have lots
    of measures, but we haven't figured out what to do with them
    yet. That is the next generation of reform that I think we
    need.
        Senator Carper. OK. Let us talk about the importance of
    leadership. I don't think the City of Baltimore would have gone
    to CitiStat without the leadership of their mayor. My guess is
    that the State of Maryland wouldn't have gone to StateStat
    without the leadership of Governor O'Malley.
        In Federal Government, we elect a President every 4 years,
    we elect or reelect. We have six-year terms here in the Senate
    and two-year terms, as you know, in the House. Folks who serve
    in the Executive Branch as cabinet secretaries are--they can
    serve for 4 years, occasionally 8 years, but more often than
    not they serve less than 4 years. Folks that are deputy
    secretaries and associate secretaries and under secretaries,
    they tend to come and go.
        How important is leadership and the continuity of
    leadership at the top and then down to different levels of
    government here in Washington? How important is the consistent
    message, that this is important, we have got to do this, and to
    hear that not just from the very top of the government, but
    also the top of each department and with the agencies
    themselves? How important is that? Please.
        Ms. Steinhardt. I would say it is absolutely the first
    step. It is the precursor. Nothing else--none of the other
    recommendations, none of the other practices that we talk about
    would be successful if the leader of the organization doesn't
    care about this because nobody else will if the leader of the
    organization does not. It doesn't end there, but it starts
    there.
        Senator Carper. All right.
        Mr. Peacock. I agree. It is a prerequisite, but there are
    two ways to institutionalize these systems and this way of
    thinking. One is through standard operating procedures and
    organizational structure, so for instance, at EPA, we now have
    put in place as part of a regular annual process we go through
    scrubbing our measures to make sure that they are meaningful,
    and that is actually the process by which we are reducing our
    measures.
        And second, in terms of an organization, for instance, we
    have now created a Program Analysis Division at EPA which
    essentially, with the Office of the Chief Financial Officer,
    makes the system run, produces the statistics. It is
    essentially like the EPAStat Office or the analogy which these
    cities and the State of Maryland, for instance, have.
        And then the second way to institutionalize this is the
    career people. Obviously, the policy officials, the political
    appointees aren't there very long, but we have career officials
    and managers who, at least at EPA, have been there a very long
    time. Particularly as the value of these systems are
    demonstrated to them, they become believers and advocates for
    them, and whether or not you have what the political appointees
    may be doing, or even if the standard operating procedures are
    in place, if this is a value to good managers in terms of
    managing their programs, then these systems will be used.
        Mr. Kettl. Mr. Chairman, the people question is absolutely
    essential. There is a paradox in the way in which the Federal
    Government has evolved, I think. As it has gotten more complex,
    the role of people in leadership has become more important. We
    had this model for a long time that said we will create a very
    complex system and we will slide people in and out in standard
    operating procedures because they will be pretty much
    interchangeable. The idea of being able to create a civil
    service system and all kinds of standard operating procedures
    were not to make people unimportant, but to make it less
    important who was serving in those positions.
        It is increasingly important now, the more complex the
    system has gotten, to have the right people, smart people in
    the right places to drive things forward or otherwise things
    just don't happen, and the way in which it does happen, and,
    for example, the Coast Guard has proven this, is making that
    part of the organization's culture. When the standard operating
    procedure becomes a culture that reinforces the pursuit of
    strong outcomes, that is what it is that produces this piece.
        Unfortunately, too often in the Federal Government, it is
    something that happens either by accident or by people having
    to do it despite the fact that the game somehow seems rigged
    against them. We have got to find a way to institutionalize
    strong performance-oriented cultures into the very core of the
    way government operates.
        Senator Carper. In looking at our third panel and thinking
    about the agencies that we are holding up here for acclaim, for
    applause, because they have gotten it and they are really
    setting an example for the rest of us, EPA is one of those and
    I think it is fair to say that, Mr. Peacock, he probably
    wouldn't admit this, but he is a big reason why EPA has moved
    down this road and has good results to show for it.
        But among the other agencies that are here, the Nuclear
    Regulatory Commission, NASA, the VA, and I wonder if there is
    any common thread with all of those or if you can think if
    maybe there is. I am not sure, but I think there might be, but
    any thoughts?
        Mr. Kettl. Mr. Chairman, I think it is in some ways a very
    simple answer to a very complex problem. They know what they
    are trying to do. They focus relentlessly on doing it. They
    reinforce people for doing it. They keep track of how well they
    are doing it. And it is a simple process that focuses on
    results and having a culture that reinforces what they are
    doing and they focus on what their outcome is.
        Their job is not simply to, at the VA, for example, taking
    a stack of papers on veterans' benefits from Part A and moving
    them over to Part B. They focus on the veterans. They have had
    a system, for example, of veteran-centered health care for a
    long time that is what drives them and drives what it is that
    they do. They know what they are there for and they know who
    they are trying to serve and they focus the system on serving
    them. And developing standard operating procedures and
    leadership to try to do that is the key to making a high-
    performing organization.
        Senator Carper. Anyone else?
        Mr. Peacock. I think Dr. Kettl is exactly right. A clear
    mission that everybody can see and believes in has got to be a
    prerequisite for this. I don't remember who mentioned it, but
    this clear line of sight idea that everybody in the
    organization can see not only the mission of the agency, but
    how it connects to what they do on a day-to-day basis, and that
    is a prerequisite for this.
        There are other agencies that have a clear mission and I
    think everybody sees it but aren't necessarily doing a great
    job at this. So while it is a prerequisite, I don't think it is
    sufficient.
        Senator Carper. Ms. Steinhardt.
        Ms. Steinhardt. I agree completely with Dr. Kettl and Mr.
    Peacock's comments. I would add, though, and I don't
    necessarily apply this to VA and NRC, perhaps to NASA, but
    sometimes it is a crisis that focuses an agency on what it is
    about and what its mission and what its desired goals ought to
    be.
        This happened to the IRS, actually, in the mid-1990s. There
    were oversight hearings, a lot of press attention,
    Congressional attention on allegations of taxpayer abuse and
    dismal customer-taxpayer service and IRS had no information,
    performance information, to refute all of these horror stories
    that were coming out. And the Congress itself, I think, played
    a very key role in setting some very specific performance
    targets for the agency, and the agency began to collect the
    information, became very performance focused and it really
    turned itself around.
        In the period certainly from the 2000 survey, when we
    started, when we were asking about managers' use of information
    there to the most recent survey, they have showed a really
    positive improvement there, and more significantly in actually
    how the agency is performing, they have really made great
    strides.
        Senator Carper. All right. Well, good. There is still
    plenty of work to be done. Every year, we hear the size of the
    tax gap, and we learned last year it is over $300 billion,
    monies that are owed, not being collected. We deal with this on
    this panel. We deal with the improper payments. Not everyone,
    not all of the Federal agencies are reporting their improper
    payments, but of those who do, we know that our improper
    payments are, I think, roughly $50 billion a year, mostly
    overpayments. And it occurs to me that if somehow there were no
    tax gap and if we weren't making these improper payments, we
    would basically have a balanced budget.
        Ms. Steinhardt. But today, we know the size of the tax gap,
    or at least now IRS is able to estimate the size of the tax
    gap.
        Senator Carper. We weighed the pig. [Laughter.]
        Ms. Steinhardt. Well, first step.
        Senator Carper. There you go. A couple more questions and
    then we will excuse this panel and invite our last panel to the
    table.
        Ms. Steinhardt, in light of your survey results showing
    improvements in the amount and the types of performance
    measurements that agencies have at their disposal, why do you
    think so little change in managers' use of that information is
    being displayed in their decisionmaking?
        Ms. Steinhardt. Well, just to comment on the survey
    results, a lot of the results--the average, rather, is affected
    by the fact that we have a couple of large agencies like DOD
    and the Interior Department that are low, relatively low users
    of performance information. The smaller agencies like NASA,
    NRC, EPA, tend to be higher, but they don't color the average
    so much.
        In terms of why we are not seeing so much progress or use
    in those other agencies like Interior, for example, that is the
    subject of the second part of our study, so we hope to be back
    here. So we hope to be back here with some more concrete
    information on what those agencies can do to improve.
        Senator Carper. All right. Thank you.
        Mr. Peacock, how do you suppose the next Administration can
    get agency leaders to buy into such assessments and use a
    review process in resulting performance information to improve
    their management decisions?
        Mr. Peacock. I mean, there are so many directions one could
    go in that would be worthwhile, although I think that Dr. Kettl
    hit on some of them. We now have a wealth of information that
    wasn't there before, and looking at this information but from a
    cross-agency perspective, looking at it from the sense of the
    public who is more interested in protecting wetlands rather
    than whether it is the Corps of Engineers or the Department of
    Interior or EPA that is protecting the wetlands and being able
    to compare across agencies which programs are doing this the
    best, doing it the most efficiently, and then using that
    information in terms of making budget decisions or policy
    decisions.
        I mean, there would be any number of ways to do that, but
    there probably needs to be less emphasis now on trying to
    collect this information and more emphasis on actually bringing
    it in-house and someone sifting through it to actually use it
    and make decisions using it.
        Senator Carper. OK. Thank you.
        And finally, one more for Dr. Kettl, if I could. In your
    work on how government responded to Hurricane Katrina, you have
    described the importance, and I am going to quote here, ``the
    importance of working in horizontal networks instead of
    struggling over a vertical chain of command.'' As GAO has
    testified, there are many programs that need to work together
    to accomplish common outcomes. I think Mr. Peacock has said as
    much, too. What are some approaches the next Administration
    could take to bring about a more cross-cutting focus on
    performance issues?
        Mr. Kettl. The first thing, Mr. Chairman, is to underline
    the importance of these cross-cutting approaches to begin with
    because increasingly, what it is that government does has to
    happen through those relationships.
        The second thing is to focus on a smaller number of more
    important outcome-based measures, less on how many applications
    for processing, more on what kind of results we are producing,
    getting the information in real time, getting the information
    displayed graphically. One of the things about Mayor and then
    Governor O'Malley's work is that if you look at the maps, you
    just look at it and it talks to you. It tells you what the
    problem is and where to go about doing it.
        For example, he had a series of charts about the decline in
    the murder rate in the three kidney-shaped areas that he talked
    about. You look at that and you see it, you can see the change,
    you can see the progress, you see where the problem is. It
    tells you what it is that you need to work on and where you
    need to go. There is something about information that is
    focused on what matters, that is real time, that focuses on
    where it is that it happens that drives the collaborations that
    are required and then makes it possible to have the
    conversations about who it is who is going to contribute to the
    solutions to the problems that in the end matter.
        That is the secret to the successes in EPA's work, of
    trying to figure out what are we trying to do. How can we make
    the air cleaner? Who has got a piece of the action? How can we
    measure whether or not we are doing it? And it is that kind of
    relentless cycle of follow-through, not once a year, once every
    5 years collecting some interesting reports and putting it on a
    chart. It has got to be real-time information that drives
    action and drives action the citizens care about.
        I can say that with confidence not because I know it to be
    true, but because I have watched others do it. The successful
    managers around the Federal Government are doing that and more
    of that is what it is that we need.
        Senator Carper. I will just close with an observation. When
    I was privileged to serve as the chief executive of our State,
    there are a number of goals that we set. We wanted to raise
    student achievement, and we didn't want to just put it all on
    the schools or on the Department of Education. We wanted to
    reduce the incidence of teenage pregnancy. We just didn't want
    to put it all on the kids department or on the Department of
    Health and Social Services. We wanted to reduce the incidence
    of recidivism in our prisons, but we didn't want to put it all
    on our prisons. We wanted to reduce the crime rate, but we
    didn't want to put it all on our police.
        And we ended up establishing a Family Services Cabinet
    Council which included about half of the cabinet, and I would
    meet with them every month and we would set measurable goals
    and then try to hold one another accountable for progress. But
    I think the fact that I sat with the cabinet secretaries every
    month and then they would meet once a month without me with
    members of my governor's office staff just to follow up had
    some pretty good effect.
        There is something to be said for trying to hold a
    particular agency responsible for getting, say, ``A'' done, but
    sometimes ``A'' needs to be done in collaboration with a bunch
    of other folks. So you need somebody saying, you guys and gals
    have to work together. We do have stovepipes in the Federal
    Government just like we did in State Government and they are
    hard to get rid of.
        I want to thank you so much for your testimony today,
    really for your terrific work. Ms. Steinhardt, to you and your
    colleagues at GAO who are just enormously helpful to this
    Subcommittee and really to our Executive Branch, too. To Mr.
    Peacock, who has worked in OMB and done great work there and
    now gone to EPA and helped to show as a practitioner what we
    can do and always is a source of inspiration, actually. And Dr.
    Kettl, we have a lot of people who testify before us. Few are
    as clear and concise, particularly academics---- [Laughter.]
        Who are as concise and able to put in terms that everybody,
    even I, can understand. It is a real gift. So we thank you. It
    is easy to see why you have had real success and are highly
    sought after.
        I want to leave the record open for a week or two so that
    folks who are not here can have a chance to submit some
    questions, and if you get those and would respond to them in a
    timely way, we would be most grateful.
        But thank you so much for joining us today. Thank you.
        Ms. Steinhardt. Thank you.
        Mr. Peacock. Thank you.
        Mr. Kettl. Thank you, Mr. Chairman.
        Senator Carper. To our third panel, welcome. Thank you for
    hanging in here until almost 4:30. We promise not to keep you
    much longer, but thanks a lot. Have you all been here for the
    presentations of our first two panels?
        Mr. Tucker. Yes, sir.
        Senator Carper. Good. Well, a special thanks.
        One of the questions that I asked is, what are the common
    threads between the agencies that we are really holding up for
    acclaim here, who seem to get it and have been achieving better
    results because of that fact, in terms of measuring outcomes
    and actually using the information to change performance. That
    is one of the questions that I am going to be asking each of
    you, so be thinking about that, if you will.
        Jim Dyer is the Chief Financial Officer for the Nuclear
    Regulatory Commission. We were privileged in another committee
    that I serve on and chair, a subcommittee with Senator
    Voinovich, to have a chance to work with your agency a lot and
    have a very high regard for you. What is it, the best agency in
    the Federal Government in which to work?
        Mr. Dyer. The best place to work.
        Senator Carper. The best place to work, yes. But there, you
    serve as the person who is responsible for planning, for
    budgeting, and financial management of agency resources. You
    are also, I understand, currently the agency's Performance
    Improvement Officer, responsible for leading the agency's
    performance management activities.
        Prior to your position as Chief Financial Officer, I
    understand you served as Director of the Office of Nuclear
    Reactor Regulation, where you were responsible for agency
    safety programs for the commercial power, research, and test
    reactors of the United States. A couple of people suggested to
    me you also may have at one time served in the U.S. Navy and
    that you were a submarine officer from 1973 to 1977. I was on
    active duty from 1968 to 1973 looking for submarines and had a
    pretty good naval flight officer, P-3s, Orions, and our job was
    to hunt for Red October. We found plenty of them, but whenever
    we were looking for you guys, for our guys, we could almost
    never find you. You are mighty good at what you did. You served
    in the Naval Reserve until your retirement in 1995, and I
    served in the Reserves until my wife made me quit in 1991. But
    thank you for your service then and for your service today.
        Scott Pace is the Associate Administrator for Program
    Analysis and Evaluation at NASA. He is responsible for
    providing objective studies and analyses in support of policy,
    program, and budget decisions by the NASA Administrator.
        He previously served as Chief Technologist for Space
    Communications, where he participated in the negotiations that
    resulted in the 2004 GPS-Galileo agreement between the U.S. and
    the European Commission, and Mr. Pace also previously served as
    the Deputy Chief of Staff to NASA Administrator Sean O'Keefe.
    What is he doing these days?
        Mr. Pace. I believe he is working in industry these days.
        Senator Carper. All right. Tell him the Junior Senator from
    Delaware was asking for him if you run into him, please.
        Mr. Pace. Certainly.
        Senator Carper. Daniel Tucker, Deputy Assistant Secretary
    for Budget for the Department of Veterans Affairs. In this
    capacity, Mr. Tucker helps to oversee the $90 billion budget
    for the Department. In addition, Mr. Tucker serves as
    Performance Improvement Officer for the Department and is
    responsible for overseeing the development of Veterans Affairs'
    annual performance plan and managing Program Assessment Rating
    Tool evaluations. Prior to his current position at the VA, he
    served as the Chief Financial Officer for the National Cemetery
    Administration.
        With those introductions behind us, let me just ask Mr.
    Dyer if you would go ahead and lead us off. Summarize, if you
    will, in roughly 5 minutes and then we will have some questions
    and call it a day. But thank you. Please proceed.

       TESTIMONY OF JAMES DYER,\1\ CHIEF FINANCIAL OFFICER, AND
       PERFORMANCE IMPROVEMENT OFFICER, U.S. NUCLEAR REGULATORY
                               COMMISSION

        Mr. Dyer. Thank you, Mr. Chairman. It is an honor to appear
    before you today to share our approach for using performance
    information to improve management of U.S. Nuclear Regulatory
    Commission programs.
    ---------------------------------------------------------------------------
        \1\ The prepared statement of Mr. Dyer appears in the Appendix on
    page 92.
    ---------------------------------------------------------------------------
        The mission of the NRC is to license and regulate the
    Nation's civilian use of byproducts, source, and special
    nuclear materials to ensure adequate protection of public
    health and safety, promote the common defense and security, and
    protect the environment.
        The NRC is pleased that the Government Accountability
    Office recognized our improvements in the use of performance
    information and we believe it is the result of three factors:
    One, a strong commitment by the NRC commissioners and senior
    management to continuous improvement; two, the effective
    implementation of our planning, budgeting, and performance
    management process; and three, NRC openness with employees and
    our external stakeholders.
        We also believe that the recent government activities to
    create the Performance Improvement Officers and Performance
    Improvement Council can further improve the NRC performance
    management.
        We created the NRC planning, budgeting, and performance
    management process in response to the Government Performance
    and Results Act in 1997 and still use the four simple
    integrated components of the process to manage our program
    performance.
        First, our current strategic plan concisely identifies two
    strategic goals for safety and security to accomplish our
    mission and an organizational excellence objective which
    characterizes the manner in which we intend to achieve these
    goals.
        Second, our budget process involves multiple levels within
    the NRC organization, so we obtain staff's commitment to
    complete the planned activities on a schedule and within
    budget.
        Third, NRC executives monitor performance using office
    operating plans that track budget expenditures and performance
    targets well beyond the detail in the NRC performance budget
    delivered to Congress to improve our accountability.
        And fourth, our program assessments integrate the inputs
    from several sources, including Congressional hearings and the
    OMB PART tool results to develop program improvements.
    Additionally, we have recently started using the Lean Six Sigma
    evaluation process to more systematically assess program
    performance. These program assessments provide a significant
    input for NRC's senior executive and manager appraisals and
    awards.
        We also strive to effectively communicate NRC performance
    expectations and results to our staff and external stakeholders
    to promote openness and increased accountability. The NRC uses
    a strategic plan, the performance budget, Congressional
    reports, and our performance and accountability report to
    communicate with our stakeholders and receive their feedback.
    Commission meetings concerning NRC program performance reviews
    are some of our best attended public meetings. NRC offices
    publish quarterly operating plan results on their internal
    websites, routinely discuss the performance results during
    periodic staff meetings, and in newsletters, and recognize
    staff contributions to agency mission through the awards
    process.
        However, we do face challenges with effective performance
    management. As you are well aware, the NRC has had significant
    growth in the past few years, and this has increased the
    complexity of performance monitoring, with new programs, new
    organizations, and an increased level of work within the
    agency. Better performance management through expanded use of
    techniques such as the Lean Six Sigma and evaluation is needed
    for improved regulatory program consistency, efficiency, and
    effectiveness.
        Additionally, our current system of performance monitoring
    and assessments is very labor intensive. We need to modernize
    methods for processing performance information using the latest
    technology. Improving our budget structure and integrating cost
    and performance information in a more timely manner will
    facilitate improved performance management.
        Also, our performance metrics focus on the quantity and
    timeliness of our products over the quality because of the ease
    of measurement. Most often, quality issues are identified by
    schedule delays because of required rework. We need to develop
    methods for more timely measurement of the quality performance
    of our programs.
        The Office of Management and Budget-led Performance
    Improvement Council provides a forum to discuss these
    challenges through sharing agency best practices for improving
    the use of performance information. Presentations by other
    agencies on their experience with implementing Lean Six Sigma
    evaluations have been valuable to the NRC, and the Council's
    planned agenda items offer the potential to further improve our
    performance management.
        In conclusion, we appreciate the Government Accountability
    Office's recognition, but realize that we can further improve
    our use of performance information for managing our programs.
    We intend to continue to improve our performance through
    effective use of internal assessments, external oversight
    inputs, public feedback, and sharing experience with other
    Federal agencies.
        Thank you. I look forward to answering your questions.
        Senator Carper. Mr. Dyer, thank you very much.
        Mr. Pace, your entire statement will be made part of the
    record for each of you, so if you want to summarize, feel free.

    TESTIMONY OF SCOTT PACE,\1\ ASSOCIATE ADMINISTRATOR FOR PROGRAM
     ANALYSIS AND EVALUATION, AND PERFORMANCE IMPROVEMENT OFFICER,
             NATIONAL AERONAUTICS AND SPACE ADMINISTRATION

        Mr. Pace. Just the highlights. Good afternoon, Mr.
    Chairman. Thank you, and it is a pleasure to be here.
    ---------------------------------------------------------------------------
        \1\ The prepared statement of Mr. Pace appears in the Appendix on
    page 97.
    ---------------------------------------------------------------------------
        As you mentioned my background at NASA, probably the most
    relevant point for today's hearing is I am also the agency's
    Performance Improvement Officer under Executive Order 13-450
    and the lead for various program improvement initiatives under
    the President's Management Agenda, which, of course, as you
    know, Sean O'Keefe had a lot to do with in writing. So I have
    had maybe a bit of an advantage.
        I have been able to observe the full spectrum of NASA's
    activities, its performance, where we have excelled and where
    we can improve, and I would like to share some observations, I
    think, on why NASA generally receives good marks and continues
    to receive strong bipartisan support from Congress, for which
    we are very grateful.
        First of all, as some of the other speakers, I think, have
    mentioned, NASA is really fortunate in being charged by the
    Administration and Congress with missions that are well suited
    to performance management. We have a performance-based culture
    that values mission success as a central tenet, and through
    hard experiences, both good and bad, NASA has sought to put the
    right tools and governance processes in place to better ensure
    accountability, transparency, and oversight. These processes
    are not ends in and of themselves but means for accomplishing
    the missions that are assigned to us by the Administration and
    Congress.
        In our experience, the best tools for creating
    accountability and transparency are those that provide
    consistent external reporting requirements, that provide
    flexibility internally in the design of measurement techniques
    tailored to our unique missions. Consistent external reporting
    helps focus the agency on the most important measures. As was
    mentioned, if you measure too many things, you are probably
    deluding yourself. And flexibility is important to help avoid
    the trap of imposing simple one-size-fits-all performance
    measures that can really mask more than they can reveal.
        While our system is generally working well, we are not
    without challenges and we need the support of Congress in
    maintaining our commitment to the efficient and effective
    execution of agency missions. Our mission is very concrete and
    our goals readily flow down to every level of the agency. Fly
    each Space Shuttle mission safely until retirement. Complete
    the International Space Station. Develop the next generation of
    launch vehicles that will return us to the moon, Mars, and
    beyond. And the performance against concrete goals is,
    therefore, measurable, traceable, and thus actionable.
        Our budget is aligned to our goals. We have well-
    established program and project management policies driven by
    the need to deliver a wide portfolio of missions of many
    different sizes, many different destinations.
        Our programs are typically milestone-driven and we face
    hard deadlines, such as planetary windows, which we just have
    to hit. Our schedules are complex and must be integrated. Many
    organizations have to come together to fly each mission. And as
    an agency, we are comparatively small and compact, and
    therefore the key strategic informed conversations can be held
    at the highest levels of the agency. So our internal decision
    loops can be fairly tight.
        We are currently focused on aligning all of our external
    reporting to a single set of measures. We are linking internal
    performance indicators directly to these measures and
    commitments, in particular GPRA and the PART tools, as has been
    discussed earlier.
        I could describe our processes in a lot of detail, but the
    real demonstration, I think, of performance management is
    mission success. We currently have 56 robotic spacecraft
    operating throughout, and in some cases beyond, the solar
    system. We continue to conduct groundbreaking scientific and
    aeronautics research. The Space Station is nearing completion.
    We have seven more assembly flights to go plus two logistics
    flights, which we hope we will be able to fly before Shuttle
    retirement in 2010. This October, we are looking toward a final
    Shuttle mission to the Hubble space telescope to finish its
    repair.
        In the risky business we are in, we are going to continue
    to face challenges. Nine projects have breached cost or
    schedule thresholds in fiscal year 2008 against performance-
    based lines that we established with the Congress in our major
    program annual reports. We are continuing aggressive effort to
    improve the fidelity of our up-front estimates of cost and
    applying more rigor to our life cycle cost estimates than
    really any other time in the agency. There are many sources of
    these cost growths, some we can control, some we can't, and we
    are working on each one.
        We accept the need to improve performance and transparency,
    but at the same time, we are struggling with the issue of
    reporting complexity. Of course, if a project is in trouble, it
    is in trouble, but under current reporting requirements from
    the multiple stakeholders, such as the OMB, Congress, and the
    Government Accountability Office, we have a variety of
    different trigger points and thresholds that include life cycle
    costs, development costs, schedule growth, key milestone slips,
    and each one of them can be slightly different.
        So we are looking for some greater consolidation in how
    these various breaches are defined, greater consolidation of
    requirements for what is in and out of that reporting, not only
    to reduce our reporting transaction costs, but also to ensure
    that agency attention and effort is focused on mitigating the
    most significant project performance issues so we stay focused.
        I really want to thank you very much for your time this
    afternoon and really appreciate you drawing the Committee's
    attention to this important topic. It is something that is near
    and dear to us and we appreciate your interest. Thank you.
        Senator Carper. Thank you, and thank you for coming and for
    the example that you provide. Mr. Tucker.

    TESTIMONY OF DANIEL TUCKER,\1\ PERFORMANCE IMPROVEMENT OFFICER,
    AND DEPUTY ASSISTANT SECRETARY FOR THE BUDGET, U.S. DEPARTMENT
                          OF VETERANS' AFFAIRS

        Mr. Tucker. Thank you, Mr. Chairman. I appreciate the
    opportunity to be here today to discuss how the Department of
    Veterans Affairs is using performance information to improve
    service delivery to veterans and accountability for results.
    ---------------------------------------------------------------------------
        \1\ The prepared statement of Mr. Tucker appears in the Appendix on
    page 100.
    ---------------------------------------------------------------------------
        Since the passage of GPRA 15 years ago, we have seen a
    major transformation in the manner in which performance
    information is used in budgeting and performance management
    throughout the VA. We have made noteworthy progress in using
    performance information to better justify our requests for
    resources, monitor our programs throughout the year, and
    document our accomplishments and challenges in a manner that is
    transparent to veterans, our stakeholders, and the general
    public.
        Every year, we reevaluate the performance measures included
    in our Congressional budget justifications and operating plans.
    We analyze each measure to ensure it is still appropriate and
    meaningful and we develop new or improved metrics that present
    a better gauge of program results. We use customer feedback to
    ensure that the performance metrics we employ are measuring
    those things that are most important to veterans and their
    families.
        Today, our budget request for every program contains a
    wealth of information on our strategic goals and objectives,
    our historical and projected performance levels, and the means
    and strategies we will use to achieve our goals. In other
    words, our budget is justified not only by the kinds of
    activities we will conduct, but more importantly, by the
    results we expect to achieve.
        VA employs a variety of mechanisms to monitor program
    performance. The most important of these tools is our monthly
    performance review. Chaired by the Deputy Secretary, these
    reviews involve senior leaders throughout the VA and focus on
    financial management and program performance as well as the
    execution of major construction and information technology
    projects. These meetings play a vital role in keeping the
    Department focused on its highest priorities, achieving key
    performance goals, and resolving operational challenges.
        On November 15 each year, we publish our annual Performance
    and Accountability Report. This report presents a detailed
    description of how well VA performed relative to the
    performance goals we established at the beginning of the year.
        Our commitment to transparency and reporting has been
    highlighted by the Mercatus Center of George Mason University.
    This independent research organization conducts an extensive
    evaluation of all agency performance reports. They have ranked
    VA among the top-rated reports all 9 years they have conducted
    their analysis. For the last 2 years, the Mercatus Center has
    presented an award to VA for achieving the highest score in the
    Federal Government in transparency in reporting and the
    Department is extremely proud of this recognition.
        Another one of our major accomplishments has been our
    success in establishing a performance culture in VA. This has
    largely involved a gradual movement away from a focus on inputs
    toward a more meaningful discussion of program outputs and
    outcomes. And education and training within the Department for
    staff and managers is critical to this successful shift in
    emphasis.
        Our monthly performance reviews have also been instrumental
    in institutionalizing a performance culture at VA. Most
    importantly, these monthly meetings are a clear demonstration
    of the interest and support of VA's top policy issues in using
    performance measurement to oversee departmental programs and
    operations.
        Another key strategy we employed to institutionalize a
    performance culture was to develop a set of key performance
    measures. Several years ago, we realized that we have been
    pretty successful in developing improved performance metrics,
    but we had created so many that it is hard to figure out what
    was most important. So to correct this problem, we identified a
    set of about 25 key measures that we considered critical to the
    success of the Department. These key measures formed the
    foundation of our budget request, our monthly performance
    reviews, and our annual performance report.
        While we have successfully tackled many of these
    challenges, we still have more that need further attention. We
    have a sound set of outcome measures for some of our programs,
    particularly medical care, but there are other program areas
    for which we still need better indicators of the extent to
    which VA programs improve quality of life for veterans and
    their families.
        An additional challenge all agencies face is how to more
    tightly link cost data to program performance. In particular,
    we need to strengthen our ability to demonstrate how
    performance could change if resource levels vary. As with all
    performance metrics, this will require the necessary
    information systems and analytical tools to produce valid and
    reliable data.
        That concludes my statement, Mr. Chairman. I will be happy
    to answer any questions.
        Senator Carper. All right, Mr. Tucker. You are right on the
    money. That was exactly 5 minutes. Thank you.
        Thank you all for wonderful testimony. Why do you suppose
    your agencies have risen to the top, at least with respect to
    your use of information to be able to improve performance? What
    common threads are there with your agencies in terms of your
    experience and improvement? Mr. Dyer, when you retired from the
    Navy, what was your rank?
        Mr. Dyer. I was a captain, sir.
        Senator Carper. Captain Dyer, OK. I was a captain, too.
        Mr. Dyer. Yes, sir.
        Senator Carper. A recovering captain. A recovering
    governor.
        Mr. Dyer. Senator, as I heard the other two presentations,
    that was the first time I had heard or seen their text, too,
    the thing that I heard--the three reasons I thought that we had
    improved in the past few years, I heard that come out in
    theirs, and that is a commitment to doing it--and we heard it
    from the earlier panels, too--a commitment to doing it, a
    mission statement that is simplified and that allows you to go
    to outcomes, which drive outputs, which drive the inputs to the
    budget, and that kind of a focus on executing to achieve the
    outcomes intended.
        Senator Carper. Mr. Pace.
        Mr. Pace. I would agree. I think the most important
    advantage, I remember the prior panel talking about as things
    become more and more complex, people become more and more
    important because they are not interchangeable and I think that
    the performance culture that we have at NASA, which is
    reinforced in so many different ways, has been actually really
    crucial, so that when you come in with a performance
    measurement system of some sort, initially, it is looked at
    askance. What is this, a new unfunded mandate dropping down on
    us?
        But when people get into it and see that you are going to
    use that information to make better decisions that then have
    some linkage to mission success, then there is a greater
    acceptance of it. It is something that actually reinforces the
    performance culture of the agency because it is not seen as
    alien to the agency but, in fact, is integral and consistent
    with the values of the agency, and that in turn is reflected by
    statements from the top management down through the system.
        So it is a very interwoven problem, but as you hear, it is
    a very consistent one that repeats in many organizations.
        Senator Carper. All right. Mr. Tucker.
        Mr. Tucker. I think from what the panel members here have
    stated as well as the members of the previous panels, the thing
    that you hear consistently is top management interest and top
    management involvement. You have to have the leadership
    engaged. If the leadership is engaged and focused on using
    performance information, that will drive change throughout the
    organization.
        Senator Carper. Let me just interrupt for a second. I have
    been privileged to know three VA Secretaries during my time
    here in the Senate, and that was in 7 years, so you have had a
    number of Secretaries. Is it Tom Bowman? Who is the Chief of
    Staff----
        Mr. Tucker. Tom Bowman is.
        Senator Carper. I think Mr. Bowman has been around for a
    while----
        Mr. Tucker. Yes, sir.
        Senator Carper [continuing]. And maybe he provides the
    continuity. An old Marine, as I recall.
        Mr. Tucker. Yes, sir.
        Senator Carper. But you have had a fair amount of turnover
    at the top in the VA, but a continued commitment to
    improvement.
        Mr. Tucker. I think the commitment from those that remain,
    like Tom Bowman is a good example as the Chief of Staff, he
    sits in on our monthly performance reviews every month. He is
    certainly a very active member of the management team. I think
    also what is important, especially at the VA, even though we
    are a very large agency and we have a great diversity of
    programs, we are all there to serve veterans and we are all
    there to serve their families and those that are eligible for
    the different benefits that we provide, and I think that is
    part of our performance and the success of our performance
    culture at the VA, is a real clear mission statement and we
    know who we are there to serve.
        Senator Carper. If you think of it, let Tom Bowman know we
    just opened our second VA outpatient clinic in Delaware. We
    only have three counties. We have two of these clinics now. We
    just opened two of them in the last 2 or 3 months. But he was a
    big help in getting that done, so tell him we are grateful for
    that and for his service.
        Mr. Tucker. I will do that.
        Senator Carper. You have mentioned performance culture. Mr.
    Pace has mentioned it. I think Mr. Dyer has mentioned it. One
    of my other hats is chairman of the subcommittee that has
    jurisdiction over nuclear safety. We talk a fair amount about
    safety culture, and during the time in my 23 years in Naval
    aviation, oftentimes our commanding officers would say to us,
    and again, our job was to hunt for Red October, track Soviet
    nuclear submarines, and do all kinds of surface surveillance
    and fly missions off the coast of Vietnam and Cambodia and
    other places.
        But our skippers would always say the most important thing
    you are doing in your mission today isn't whether you are
    trying to mine some harbors or whether you are doing certain
    service missions where you are tracking the bad guys, the bad
    submarines, the Ruskies, whatever it was. He said your most
    important thing that you are doing is taking off safely, flying
    safely, coming back and landing safely, and walking away from
    it and going home. So that is the most important thing that you
    are doing.
        Almost everything that we did, whether it was in the
    aircraft, the folks who were maintaining our aircraft, training
    us, everything pointed toward safety. It was really a culture
    of safety and we tried to do it in a way so it wasn't a
    ``gotcha'' system, so that if people observed things that were
    wrong, they would actually step forward and say, ``This isn't
    really a very safe practice,'' and not be penalized or punished
    for having said that.
        So we try to take the same thing, the same approach with
    respect to our nuclear power plants and to encourage them, and
    I know that the NRC are very much involved in doing that for
    us. The safety culture, we have 104 nuclear power plants. We
    want every one of them to be as committed to safety as we were
    in my squadron, our squadrons in the Navy.
        Talk to me, if you will, about performance culture. How do
    we establish a kind of--obviously, you have a performance
    culture at your agencies and we have a bunch of agencies who
    don't have it. You are the exception, not the rule. How do we
    establish the kind of performance culture at more agencies,
    much as we are trying to establish safety cultures at 104
    nuclear power plants? And that would be a question for each of
    you. I don't know who cares to go first. Mr. Dyer, feel free.
        Mr. Dyer. Yes, sir, Mr. Chairman. Just like you talked
    about the safety culture at the nuclear plants and the
    guidelines that go along with them, I think starting with the
    performance culture at the NRC, it begins actually with the
    formulation of the budget. When we get in to recognizing the
    roles and responsibilities and of the individuals within the
    organization and get their commitment that we are going to
    expend this amount of resources to achieve this kind of
    results, we get that kind of commitment of them that they have
    bought in, that they sign up for it, and it is not something
    that we pass down from on top and just say, you are going to
    get X number of resources and get it done. It is they have an
    input to it that starts, that contractor, that bond, if you
    will, to actually perform the work and get the results that are
    intended.
        Senator Carper. All right. Thank you. Mr. Pace.
        Mr. Pace. Really, I want to also agree with that. One of
    the crucial issues we have is aligning the resources that
    content, that we make program commitment agreements and we say
    we want to do this vehicle, do this kind of mission, this is
    what it is going to cost, and if we change the resources, we
    change the content that we ask of people. And if we change what
    we ask of people, we also change the resources. The two in the
    budget process have to be very closely linked. So if the
    schedule slips or something else slips or new problems crop up,
    that we account for it.
        And I think that is part of having really almost an ethic,
    a moral duty, really, to the people we ask to do the work, that
    we are going to back them up and they, in turn, have an
    obligation to bring forward their best estimates about what it
    is going to take because we are both trying to achieve that
    mission success. So the budget process is at the center of it.
        The second thing I would add is that along with having a
    safety culture, it is important that our independent reviews
    and our assessments are done with the mindset of assisting the
    project to be successful, not of a mindset of auditing the
    project or trying to come up with a certain number of gigs. The
    mentality we try to bring to it is we are kind of like the
    graduate TAs. We are there to get the other students to pass.
    We may beat on them pretty hard, may review them pretty hard,
    but our goal is we want them to pass, we want them to succeed,
    and we are there to help them succeed, and having that
    mentality of an assist function also gets people to buy in,
    bring their problems forward because we are not out there to
    have them fail. We are there for mission success.
        Senator Carper. Thank you. Mr. Tucker, anything you would
    like to add?
        Mr. Tucker Just adding on top of what the other witnesses
    have said, I think one of the big changes that we have seen in
    establishing a performance culture at the VA is the shift from
    the discussion on inputs into outcomes. A lot of folks have
    said that today.
        I can think back years ago and when we would have monthly
    budget reviews in the Department. It was always just simply a
    discussion of, well, what is the variance from plan? Did you
    spend more than your monthly plan this month? Did you spend
    less? How much more? How much less? People were either happy or
    sad about the results, but the discussion didn't really focus
    on program results.
        Those are the kinds of discussions that we have now. We
    have the data. We have the data systems in place to facilitate
    that discussion. And management from the very top all the way
    down to the medical center level use that information to
    improve performance. The measures are the same whether they are
    at the national level, the network level, or the local level.
        An example I can give you is on wait times for VA patients
    to get into our health care system. We have used performance
    information to really drive improvement in reducing the number
    of veterans that are waiting more than 30 days for an
    appointment from--it was over 250,000 in April 2006 and we have
    driven that down to about 51,000 in June, just last month. So
    that is one example of how leadership is focused at the top all
    the way down to the facility level to drive performance
    improvement.
        Senator Carper. OK. Thank you. Mr. Pace, maybe one last
    question for you. According to a GAO survey, I think it was
    done in 2007, a large majority of your managers said that they
    use performance information to identify and solve program
    problems. Do you think that looking at program performance
    information has helped improve NASA's programs, and if so, how?
    You have talked about this already, but do you want to add
    anything else?
        Mr. Pace. I think one of the major areas of differences
    where I have certainly seen a change in the time that I have
    been at NASA is when I first came there, we would go to program
    reviews and people would be talking about the status and in
    some cases how they felt about the program or project, as Mr.
    Tucker was just saying. What you see in the status reviews that
    we have monthly now and baseline performance reviews is much
    more toward quantitative. So programs are coming in with--here
    is where we are on the finances. Here we are on the human
    capital. This is what the state of the asset and facilities is
    looking like. This is where the program has hit various
    milestones. So monthly meetings, much more quantitative.
        That then sparked discussions. I mean, the most important
    thing that occurs in that quantitative data and where people
    see various disconnects is then a conversation occurs at the
    senior management level that says, why are those disconnects
    happening? What can we do about them? What is the resolution
    path forward?
        So I think that it is helping NASA quite a bit. We are
    doing things that are consistent with our culture, but we are
    doing them, I think, in a much more rigorous and consistent
    way. We are still learning. We still have plenty of other areas
    that are imperfect, but I think the intention on performance
    being quantitative and getting those discussions out in front
    of senior management has been very helpful.
        Senator Carper. Thank you. Mr. Dyer, according to the GAO's
    same 2007 survey results, the NRC's managers report a
    significant increase in the use of performance information
    since responding, I think to a survey 7 years earlier, the 2000
    survey. In fact, I am told that NRC managers are now among the
    most positive among their government-wide counterparts in their
    reported use of performance information. I just ask, in your
    view, how and why you think this change came about.
        Mr. Dyer. Mr. Chairman, I believe we have had this same
    process in place since 1997 and every year we do a little bit
    better. We revise our performance indicators. We reach out and
    get them to lower levels in the organization. And I think the
    people can trend progress and we report out on it and we
    provide feedback to the staff on it and they recognize that
    these things are important and that they now accept them and
    embrace them.
        Senator Carper. All right. One of the things that I find
    remarkable when we have testimony from the NRC and from the
    nuclear industry and others who are interested in the industry
    is the increase in the power plants that we have, their ability
    to provide power. If they are able to provide power 100 percent
    of the time, that would be terrific. But it wasn't that long
    ago when they were down around 60, 70 percent, and today they
    are up over 90, 91 percent. That is a remarkable achievement.
        Mr. Dyer. Yes, sir.
        Senator Carper. A lot of people deserve credit for that and
    I know among those people are the NRC.
        The last question would be really for all of you. Each of
    your agencies has had programs that were assessed by OMB
    through the PART process. What suggestions do you have for the
    next Administration, whoever might lead that, but the next
    Administration on how to design and structure a performance
    assessment process such as PART?
        Mr. Tucker. I will go first.
        Senator Carper. Yes, Mr. Tucker, feel free.
        Mr. Tucker. I think what will be helpful for OMB to do is
    to work collaboratively with the agencies to look at the
    criteria that is used and really to continue the process and
    refine the process of the evaluation and performance measures.
    I mean, I think that has been the key benefit that the VA has
    obtained from going through the PART process is having a really
    hard look at the performance metrics that we use, whether they
    are outcome-focused or efficiency focused, and developing
    action plans and requiring the action plans to be updated on a
    regular basis so you are continually improving the metrics that
    the organization is using to assess performance.
        I think one other thing--I will just put in a plug--is
    recently in response to the Executive Order that the President
    signed back in November, OMB did establish an interagency
    Performance Improvement Council. I think that council should
    continue into the next Administration. It is just getting
    started. It is growing. It is getting traction, starting to get
    some legs under it.
        Senator Carper. Who heads up that council?
        Mr. Dyer. Robert Shea is leading the Performance
    Improvement Council at OMB right now.
        Senator Carper. OK. Thank you.
        Mr. Dyer. But I hope that it will be as successful in the
    future as the Budget Officers Advisory Council has been that
    OMB had established a long time ago where the budget officers
    from the different departments come together once a month to
    share best practices, to raise issues and concerns, and just to
    have that face-to-face once-a-month meeting and discussion. And
    I think the Performance Improvement Council can do the same
    thing.
        Senator Carper. Good. I had not heard of the Performance
    Improvement Council. It sounds like a good idea. I hope it
    continues, as well. Thank you.
        Mr. Dyer. Thank you.
        Senator Carper. Mr. Pace.
        Mr. Pace. Well, I would also actually endorse some of the
    Performance Improvement Council meetings. I also go, as well.
    But I think one of the things I have found most valuable for it
    is actually the cross-agency discussions that have happened, so
    seeing what other agencies are doing, sharing notes with them,
    and comparing best practices. I thought that has been a useful
    forum. I know that other speakers have talked about cross-
    agency issues.
        Regarding the PART, I think it obviously should continue. I
    think it has been helpful. We have been able to improve the
    measures of effectiveness for some of our programs.
    Particularly we have made some improvements, I think, in our
    education efforts that have been helpful and traceable to PART.
        But the other thing I would say in terms of maybe a
    particular parochial plug that we and some other agencies have
    is in measurements of R&D. We have a very project-oriented
    agency. Other agencies are very process-oriented. But some of
    our science agencies, NSF, maybe NOAA, ourselves, Geological
    Survey, when we are doing R&D activities that stretch over a
    longer period of time, it is often harder to see the results
    each year. You have to look over a longer time horizon to judge
    what the benefits of R&D are.
        So ourselves and NSF have had some good conversations about
    this. I know with DOE Science Office, we have also had some
    good conversations, and measuring effectiveness of R&D is one
    of those areas where a simple one-size-fits-all sometimes
    doesn't work. And the conversations we have had in the PIC, I
    think have been helpful to us, coming up with some ideas we are
    going to hope to pursue.
        Senator Carper. Thank you. Mr. Dyer, the last word?
        Mr. Dyer. Mr. Chairman, I think the PART program, the PART
    is fundamentally sound and it is a good program. At the NRC, we
    have had seven PART evaluations. Six of them have been rated
    high as effective and one was moderately effective, and I
    owned, designed that one that was moderately effective, and it
    had a good therapeutic effect, I think, on the NRC that when we
    went back and pulled the string on it, what we really looked at
    and came up with was that we had a problem with our budget
    structure and that cascaded down into problems, and that wasn't
    what we thought going in.
        So I have got a healthy respect for the PART. That being
    said, it is an extremely resource-intensive process. So as we
    go forward, I would hope that we streamline the process so we
    don't have to dedicate the amount of energy that goes in in
    conducting the PART evaluations.
        Senator Carper. All right. Thank you.
        We are at the end of our hearing. I was just reflecting on
    this. It is pretty clear I haven't had a lot of company up here
    on this side of the rostrum. We have a lot of the agencies, not
    all the agencies have really emulated your success and your
    approach by taking seriously our ability to use the information
    that it has generated to improve our performance. Not everyone
    does that.
        Having said that, we are going to have a new President in 6
    months. We will have a new Congress in less than that. And we
    are going to still have a lot of problems to face here at home
    and around the world. We don't have unlimited resources. We are
    finding out just how limited our resources are, financial
    resources are to deal with those challenges.
        And if we are smart, and I hope we will be, the more of us
    here in the Legislative Branch and the folks in the Executive
    Branch, including right at the top, will realize that if we
    want to provide better results, better outcomes for the folks
    who are paying the taxes in this country, there are a variety
    of ways to do it and one of them is to take all this massive
    information that we have, try to figure out what of it we can
    learn from in order to provide cleaner air, in order to provide
    better transportation, in order to reduce our threats from
    within and without, and act on that. If we do those things,
    taxpayers will be happy and we will be better off as a Nation,
    too.
        We chose in this hearing today not to focus on the agencies
    that, frankly, weren't doing a good job. We chose to hold up a
    handful that are doing a good job in this regard and to say by
    inference to other agencies that could learn from you, take a
    look at these folks.
        I said to Governor O'Malley, we would rather see a sermon
    than hear one, and with you, you have had a chance to give us a
    little bit of a sermonette, but we have also had an opportunity
    to watch your performance over a number of years and we like
    what we see. Hopefully, by holding you up for praise and
    acclaim, we will encourage some other agencies to emulate your
    good performance.
        With that having been said, we are going to leave the
    record open for 2 weeks so we will have the opportunity for
    some of my colleagues and to submit some questions, maybe some
    statements, as well, for the record. We would ask if you do
    receive questions that you just respond promptly and we will be
    better for it.
        Thank you so much for coming, for the good work that you
    and your colleagues back in your agencies are doing. Convey our
    congratulations and our thanks to them, if you will. Thank you
    so much.
        Mr. Dyer. Thank you.
        Mr. Pace. Thank you.
        Mr. Tucker. Thank you.
        Senator Carper. This hearing is adjourned.
        [Whereupon, at 5:10 p.m., the Subcommittee was adjourned.]

                                A P P E N D I X

                                  ----------

    [GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]