[House Hearing, 108 Congress]
[From the U.S. Government Publishing Office]



 
                     FISCAL YEAR 2005 ENVIRONMENTAL
                        PROTECTION AGENCY BUDGET

=======================================================================

                                HEARING

                               BEFORE THE

                SUBCOMMITTEE ON ENVIRONMENT, TECHNOLOGY,
                             AND STANDARDS

                          COMMITTEE ON SCIENCE
                        HOUSE OF REPRESENTATIVES

                      ONE HUNDRED EIGHTH CONGRESS

                             SECOND SESSION

                               __________

                             MARCH 11, 2004

                               __________

                           Serial No. 108-46

                               __________

            Printed for the use of the Committee on Science


     Available via the World Wide Web: http://www.house.gov/science

                                 ______


                 U.S. GOVERNMENT PRINTING OFFICE

92-340                 WASHINGTON : 2004
_________________________________________________________________
For sale by the Superintendent of Documents, U.S. Government Printing 
Office Internet: bookstore.gpo.gov Phone: toll free (866)512-1800; 
DC area (202) 512-1800 Fax: (202) 512-2250 Mail: Stop SSOP, 
Washington, DC 20402-0001














                          COMMITTEE ON SCIENCE

             HON. SHERWOOD L. BOEHLERT, New York, Chairman
RALPH M. HALL, Texas                 BART GORDON, Tennessee
LAMAR S. SMITH, Texas                JERRY F. COSTELLO, Illinois
CURT WELDON, Pennsylvania            EDDIE BERNICE JOHNSON, Texas
DANA ROHRABACHER, California         LYNN C. WOOLSEY, California
KEN CALVERT, California              NICK LAMPSON, Texas
NICK SMITH, Michigan                 JOHN B. LARSON, Connecticut
ROSCOE G. BARTLETT, Maryland         MARK UDALL, Colorado
VERNON J. EHLERS, Michigan           DAVID WU, Oregon
GIL GUTKNECHT, Minnesota             MICHAEL M. HONDA, California
GEORGE R. NETHERCUTT, JR.,           BRAD MILLER, North Carolina
    Washington                       LINCOLN DAVIS, Tennessee
FRANK D. LUCAS, Oklahoma             SHEILA JACKSON LEE, Texas
JUDY BIGGERT, Illinois               ZOE LOFGREN, California
WAYNE T. GILCHREST, Maryland         BRAD SHERMAN, California
W. TODD AKIN, Missouri               BRIAN BAIRD, Washington
TIMOTHY V. JOHNSON, Illinois         DENNIS MOORE, Kansas
MELISSA A. HART, Pennsylvania        ANTHONY D. WEINER, New York
J. RANDY FORBES, Virginia            JIM MATHESON, Utah
PHIL GINGREY, Georgia                DENNIS A. CARDOZA, California
ROB BISHOP, Utah                     VACANCY
MICHAEL C. BURGESS, Texas            VACANCY
JO BONNER, Alabama                   VACANCY
TOM FEENEY, Florida
RANDY NEUGEBAUER, Texas
VACANCY
                                 ------                                

         Subcommittee on Environment, Technology, and Standards

                  VERNON J. EHLERS, Michigan, Chairman
NICK SMITH, Michigan                 MARK UDALL, Colorado
GIL GUTKNECHT, Minnesota             BRAD MILLER, North Carolina
JUDY BIGGERT, Illinois               LINCOLN DAVIS, Tennessee
WAYNE T. GILCHREST, Maryland         BRIAN BAIRD, Washington
TIMOTHY V. JOHNSON, Illinois         JIM MATHESON, Utah
MICHAEL C. BURGESS, Texas            ZOE LOFGREN, California
VACANCY                              BART GORDON, Tennessee
SHERWOOD L. BOEHLERT, New York
                ERIC WEBSTER Subcommittee Staff Director
            MIKE QUEAR Democratic Professional Staff Member
            JEAN FRUCI Democratic Professional Staff Member
                 OLWEN HUXLEY Professional Staff Member
                MARTY SPITZER Professional Staff Member
               SUSANNAH FOSTER Professional Staff Member
       AMY CARROLL Professional Staff Member/Chairman's Designee
                ADAM SHAMPAINE Majority Staff Assistant
                MARTY RALSTON Democratic Staff Assistant














                            C O N T E N T S

                             March 11, 2004

                                                                   Page
Witness List.....................................................     2

Hearing Charter..................................................     3

                           Opening Statements

Statement by Representative Vernon J. Ehlers, Chairman, 
  Subcommittee on Environment, Technology, and Standards, 
  Committee on Science, U.S. House of Representatives............    31
    Written Statement............................................    31

Statement by Representative Brad Miller, Member, Subcommittee on 
  Environment, Technology, and Standards, Committee on Science, 
  U.S. House of Representatives..................................    32
    Written Statement............................................    33

                               Witnesses:

Mr. Clay Johnson, III, Deputy Director of Management, Office of 
  Management and Budget
    Oral Statement...............................................    35
    Written Statement............................................    36
    Biography....................................................    38

Dr. Paul Gilman, Assistant Administrator for Research and 
  Development, United States Environmental Protection Agency
    Oral Statement...............................................    39
    Written Statement............................................    40
    Biography....................................................    71

Mr. Paul L. Posner, Managing Director, Federal Budget and 
  Intergovernmental Relations, General Accounting Office
    Oral Statement...............................................    71
    Written Statement............................................    73

Dr. Genevieve Matanoski, Professor, Department of Epidemiology, 
  Johns Hopkins University; Chair, EPA Science Advisory Board 
  Review of EPA's Fiscal Year 2005 Budget; Former Chair, EPA 
  Science Advisory Board
    Oral Statement...............................................    82
    Written Statement............................................    83
    Biography....................................................    86

Dr. Costel D. Denson, Professor, Department of Engineering, 
  University of Delaware; Member of the National Research Council 
  Panel Review of The Measure of STAR: Review of the EPA's 
  Science to Achieve Results (STAR) Research Grants Program
    Oral Statement...............................................    87
    Written Statement............................................    88
    Biography....................................................    92

Discussion
  Rationale for Science to Achieve Results (STAR) Research Grants 
    Program Budget Cuts..........................................    92
  Incorporation of STAR Scientific Research into Decision-making.    95
  Effect of Ecological Research Programs Budget Cuts on Human 
    Health Research Programs.....................................    96
  Immediate Effects of Science to Achieve Results (STAR) Grants 
    Program Budget Cuts on EPA Research..........................    97
  Application of the Program Assessment Rating Tool (PART) and 
    Alternative Review Methods Within EPA's Office of Research 
    and Development..............................................    99
  Termination of EPA's Building Decontamination Research Program.   102
  Peer Review and Evaluation of R&D Programs.....................   103
  Evaluating Short-term vs. Long-term Research...................   105
  Concluding Remarks.............................................   106

              Appendix: Answers to Post-Hearing Questions

Mr. Clay Johnson, III, Deputy Director of Management, Office of 
  Management and Budget..........................................   108

Dr. Paul Gilman, Assistant Administrator for Research and 
  Development, United States Environmental Protection Agency.....   111












        FISCAL YEAR 2005 ENVIRONMENTAL PROTECTION AGENCY BUDGET

                              ----------                              


                        THURSDAY, MARCH 11, 2004

                  House of Representatives,
      Subcommittee on Environment, Technology, and 
                                         Standards,
                                      Committee on Science,
                                                    Washington, DC.

    The Subcommittee met, pursuant to call, at 10:00 a.m., in 
Room 2318 of the Rayburn House Office Building, Hon. Vernon J. 
Ehlers [Chairman of the Subcommittee] presiding.




                            hearing charter

         SUBCOMMITTEE ON ENVIRONMENT, TECHNOLOGY, AND STANDARDS

                          COMMITTEE ON SCIENCE

                     U.S. HOUSE OF REPRESENTATIVES

                     Fiscal Year 2005 Environmental

                        Protection Agency Budget

                        thursday, march 11, 2004
                         10:00 a.m.-12:00 p.m.
                   2318 rayburn house office building

1. Purpose

    On Thursday, March 11, 2004 at 10:00 a.m. the House Science 
Subcommittee on Environment, Technology, and Standards will hold a 
hearing to examine the Environmental Protection Agency's (EPA) fiscal 
year (FY) 2005 budget request that proposes steep cuts in the Science 
to Achieve Results (STAR) grants program. Managed by EPA's Office of 
Research and Development (ORD), the STAR program supports research at 
colleges and universities on a wide array of environmental science 
issues.
    The hearing will examine why the Administration has proposed a $35 
million (or 35 percent) reduction in the grant program and the 
consequences of the reduction.\1\ Specifically, the proposed reduction 
would reduce EPA-sponsored external (also called extramural) research 
on pollution prevention and eliminate STAR funding for research in the 
areas of ecological systems, endocrine disruptors, hazardous substances 
and mercury. The hearing will also examine the extent to which the 
Office of Management and Budget's (OMB) effort to assess the 
performance of government programs under OMB's Program Assessment 
Rating Tool (PART) led the Administration to propose these reductions.
---------------------------------------------------------------------------
    \1\ The proposed reductions to the STAR program described here and 
in detail below are compared to the President's FY04 budget request. 
EPA's final FY04 spending will not be available until the agency's 
operating plan is completed.
---------------------------------------------------------------------------
    The Subcommittee plans to explore the following questions:

        1.  Why does EPA's FY05 budget request propose to eliminate 
        funding for EPA's STAR research grants to colleges and 
        universities for research on ecological systems, pollution 
        prevention, endocrine disruptors, hazardous substances, and 
        mercury? What are the consequences of these reductions?

        2.  To what extent did the OMB's PART review of EPA's 
        ecological and pollution prevention research programs drive 
        these reductions?

        3.  How can the performance of environmental research programs 
        best be measured? How do OMB's efforts to assess the 
        performance of EPA's program differ from those of the National 
        Academy of Sciences and EPA's Science Advisory Board?

2. Witnesses

          Mr. Clay Johnson III, Deputy Director for Management, 
        Office of Management and Budget.

          Dr. Paul Gilman, Assistant Administrator, Office of 
        Research and Development, Environmental Protection Agency.

          Mr. Paul Posner, Managing Director, Strategic Issues, 
        U.S. General Accounting Office (GAO); Managed GAO's recent 
        report, Performance Budgeting: Observations on the Use of OMB's 
        Program Assessment Rating Tool for the Fiscal Year 2004 Budget, 
        GAO-04-174.

          Dr. Genevieve Matanoski, Professor, Johns Hopkins 
        University, Chair, EPA Science Advisory Board's Review of EPA's 
        FY 2005 Budget Request; former Chair, EPA's Science Advisory 
        Board (SAB).

          Dr. Costel Denson, Professor, University of Delaware, 
        member of the National Academy of Sciences panel that authored 
        The Measure of STAR: Review of the U.S. Environmental 
        Protection Agency's Science to Achieve Results (STAR) Research 
        Grants Program (2003).

3. Brief Overview

    The President's budget for FY05 proposes to cut Science and 
Technology (S&T) at EPA by $92 million, or about 12 percent. The most 
significant percentage cut to S&T (other than the elimination of 
earmarks) would reduce funding for EPA's external grants program by 
almost 35 percent.
    These proposed cuts to the grant program, known as Science to 
Achieve Results (STAR), are especially noteworthy because in the last 
decade a number of outside experts, including the National Academy of 
Sciences (NAS), have urged EPA to fund research outside of its own 
laboratories. Most notably, the NAS praised the STAR grant programs in 
its 2003 report, The Measure of STAR: Review of the U.S. Environmental 
Protection Agency's Science to Achieve Results (STAR) Research Grants 
Program. The report argues that the external STAR grants are a critical 
means for the agency to have access to expertise that it does not have 
in-house, and to respond quickly to emerging issues. The external 
grants have also often been favored by critics of EPA who view EPA's 
in-house scientists as too likely to come up with research results that 
would favor a pre-existing regulatory agenda.
    The cuts to STAR have also attracted attention because they can be 
seen as a test case of how the Administration is using a new system the 
Office of Management and Budget (OMB) has created to evaluate the 
effectiveness of federal programs and inform spending decisions. The 
system is known as the Program Assessment Rating Tool (PART). Each 
year, OMB is selecting programs at each agency to evaluate using PART. 
It hopes to have reviewed all programs within the next four years.
    The proposed cuts to STAR raise two sets of questions about PART: 
How fairly does the PART tool evaluate programs? And how does the 
Administration apply PART evaluations in making budget decisions? The 
General Accounting Office (GAO) recently weighed in on the first 
question with a report, Performance Budgeting: Observations on the Use 
of OMB's Program Assessment Rating Tool for the Fiscal Year 2004 Budget 
(see more detail below).
    The proposed cuts to STAR raise particularly thorny questions about 
the use of PART. Most problematically, STAR grants are part of larger 
programs targeted at particular environmental issues. The STAR grants 
targeted for cuts are part of five separate research programs: 
ecological systems, pollution prevention, endocrine disruptors, 
mercury, and hazardous substances. The PART was applied to the first 
two programs as a whole; the external grants were not evaluated as a 
separate element. The other three programs were not evaluated at all. 
Other issues about the way PART relates to the STAR program and EPA's 
research and development (R&D) programs are described below.

The Five Programs--What They Do And What Would Be Cut

    Ecological Research. EPA characterizes the $110 million ecological 
research program as a core or basic research program. Its goal is to 
develop the scientific understanding to determine ecosystem conditions 
and trends, diagnose impairments, forecast ecosystem vulnerability and, 
ultimately, restore degraded ecosystems. For example, recent STAR 
funded research has been instrumental in developing scientifically 
grounded ways to measure water and ecosystem quality\2\ along the 
Nation's coastal areas and in the mid-Atlantic region.
---------------------------------------------------------------------------
    \2\ A large portion of the STAR grants in ecological research 
support the measurement of ecological conditions and most of ecological 
condition research is carried out through the Environmental Monitoring 
and Assessment program (EMAP) program, a program that the Committee has 
supported in recent years.
---------------------------------------------------------------------------
    According to EPA, the $22 million cut to STAR would eliminate 50 
grants in FY05 across all areas of the ecological research program. The 
proposed cuts would slow research on water quality in FY05 in the Ohio 
and Mississippi River basin and eliminate grants for research in such 
areas as western rivers and streams, the Great Lakes, and the Gulf of 
Mexico.
    Pollution Prevention. EPA's pollution prevention research program 
develops tools, technologies, and new systems for preventing pollution. 
Most of the $50 million program funds applied research, such as the 
development of innovative technologies for reducing the use of 
hazardous solvents. However, a small portion of the program supports 
extramural grants for more basic research on sustainable technologies. 
For example, in work that EPA carries out in partnership with the 
National Science Foundation, EPA supports research in so-called ``green 
chemistry,'' which promotes safer chemicals and chemical manufacturing 
processes.
    The proposed budget would redirect $5 million from research to a 
pollution prevention outreach program in another part of the agency. 
Redirecting these funds would eliminate $3 million in STAR funding, 
which is EPA's contribution to the EPA-NSF partnership.
    Endocrine Disruptors. EPA's $13 million research program focuses on 
providing a better understanding of the effects of endocrine disruptors 
and on humans, wildlife, and the environment. EPA also is developing 
new methods to screen chemicals for their potential endocrine effects. 
The program invests in both basic and applied research.
    The proposed budget would cut $4.9 million, which would eliminate 
the entire STAR grant research program on endocrine disruptors. The 
funds would otherwise have supported research on the extent to which 
humans and wildlife are exposed to endocrine disruptors, an area that 
the NAS and the World Health Organization have identified as an 
important research gap.
    Mercury. The goals of EPA's $7 million mercury research program are 
to reduce and prevent the release of mercury into the environment and 
to understand how mercury moves through the environment. This research 
supports a variety of the agency's air and water regulatory programs by 
developing control technologies, measuring mercury deposition, and 
attempting to understand the effect of mercury on wildlife. The 
proposed $1.9 million reduction to the STAR grant portion of the 
mercury research program would eliminate STAR research in FY05 on how 
and where mercury moves through the environment.
    Hazardous Substances. EPA established five university-based centers 
affiliated with 22 universities to address concerns about hazardous 
substances in the environment. Each center has developed a research 
program to meet regional needs. Centers are based at Johns Hopkins 
University, Louisiana State University at Baton Rouge, Purdue 
University, Colorado State University at Fort Collins and Oregon State 
University at Corvallis. The proposed $2.3 million dollar reduction in 
STAR funding would eliminate ORD's contribution to these centers.

What is the Value of Extramural Research at EPA?

    The NAS reviewed EPA's STAR program in 2003. The report, The 
Measure of STAR: Review of the U.S. Environmental Protection Agency's 
Science to Achieve Results (STAR) Research Grants Program, reaffirmed 
earlier National Academy recommendations that EPA should maintain an 
appropriate balance between its intramural and extramural research 
programs. (See the Attachment A for a summary of the report.) The 
report gave the STAR program a strong endorsement, calling it ``EPA's 
pre-eminent program that solicits independent scientific and 
technologic research from the Nation's best academic and non-profit 
research institutions.'' It also described STAR as enabling EPA to have 
access to the broad research community, fund research at the cutting 
edge of science, respond quickly to new issues, and address research 
gaps when EPA lacks the appropriate in-house expertise. It specifically 
praised the unique contributions that the STAR program is making to 
endocrine disruptors and ecological indicators research. The report 
concluded by recommending that ORD maintain STAR funding at a level 
somewhere between 15 and 20 percent of ORD's total budget.\3\
---------------------------------------------------------------------------
    \3\ If this were the case, and ORD were to be funded at $572, the 
level requested in the President's FY05 budget request, STAR funding 
would then need to be between $86 million and $114 million. However, 
the President's request for STAR in FY05 appears to be no more than $65 
million, and may be even less than that.
---------------------------------------------------------------------------

What is the Program Assessment Rating Tool (PART) and what did GAO 
                    conclude in its recent assessment of the PART?

    The Program Assessment Rating Tool (PART) is a new evaluation tool 
developed by OMB to assess the performance of federal programs and to 
link that performance to spending decisions. It judges programs on 
their purpose and design, strategic planning, management and results, 
heavily weighting the results portion of the review. Based on the 
review, OMB rates a program as either effective, moderately effective, 
adequate, ineffective, or ``results not demonstrated.'' (See Attachment 
B for a more detailed description of the PART.) OMB plans to apply the 
tool to all federal programs within the next four years.
    The Government Accounting Office (GAO) recently reviewed the PART 
process, though it did not specifically review its application to R&D 
programs (Performance Budgeting: Observations on the Use of OMB's 
Program Assessment Rating Tool for the Fiscal Year 2004 Budget, GAO-04-
174). (See Attachment C for a summary of the GAO report.) According to 
GAO, the PART process has reinvigorated the Executive Branch's focus on 
performance budgeting. However, GAO also concluded that OMB must do 
more to ensure fair and consistent application of the PART. GAO 
described the PART as ``a work in progress'' and needing ``[a]dditional 
guidance and considerable revisions. . .to meet OMB's goal of an 
objective, evidence-based assessment tool.'' GAO concluded that there 
are inherent challenges when applying the PART's restrictive yes/no 
format to programs with multiple purposes and goals, and recommended, 
among other things, that OMB clarify subjective terminology, provide 
flexibility in judging complex programs, clarify when output and 
outcome measures are appropriate, and increase dialogue with agency 
staff on such things as the definition of the program that will be 
reviewed.
    The way OMB applied the PART to EPA's S&T programs may illustrate 
some of GAO's concerns. For example, many of EPA's R&D programs have 
multiple goals and purposes, such as combining basic and applied, as 
well as intramural and extramural research, in one program. In 
addition, OMB's decision to evaluate the overall ecological research 
program may have led to challenges for EPA, which had never evaluated 
the entire program.\4\
---------------------------------------------------------------------------
    \4\ EPA has evaluated subprograms that cut across all of its 
research programs, such as the STAR program, and elements of the 
ecological research program, such as Environmental Monitoring and 
Assessment Program.
---------------------------------------------------------------------------

What did the PART evaluation conclude about the ecological research and 
                    pollution prevention programs and was the PART 
                    applied fairly?

    OMB used the PART to evaluate two of the five programs with 
extramural research elements that were cut in the President's budget. 
OMB concluded that the ecological and pollution prevention research 
programs could not ``demonstrate results,'' because neither had 
adequate standards to measure the progress of the programs.\5\
---------------------------------------------------------------------------
    \5\ See OMB's PART summaries, where it specifies reductions to 
these programs at http://www.whitehouse.gov/omb/budget/fy2005/pdf/
ap-cd-rom/part.pdf
---------------------------------------------------------------------------
    EPA has said it disagrees with how OMB applied the PART and the 
conclusions it reached. The primary area of disagreement appears to be 
over how to measure the performance of EPA's R&D programs.\6\ For 
example, EPA views the ecological research program as more of a basic 
research program, making it similar to research supported by the 
National Science Foundation (NSF). In OMB's PART review of NSF 
programs, it measured the performance of those programs using process 
indicators, such as whether the agency has conducted the appropriate 
peer reviews and how quickly it processes research grants. EPA 
maintains that its program should be similarly evaluated.
---------------------------------------------------------------------------
    \6\ EPA also disagrees with OMB's conclusion that the ecological 
research program failed to coordinate its research agenda within EPA or 
with outside agencies or researchers (suggesting redundancies with 
other programs), and that previous evaluations of the ecological 
research program were too focused on process measures.
---------------------------------------------------------------------------
    OMB, however, seems to want more than process measures to evaluate 
EPA's R&D programs. According to both EPA and the public PART review 
documents, OMB seems to want EPA to measure its programs less on 
process and more on output and outcome measures, such as the degree to 
which others used the products of the research, and how much pollution 
the research reduced or might reduce in the future. EPA agrees that its 
research strategic plans should make the connection between research 
and eventual reductions in pollution. However, it does not believe that 
either its basic or applied research programs should be held 
accountable for the actions of others who are outside of EPA's control 
and who may or may not use EPA's research products.
    EPA seems to believe that its R&D programs should be evaluated 
through a peer review process that considers the degree to which the 
research reflects the state of the science, adds knowledge to the 
field, and creates tools and methods that others could use. This view 
seems consistent with the views of EPA's SAB and various NAS reports, 
such as the STAR report, on the proper way to evaluate R&D. These 
reports have also praised EPA's for developing high quality basic, 
extramural research programs that develop knowledge, but are not tied 
to regulatory results.

Why was the ecological research and pollution prevention program cut 
                    and who decided to cut it? Was the decision related 
                    to the PART?

    The Administration's proposed budget clearly shows that OMB decided 
to cut the ecological and pollution prevention research programs 
because of their low PART scores. OMB's specific mention of the amount 
of the proposed reduction in the PART summaries makes this readily 
apparent. However, what is less clear is why these programs were cut 
when many other EPA programs could not demonstrate results either. OMB 
evaluated a total of 20 EPA programs in FY04 and could not determine 
results for 13 of them. Of those 13, some are proposed for decreases, 
some for increases and others for flat funding. The rationale for the 
uneven treatment is unclear.

Why was the STAR portion of these programs cut and who decided to cut 
                    it?

    OMB's PART review did not address the STAR program, other than a 
brief mention of NAS's positive review of the STAR-funded ecological 
research. As a result, it appears that the STAR cuts may have emerged 
from EPA as it decided how to allocate the overall reductions required 
by OMB. The ultimate reason, however, is difficult to know because 
final decisions on reductions are usually made in negotiations between 
the agency and OMB, which are not made public.

4. Witness Questions

Questions for Clay Johnson III, Deputy Director for Management, Office 
        of Management and Budget
    In your testimony, please describe the justification for the 
proposed reductions to EPA's STAR grant research on ecological systems, 
pollution prevention, endocrine disruptors, and mercury. In particular, 
please focus your testimony on the following questions:

        1.  To what extent is the proposed reduction in each of these 
        research areas based on the evaluation of EPA's research 
        programs that OMB undertook with the PART?

        2.  Given that the PART review did not specifically assess the 
        extramural portion of the research programs, why was the 
        extramural portion of the program cut?

        3.  Why does OMB's PART review of the ecological research 
        program tend to treat it as an applied research program when 
        EPA characterizes it more as basic research? Is a single 
        performance score under the PART tool appropriate for reviewing 
        a diverse research program, such as EPA's endocrine disruptor 
        research, that combines basic, applied, intramural and 
        extramural research? To what extent, should EPA's R&D 
        performance measures be based on specific regulatory program 
        outcomes or environmental outcomes?

        4.  Does OMB agree with the Government Accounting Office's 
        recommendations for improving the PART process and its content? 
        How will you implement those recommendations?
Questions for Paul Gilman, Assistant Administrator, Office of Research 
        and Development, U.S. EPA
    Please give a brief description of the STAR extramural research 
program and how it fits into EPA's overall R&D program. In addition, 
please answer the following questions:

        1.  Given the positive review by the National Academy of 
        Sciences of the STAR program last year, why was the STAR 
        program cut?

        2.  Given the elimination of EPA's STAR grants for research on 
        ecological systems, pollution prevention, endocrine disruptors 
        and mercury, does EPA now believe it no longer important to 
        seek out the expertise of university researchers in these 
        fields.

        3.  How does EPA's ecological research compare to research 
        supported by the National Science Foundation and other federal 
        research programs? To what extent does EPA coordinate its 
        research with those agencies?

        4.  Does EPA characterize its research on ecological systems, 
        pollution prevention, endocrine disruptors and mercury as basic 
        or applied research?

        5.  What performance measures are most appropriate for 
        evaluating these programs? To what extent does EPA believe that 
        the performance measures for these programs should be tied to 
        the outcomes of specific regulatory programs or environmental 
        outcomes?

        6.  What research would not be done as a result of the proposed 
        reductions and what impact would this have on our scientific 
        understanding and EPA's regulatory programs?
Questions for Paul Posner, Managing Director, Strategic Issues, U.S. 
        Government Accounting Office.
    In your testimony, please describe GAO's findings and 
recommendations concerning the PART and answer the following questions:

        1.  What unique problems do research programs raise for 
        evaluation tools like PART? What types of evaluation techniques 
        and performance measures are most appropriate for reviewing 
        basic research programs and what types are most appropriate for 
        applied research? How should OMB decide whether a research 
        program should be evaluated as a basic or applied program?

        2.  How should the PART deal with programs that have several 
        distinct elements, for instance a single research program that 
        funds both basic and applied, and intramural and extramural 
        research?
Questions for Gene Matanoski, Professor, Department of Epidemiology, 
        Johns Hopkins University; Chair, EPA Science Advisory Board 
        FY05 Budget; Former Chair, EPA Science Advisory Board
    In your testimony, please describe the Science Advisory Board's 
views on the proposed cuts to the STAR grant program and OMB's PART 
review of the ecological and pollution prevention research programs. In 
addition, please answer the following questions:

        1.  What has the SAB recommended to EPA in terms of balancing 
        its research and development (R&D) investments between 
        intramural and extramural research and between basic and 
        applied research? Are SAB's recommendations consistent with 
        recommendations from other reviews of EPA's science programs?

        2.  What performance measures are most appropriate for 
        evaluating EPA's research programs on ecological systems, 
        pollution prevention, endocrine disruptors and mercury? To what 
        extent should performance measures differ for basic and applied 
        research programs at EPA? Should EPA's R&D performance measures 
        be tied to the outcomes of specific regulatory programs or 
        environmental outcomes?
Questions for Costel Denson, Professor, Department of Engineering, 
        University of Delaware; Member of the National Research Council 
        panel for the report, The Measure of STAR

        1.  How important are the extramural portions of EPA's research 
        efforts, including those for research on ecological systems, 
        pollution prevention, endocrine disruptors, and mercury? What 
        are the likely effects the elimination of these grants will 
        have on our scientific understanding and EPA's regulatory 
        programs?

        2.  How important is it that EPA ensures that some portion of 
        its environmental research funding support extramural research? 
        What portion is an optimal amount?

        3.  What performance measures are most appropriate for judging 
        the performance of EPA's STAR grant program? To what extent 
        should the STAR program be evaluated as basic research or 
        applied? How well does the STAR program perform relative to 
        other federal research programs of similar design?

        4.  What actions should EPA take to strengthen its STAR 
        research grant program?

5. Attachments:

Attachment A. Summary of the NAS Report on STAR

Attachment B. Summary of the PART program and process.

Attachment C. Summary of the GAO Report on PART





Attachment B

                Summary of the PART program and process

What is the Program Assessment Rating Tool (PART)?

    The PART is the latest executive branch initiative designed 
to better align spending decisions and program performance, 
often called ``performance budgeting. The current statutory 
framework for performance budgeting is the Government 
Performance and Results Act, which became law in 1993. 
President George W. Bush announced his strong support for 
linking performance and budget, when he made it a major goal of 
the ``President's Management Agenda.'' A key element in 
accomplishing this objective is the Office of Management and 
Budget's PART. The Administration views the PART as enabling 
more effective implementation of GPRA, and as aligned more 
closely with budgeting decisions than GPRA.
    According to OMB, the PART is a diagnostic tool meant to 
provide a consistent approach to evaluating federal programs\7\ 
as part of the budget formulation process. The PART, which is 
implemented by OMB's budget examiners, requires an examiner to 
answer 25 yes/no questions under four overarching categories. 
Each of the four categories is given a specific weight for 
determining an overall numerical score for each program. The 
categories and their weightings are: (1) program purpose and 
design (e.g., is the design clear and purpose sensible?) (20 
percent); (2) strategic planning, (e.g., has the program set 
appropriate annual and long-term goals?) (10 percent); (3) 
program management, (is there sound financial and management 
oversight?) (20 percent); and (4) program results, (has the 
program is met its annual and long-term goals?) (50 percent). 
OMB asks a few supplementary questions, which vary depending on 
what type of program is under review, such as block grant, 
regulation, or R&D. Based on the total score, programs are 
rated as either effective, moderately effective, adequate, 
ineffective, or results not demonstrated.
---------------------------------------------------------------------------
    \7\ There is no standard definition of program, though it is 
intended to capture a set of activities clearly recognized as a 
program, having a discrete budget, or related to the level at which 
budget decisions are made.
---------------------------------------------------------------------------
    OMB applied the PART to 234 programs in FY 2004 across all 
federal agencies, and plans to rate nearly 100 percent of all 
remaining programs over the next four years. Of the 234 
programs, over 100 programs received ratings ``results not 
demonstrated.'' According to a recent GAO study, discretionary 
programs that received effective scores tended to see budget 
increases and programs that received ineffective scores tended 
to receive budget decreases. For programs that received 
``results not demonstrated'' scores, the budget story was more 
mixed. According to GAO, programs that received this score 
tended to indicate programs for which OMB and the Agency could 
not agree on appropriate performance measures.





    Chairman Ehlers. Good morning. Welcome to today's hearing 
on the Environmental Protection Agency's Fiscal Year 2005 
Science and Technology Budget. I will try to be brief with my 
opening statement so we can get right to the business of the 
day, and I will also note that Congressman Miller is here 
filling in for Congressman Udall, who is the Ranking Member.
    The role of science in technology at EPA is more important 
than ever. EPA is being asked to address increasingly complex 
technical questions in its regulatory processes and in its 
search for emerging environmental problems. That is why I am 
surprised and concerned at the Administration's proposal to cut 
$22 million, which is nearly 12 percent, from EPA's science 
budget. It is hard to understand why, even in this tight budget 
time, the science budget would deserve such a substantial cut.
    Today, the Subcommittee hopes to learn why the 
Administration proposes a cut of $35 million or 35 percent to 
EPA's competitive external research grants to colleges, 
universities and other researchers, as well as find out the 
impact of those reductions. This proposed cut is in the Science 
to Achieve Results grant program, affectionately referred to as 
STAR grant. It would mean less extramural research on 
ecological systems, pollution prevention, endocrine disrupters 
and mercury, among other topics.
    Today, we are also interested in the understanding of the 
extent to which the Office of Management Budget's effort to 
assess the performance of governing programs under its new 
Program Assessment Rating Tool called PART led to these 
proposed cuts. As a strong supporter of linking performance to 
funding decisions, I look forward to a healthy discussion on 
how we can best evaluate environmental research and 
developmental programs. The PART is playing an increasingly 
important role in the Administration's budget decisions. It has 
been applied to hundreds of federal programs in the last two 
years and will be applied to the remainder over the next four. 
With this in mind, it is critically important for this 
Committee and the Congress overall to understand how this 
assessment tool is being used.
    I would also like to mention my concern about the 8.3 
million reduction the Administration proposes that would 
eliminate EPA's research on building decontamination. EPA has a 
unique role in cleaning contaminated buildings and it is not 
clear why this work is being cut, or whether another agency, 
such as the Department of Homeland Security, will pick it up. 
The full Science Committee plans to address this question 
further at our joint hearing with a select Committee on 
Homeland Security on March 25.
    We have an excellent panel of witnesses today, and I expect 
that we will learn a great deal. I am looking forward to the 
testimony that we will hear.
    The Chair now recognizes Congressman Brad Miller from North 
Carolina for an opening statement.
    [The prepared statement of Mr. Ehlers follows:]
            Prepared Statement of Chairman Vernon J. Ehlers
    Good morning. Welcome to today's hearing on the Environmental 
Protection Agency's fiscal year 2005 Science and Technology budget.
    The role of science and technology at EPA is more important than 
ever. EPA is being asked to address increasingly complex technical 
questions in its regulatory processes and in its search for emerging 
environmental problems. That is why I am surprised and concerned about 
the Administration's proposal to cut $92 million, which is nearly 12 
percent, from EPA's Science budget. It is hard to understand why, even 
in this tight budget time, the science budget would deserve such a 
substantial cut.
    Today, the Subcommittee hopes to learn why the Administration 
proposes a cut of $35 million, or 35 percent, to EPA's competitive, 
external research grants to colleges, universities and other 
researchers, as well as the impact of those reductions. This proposed 
cut is in the Science to Achieve Results grant program, affectionately 
referred to as STAR grants. It would mean less extramural research on 
ecological systems, pollution prevention, endocrine disruptors and 
mercury, among other topics.
    Today, we are also interested in understanding the extent to which 
the Office of Management and Budget's effort to assess the performance 
of government programs under its new Program Assessment Rating Tool, 
called PART, led to these proposed cuts. As a strong supporter linking 
performance to funding decisions, I look forward to a healthy 
discussion of how we can best evaluate environmental research and 
development programs.
    The PART is playing an increasingly important role in the 
Administration's budget decisions. It has been applied to hundreds of 
federal programs in the last two years and will be applied to the 
remainder over the next four. With this in mind, it is critically 
important for this committee, and the Congress overall, to understand 
how this assessment tool is being used.
    I would also like to mention my concern about the $8.3 million 
reduction the Administration proposes that would eliminate EPA's 
research on building decontamination. EPA has a unique role in cleaning 
contaminated buildings and it is not clear why this work is being cut, 
or whether another agency, such as the Department of Homeland Security, 
will pick it up. The Full Science Committee plans to address this 
question further at our joint hearing with the Select Committee on 
Homeland Security on March 25th.
    We have an excellent panel of witness today and I expect that we 
will learn a great deal. I'm looking forward to the testimony.

    Mr. Miller. Thank you, Mr. Chairman.
    I am pleased to be here this morning to open this hearing 
in Mr. Udall's absence. I am also very pleased to hear how much 
common ground there is between my views and those just 
expressed by the Chair.
    I represent many of EPA's scientific staff who work at the 
Research Triangle Park in North Carolina. My interest, however, 
is more than parochial. I think I have the same interest in 
EPA's research and development funding that every American has 
or should have. In the 1960s, the Federal Government moved 
several air pollution-related environmental research facilities 
to the Research Triangle Park in North Carolina. This later 
evolved into a campus for the EPA, consisting of 400 
laboratories. The laboratories focus on a wide range of 
research issues facing our nation, including air, health, 
exposure and environmental information. The Research Triangle 
is home to the EPA's largest operation outside of Washington, 
with a labor force of more than 2,500, and a $250 million 
annual contribution to North Carolina's economy.
    You do not have to spend a lot of time on environmental 
issues to realize that they are very complicated and often very 
contentious. We have made great strides in balancing and 
recognizing the need to balance and in balancing economic 
development and environmental protection, and the ability to 
strike that balance is largely because of our investment in 
research and development. Protection of the environment is 
central to the protection of public health. Clean air and clean 
water are not luxuries. They are necessities. As a former 
member of the North Carolina legislature, I am very much aware 
of the problems caused by air pollution. Every summer, air 
pollution causes 240,000 asthma attacks, 6,300 emergency room 
visits, and 1,900 hospital admissions in North Carolina. Almost 
one in five emergency room admissions in North Carolina is for 
pediatric asthma. That is a terrified parent taking their child 
to the emergency room because their child cannot catch their 
breath.
    I will continue to work to improve that environmental 
health problem to protect North Carolina's children and North 
Carolina citizens against air pollution. I cannot reconcile the 
right to clean air and clean water and the constant call to 
base regulations and environmental policy on sound science with 
the Administration's budget request for this year for research 
and development at EPA. The request essentially guts R&D 
funding. From her written testimony, I understand that Dr. 
Matanoski of EPA's Science Advisory Board shares that opinion.
    The Administration has requested cuts in research and 
development funding for EPA to about 12 percent below the 
appropriated level for this year. Now last year's request 
proposed a cut to EPA's Science and Technology accounts by 
almost five percent. The Administration is obviously going in 
the wrong direction.
    The specific program cuts contained in this budget are 
unacceptable and difficult to understand. EPA's extramural 
grant program, which I believe the Chairman just mentioned, the 
Science to Achieve Results program, which has received high 
marks in external reviews by the Science Advisory Board and the 
National Academy of Sciences, is rewarded with a 
disproportionate cut of about one-third of its budget. During 
the same month, the Senate was closed due to contamination from 
ricin; this Administration released this budget terminating its 
research on building decontamination.
    This budget request does not serve the needs of my 
constituents, I think the constituents of many on the Committee 
or in the Congress, or maintain a healthy research and 
development program within the EPA. If this is the result of 
applying the Office of Management and Budget's new budget 
evaluation tool, the PART, clearly, the tool is deeply flawed.
    I do welcome our panel of experts, our witnesses, and I 
look forward to hearing your testimony today. Thank you.
    [The prepared statement of Mr. Miller follows:]
            Prepared Statement of Representative Brad Miller
    I am pleased to be here this morning to open this hearing. I have 
the privilege of representing many of EPA's fine scientific staff that 
works at Research Triangle Park in North Carolina. As such, I have not 
only a general interest in EPA's research and development funding, but 
a hometown one as well.
    In the 1960's, the U.S. government moved several air pollution-
related environmental research functions to Research Triangle Park, 
North Carolina. This later evolved into a campus for the EPA consisting 
of 400 laboratories. The laboratories focus on a wide range of research 
issues facing our nation including air, health, exposure and 
environmental information. The Research Triangle is home to the EPA's 
largest operation outside of Washington with a labor force of more than 
2,500, and a $250 million annual contribution to North Carolina's 
economy.
    You don't have to spend much time on environmental issues to 
recognize they are contentious and complex. We have made great strides 
in balancing economic development and environmental protection because 
we have invested in research and development. Protection of the 
environment is central to the protection of public health. Clean air 
and clean water are not luxuries. They are necessities. As a former 
member of the North Carolina legislature, I am acutely aware of the 
problems caused by air pollution. Every summer, air pollution causes 
240,000 asthma attacks, 6,300 ER visits, and 1,900 hospital admissions 
in our state. I will continue to fight this environmental and health 
problem.
    I cannot reconcile the right to clean air and water and the 
constant call to base regulations and environmental policy on sound 
science with the Administration's FY 2005 request for research and 
development at EPA. The request guts R&D funding. I note from her 
written testimony, that Dr. Matanoski of EPA's Science Advisory Board 
shares my opinion.
    The Administration's request cuts research and development funding 
for EPA by about 12 percent below the appropriated level for this year. 
Last year's request proposed a cut to EPA's Science and Technology 
accounts by almost five percent. The Administration is going in the 
wrong direction.
    The specific program cuts contained in this budget are unacceptable 
and difficult to understand. EPA's extramural grant program--the 
Science to Achieve Results program--which has received high marks in 
external reviews by the Science Advisory Board and the National Academy 
of sciences is rewarded with a disproportionate cut of about one third. 
During the same month the Senate was closed due to contamination from 
ricin, the Administration released this budget terminating its research 
on building decontamination.
    This budget request does not serve the needs of our constituents or 
maintain a healthy research and development program with the EPA. If 
this is the result of applying the Office of Management and Budget's 
new budget evaluation tool--the PART--clearly this tool is deeply 
flawed.
    I welcome our panel of witnesses and I look forward to hearing your 
testimony.

    Chairman Ehlers. If there is no objection, all additional 
opening statements submitted by the Subcommittee Members will 
be added to the record. Without objections, so ordered.
    I have to issue a correction. My staff said that I referred 
to the cut as $92 million in my opening statement. I referred 
to it as $22 million. It is actually $92 million. I hope that 
will be my only senior moment today, but I wanted to make clear 
that it is a very large amount of money.
    At this time, it is my pleasure to introduce our witnesses. 
We are honored today to have with us Mr. Clay Johnson, the 
Deputy Director of Management at the Office of Management and 
Budget. We welcome you. It is good to see you in this role. I 
spent many times on the telephone with you in your first job in 
the personnel office, trying to recruit good scientists to work 
in the government and the Bush Administration, and it is a 
pleasure to see you in this role now, so thank you for coming.
    Next, we have Dr. Paul Gilman, who is no stranger in this 
room. He has appeared here frequently. He is the Assistant 
Administrator for Research and Development at the United States 
Environmental Protection Agency, of course, always known by its 
acronym EPA.
    Next, Mr. Paul Posner. Is it Posner or Posner? Posner. 
Posner. Sorry. Mr. Paul Posner is the Managing Director for 
Budget and Intergovernmental Relations at the General 
Accounting Office, which we all know by GAO. He offers--oversaw 
GAO's recent report on how PART is being implemented.
    Next, we are pleased to have Dr. Matanoski, who is a 
Professor in the Department of Epidemiology at Johns Hopkins 
University. She also chaired the EPA Science Advisory Board 
Review of EPA's Fiscal Year 2005 Budget. While I introduce Dr. 
Matanoski, I would also like to recognize Dr. William Glaze, 
who is the current chair of EPA's Science Advisory Board, who 
has helped lead the budget.
    And finally, Dr. Denson is a Professor in the Department of 
Engineering at the University of Delaware. He is also a Member 
of the National Academy of Science's Panel that reviewed the 
EPA's Science to Achieve Results Research Grants Program, 
better known by its acronym STAR. And now I would like to 
introduce not only the witnesses, but all the acronyms. We can 
soon proceed to the testimony. Our witnesses I presume have 
been informed that spoken testimony is limited to five minutes 
each, after which the Members of the Committee will have five 
minutes each to ask questions. If you cannot complete your 
statement, obviously, your written statement will go in the 
record. If you do not feel you have had a chance to say 
everything you wanted to say in your opening oral statement, 
you will have ample opportunity during the questions to raise 
any points you believe should be raised.
    At this point, we will open our first round of questions. 
The Chair recognizes himself for five minutes. Oh, I am sorry. 
That is my second senior moment. It is a bad day today. I was 
ill last night, but that is no excuse. A Member of the Minority 
wants me to lose count. I suppose he would like me to forget 
who is the minority here too. Okay. We will start with Mr. 
Johnson.

    STATEMENT OF MR. CLAY JOHNSON, III, DEPUTY DIRECTOR OF 
          MANAGEMENT, OFFICE OF MANAGEMENT AND BUDGET

    Mr. Johnson. Mr. Chairman, Mr. Miller, thank you for having 
me.
    We, all of us, are interested and working very hard to make 
the Federal Government results oriented, and I would suggest, 
Mr. Miller, that in fact, we are hitting in the right 
direction, not the wrong direction. The right direction is let 
us pay attention to the results we are getting for the money we 
are spending, and not how much money we are spending. And I 
think this is your interest and our interest as well, as I 
think this is the direction that in fact the Federal Government 
is heading.
    EPA is one of the leaders in the Federal Government at 
focusing on results. They are particularly strong in the 
financial management area in the way we keep score, the score 
card. They are one of the few agencies that have achieved 
green, which just means they are using accurate and timely 
financial information with which to make regular, frequent 
management decisions. So they are one of the agencies we are 
very, very proud of in terms of the accomplishments they have 
made.
    We are here to talk about two things, as I understand it. 
One of them is the PART and one of them is the specific 
decision that has been with regard to ecological research and 
the Pollution Prevention Program. The PART helps agencies look 
at their programs with consistency. I would suggest that it is 
a valuable tool now. I believe this is GAO's assessment of the 
program, and it is a tool that will get--it is a device that 
will get better every year. There is nothing magic about the 
PART. There is nothing sacrosanct about the 25 or 30 questions 
we ask. What is magic about it is it is a device now that will 
get better over time that--with which we can look at programs 
and ask ourselves key management structure and results, 
questions about programs in a consistent fashion, and focus 
more and more with increasing proficiency as to what we are 
getting for our money. Nothing happens automatically, as a 
result of a PART evaluation, but the information that comes out 
of the PART assessment is used to help inform decisions about 
how to better manage, better structure, and better fund 
programs, including research programs.
    The two programs in particular that are the primary focus 
of this hearing, the Ecological Research and Pollution 
Prevention Programs, were assessed with the use of the PART. 
They were considered to be less results-oriented than they 
could be or than other programs of a similar nature. In fact, 
EPA's Prevention, Pesticides and Toxics Program were considered 
to be most results-oriented, and in fact, the money that is 
recommended not be spent, the reductions that were recommended 
for these two programs in fact is being put in this other 
program, where the belief is that the return for the taxpayer 
on the $30 million will be greater than it would be if it was 
spent in the other programs. These two programs are well-funded 
at the recommended 2005 levels. They are reasonably well-funded 
programs. Therefore, we believe we do not significantly impair 
the focus of these research programs with these reductions, but 
we do produce a greater return for the taxpayer.
    The PART was used to inform this budget proposal, but there 
was nothing automatic that flowed out of this PART assessment. 
These programs were not as results-oriented as they could be, 
or we thought should be. The other obviously major part that 
went in this funding recommendation was the very tight fiscal 
situation we find ourselves in, trying to produce very, very 
tight budgets, not related to Homeland Security and Defense. 
But I look forward to your questions later on. Thank you.
    [The prepared statement of Mr. Johnson follows:]
                Prepared Statement of Clay Johnson, III

Introduction

    Thank you, Mr. Chairman, Members of the Committee, for inviting me 
to testify this morning. I want to discuss with you our assessment of 
the Environmental Protection Agency's (EPA) research programs and 
describe how the President's Management Agenda helps federal agencies 
get greater results on behalf of the American people.
    We, all of us, are in the process of making the Federal Government 
results-oriented. We here in Washington tend to focus on the amount of 
money we're spending as a validation for how much the Federal 
Government is committed to an objective. As a part of becoming results-
oriented, however, we are now focusing more heavily on the results we 
achieve on behalf of the American people. With just a little help from 
OMB, agencies are asking whether they are achieving their objectives as 
effectively and efficiently as possible. EPA is a leader in this 
effort.
    EPA is as advanced as any agency in government in having and using 
accurate financial information to make day-to-day decisions about 
program management. For example, EPA negotiates performance commitments 
with its grantees and provides resources based on those commitments. 
EPA regularly monitors grantees' performance and expenditures and, if a 
grantee isn't meeting its commitments, EPA may withhold resources from 
the non-performers and redirect those resources to grantees that are 
meeting their commitments.

The Program Assessment Rating Tool

    Applying the Program Assessment Rating Tool (PART) is one of the 
ways we are becoming results-oriented. The PART is a series of 
questions that assesses the purpose, strategic planning, management, 
and performance of individual programs. Programs must demonstrate that 
their purpose is clear, that they set aggressive, outcome-oriented 
long- and short-term goals, that they are well managed, and that they 
achieve results. With this tool, we are assessing the performance of 
every federal program, and if it is not working as intended, we are 
trying to do something about it.
    The Administration has used the PART to assess 400 programs so far, 
representing approximately $1 trillion in federal spending. We are 
using these assessments not only to guide our budget decisions, but 
also to improve the performance and management of the government's 
programs. The purpose of asking whether programs are working is to 
figure out how to fix them, not whether to spend more or less on them.

Ecological Research and Pollution Prevention PARTs

    As you know, OMB and EPA assessed EPA's Ecological Research program 
and Pollution Prevention and New Technologies program using the PART. 
According to the assessment, the Ecological Research program:

          did not adequately coordinate the expenditure of 
        resources with other EPA offices or other agencies;

          lacks adequate annual measures of its performance; 
        and

          does not have sufficient evaluations of its 
        performance.

    Like nearly 40 percent of the programs evaluated using the PART, 
the principal finding for the program was the lack of adequate 
performance measures. Therefore, EPA has committed to finding the right 
measures for this important program. The President has requested $110 
million for this program in his FY 2005 Budget, down from $132 million 
in FY 2004.
    According to the assessment, the Pollution Prevention and New 
Technologies program:

          has not addressed findings made by independent 
        evaluations; and

          has not developed adequate measures of its 
        performance.

    As a result of these findings, EPA has committed to developing 
adequate performance measures and addressing findings made in previous 
independent evaluations. The President has requested $36 million for 
this program in his FY 2005 Budget, down from $42 million in FY 2004.

Why reduce funding for these programs?

    Both the Ecological Research and Pollution Prevention programs were 
``unable to demonstrate results,'' which clearly influenced funding 
decisions related to the programs. Especially in a year like this one, 
when resources are constrained, we should be directing resources to 
those programs that can achieve the most for the money. EPA and OMB 
used the PARTs for the Ecological Research and Pollution Prevention 
programs as one factor in making budget decisions about those programs 
and to focus resources on the programs most effective in helping EPA 
accomplish its mission.
    As I've mentioned, the Pollution Prevention research program could 
not show whether the tools it is developing are used by industry, and, 
if so, to what extent they are used. Also, previous independent 
evaluations of the Pollution Prevention research program concurred with 
the PART review, especially in the areas of strategic planning and 
measurable results. On the other hand, a similar program in the EPA's 
Office of Prevention, Pesticides, and Toxics was able to show that 
industry reduced its use and emissions of toxic chemicals through the 
use of tools and methods developed by the program. We consider 
reductions in pollution to be one of the highest-level outcomes of an 
environmental program's performance. Therefore, we redirected funds to 
the pollution prevention program so EPA can continue to achieve 
pollution reduction, thereby positively impacting the quality of public 
health and the environment. Despite redirection of a small amount of 
funds from pollution prevention research to OPPTS's program, the 
Administration maintained a large amount of funding for the pollution 
prevention research program to assist it, among other things, in 
developing performance measures.
    This is our rationale for funding decisions related to EPA's 
research programs. I will leave to Dr. Gilman a more robust discussion 
of how these funding decisions were applied to specific components of 
the research programs.

Research and Development and the Investment Criteria

    The Government's investment in research & development, not only in 
the environmental arena but elsewhere, is substantial. But in a time of 
constrained resources, it is imperative that we invest in R&D wisely. 
In recognition of the special challenges that measuring R&D programs 
present, and leveraging work done by the National Academies of Science, 
the Administration developed its R&D Investment Criteria, which were 
incorporated into the PART. These criteria are some of the things we 
look at when assessing the value of particular R&D programs:

          Relevance. Programs must be able to articulate why 
        they are important, relevant, and appropriate for federal 
        investment;

          Quality. Programs must justify how funds will be 
        allocated to ensure quality; and

          Performance. Programs must be able to monitor and 
        document how well the investments are performing.

    As noted in our PART evaluations, the programs we assessed could 
improve the ways they measure their performance. The three EPA programs 
we assessed cover important issues, and receive funding totaling 
approximately $210 million. We strongly believe that programs with 
federal funding of this magnitude should be able to monitor and 
document how these investments are performing. There are other equally 
important programs that are receiving similar levels of funding, but 
whose results are more measurable. For example, the Department of 
Energy's Wind Energy program, with proposed FY 2005 funding of $42 
million, can demonstrate its contributions to the commercial success of 
wind energy use throughout the United States. The Federal Aviation 
Administration's Research, Engineering and Development program, with 
proposed FY 2005 funding of $117 million, has set a long-term goal to 
produce turbulence forecasting products that allow pilots to avoid 
hazardous flight conditions while improving safety and ensuring 
efficient airspace use.

The Future of the PART

    The PART is a vehicle for improving program performance. It is just 
a tool to achieve the goals laid out by Congress in the Government 
Performance and Results Act (GPRA). GAO has made a number of 
recommendations for improving the PART, the vast majority of which we 
agree with and are addressing. For example:

          With respect to centrally monitoring PART 
        recommendations, we have provided a simple format for agencies 
        to follow when reporting the status of recommendation 
        implementation to OMB and I receive these reports semi-
        annually. We will continue to refine this process so that 
        sufficient attention is given to recommendation follow-up.

          As the PART relies on separate evaluations of 
        evidence of a program's success, we agree with GAO that the 
        judgment about what constitutes a sufficient evaluation should 
        be based on the quality, in addition to the independence, of 
        the evaluation.

          One of the greatest opportunities for the PART is to 
        compare the performance of, and share best practices among, 
        like programs across government. We will continue to use the 
        PART for that purpose.

          We will continue to improve agency and Executive 
        Branch implementation of GPRA by insisting GPRA plans and 
        reports meet the requirements of this important law and the 
        high standards set by the PART.

          We are clarifying the PART guidance so that it is 
        well understood by those who have to use it, as well those who 
        have to administer it. We will continue to assess completed 
        PARTs to ensure they are completed consistently by agencies and 
        OMB.

Conclusion

    The PART is a valuable tool now, as the General Accounting Office 
and others have asserted, and it will get better each year. As more and 
more program assessments are conducted, the vast majority of budget and 
management decisions will be significantly influenced by information 
about how programs are performing. Agencies, including EPA, will be 
better able to describe to Congress and the taxpayer what his or her 
funding is purchasing and will be managing so that each year 
improvements in efficiency and service delivery can be documented.
                    Biography for Clay Johnson, III
    Clay Johnson is the Deputy Director for Management at the Office of 
Management and Budget. The Deputy Director for Management provides 
government-wide leadership to Executive Branch agencies to improve 
agency and program performance. Prior to this he was the Assistant to 
the President for Presidential Personnel, responsible for the 
organization that identifies and recruits approximately 4,000 senior 
officials, middle management personnel and part-time board and 
commission members.
    From 1995 to 2000, Mr. Johnson worked with Governor George W. Bush 
in Austin, first as his Appointments Director, then as his Chief of 
Staff, and then as the Executive Director of the Bush-Cheney 
Transition.
    Mr. Johnson has been the Chief Operating Officer for the Dallas 
Museum of Art and the President of the Horchow and Neiman Marcus Mail 
Order companies. He also has worked for Citicorp, Wilson Sporting Goods 
and Frito Lay.
    He received his undergraduate degree from Yale University and a 
Master's degree from MIT's Sloan School of Management. In Austin, he 
helped create the Texas State History Museum, and was also an Adjunct 
Professor at the University of Texas Graduate School of Business. In 
Dallas, he served as President of the Board of Trustees for St. Marks 
School of Texas, and as a Board Member of Equitable Bankshares, 
Goodwill Industries of Dallas, and the Dallas Chapter of the Young 
Presidents Organization.

    Chairman Ehlers. Thank you.
    Dr. Gilman.

   STATEMENT OF DR. PAUL GILMAN, ASSISTANT ADMINISTRATOR FOR 
     RESEARCH AND DEVELOPMENT, UNITED STATES ENVIRONMENTAL 
                       PROTECTION AGENCY

    Dr. Gilman. Thank you, Mr. Chairman, Members of the 
Committee. Thanks for the opportunity to be here. I am here as 
the Assistant Administrator for the Office of Research and 
Development. I also serve as Science Advisor for the Agency.
    The Office of Research and Development is a service element 
to the various programs of the Agency. It does both basic and 
applied research. Roughly half basic, half applied. It does 
research in human health, public health and environmental and 
ecological sciences. Again, approximately a 60/40 split, in 
terms of human health and environmental health. There is a 
great emphasis on quality in the research program. It ranges 
from the very earliest stages of planning. We have instituted a 
planning process that develops multi-year plans for each of the 
major research areas. Those plans are developed in 
collaboration with our customers, if you will, the various 
programs of the agency; water, air, land, and coordinated with 
other federal agencies and outside entities, as well.
    We have gone from an Agency that was viewed as a laggard in 
the use of peer-review to, I think today, an agency viewed as a 
leader in the use of peer-review, and the Extramural Grants 
Program, as you will hear later from the witness representing 
the National Research Council, is a well-respected program as 
well. We place a great emphasis on collaborative research with 
other federal agencies in order to stretch our dollars and to 
avail ourselves of their expertise. Probably a very good 
example of that collaborative effort is a report we just 
released two days ago on the health of our nation's coastal 
waters, where we collaborated with 28 different states, a 
number of federal agencies; NOAA, Department of Interior, 
Agriculture, to really produce the first scientifically 
supportable assessment of the quality of coastal waters that we 
have had in a comprehensive way.
    The programs of particular matter, ecological research and 
pollution prevention all have tough goals set for them by our 
Agency, and embedded in our multi-year plans. Like providing 
the data and tools to predict, measure, reduce and meet the 
standards for particulate matter, as well as producing that 
report I just mentioned on the coastal health of our 
ecosystems. Yet, we are still challenged to provide measures 
that truly get into the performance of those programs. I am 
committed to working with the Office of Management and Budget 
and others to create long-term annual and efficiency measures 
that capture the important work our programs are doing. In the 
end, these measures will help advance our programs by 
demonstrating the value of our achievements very clearly.
    Let me say something about some recent accomplishments to 
demonstrate the breadth of the work that we are doing. These 
goals and accomplishments all draw on the STAR program that is 
the subject of this hearing. We have recently done work that is 
directly applicable to protecting water quality, a mission for 
the Agency, that also has Homeland Security implications 
towards better understanding of how water distribution systems 
actually work, not the pre-treatment but the post-treatment 
side of the water system. We are working to improve our air 
models, to improve their performance and their accuracy. We are 
working to develop DNA-based technologies for the 
identifications of things like common household molds that are 
deleterious to our health. And a development like that is the 
kind of thing we are very proud of because we are in the 
process of licensing it to a number of companies, with over 15 
companies licensing that particular technology.
    Again, Mr. Chairman, it is a challenge for us to relate the 
results of our research on inherently long-term research to 
measures for environment and public health outcomes. We are 
committed to doing it. We have been trying to develop some 
approaches in collaboration with our Inspector General. We are 
also very interested in continuing our discussions with the 
Office of Management and Budget to utilize some different 
approaches to the evaluation of these programs, and I think 
they will prove to be effective.
    Thank you, Mr. Chairman.
    [The prepared statement of Dr. Gilman follows:]
                   Prepared Statement of Paul Gilman

Introduction

    Good morning Mr. Chairman and Members of the Subcommittee. I am 
honored to appear before you today to discuss the Fiscal Year (FY) 2005 
budget request for the U.S. Environmental Protection Agency's (EPA) 
Office of Research and Development (ORD), and to share with you the 
uniqueness and success of ORD's research program from my perspective as 
both the Assistant Administrator for ORD and the EPA Science Advisor.
    The President's FY 2005 budget request for ORD is $572.2 million. 
This includes funding for ORD's in-house program carried out by 1,975 
employees, who account for 11 percent of EPA's workforce. In addition, 
the budget request supports our Science to Achieve Results (STAR) 
research grants program. Together, our in-house and STAR programs allow 
our nation's brightest scientists to apply their talents and knowledge 
to solve environmental science problems. My testimony highlights the 
contributions we and our partners have made and describes changes to 
the Agency's research budget for STAR research in FY 2005.

ORD's Unique Contributions

    ORD conducts leading-edge research and fosters the use of science 
and technology in environmental decisions in support of EPA's mission 
to protect human health and safeguard the environment. This research 
tackles problems to which solutions will have both immediate and long-
term public health and environmental benefits. The advancement of 
science and the development of answers to questions posed by 
environmental issues makes ORD unique among federal research agencies. 
No other federal agency has a comprehensive research program devoted to 
improving our understanding of both public health and environmental 
impacts. No other agency is researching these issues in an integrated 
fashion. In addition, no other agency can claim as large an impact on 
ensuring EPA's decisions are informed by the strongest possible 
science. To further strengthen our science program, EPA has been 
implementing the National Research Council (NRC) recommendations in its 
2000 report, ``Strengthening Science at the U.S. Environmental 
Protection Agency: Research Management and Peer Review Practices,'' as 
I will describe below. In sum, ORD is conducting leading-edge research 
that informs the risk-based environmental decision-making of EPA's 
program and regional offices and helps States and Tribes decide how 
best to implement these policies.
    Ensuring these decisions are based on sound science requires 
relevant, high quality, integrated, leading-edge research in human 
health, ecology, pollution prevention and control, and socio-economics. 
To maintain both short- and long-term relevance to EPA's mission, ORD's 
scientific research activities are mainly focused on applied research, 
which is problem-driven and, to a lesser extent, basic research. To 
ensure the quality of our research program, ORD uses a coordinated, 
cooperative research planning process; rigorous, independent peer 
review; and interagency partnerships and extramural grants to academia 
that complement EPA's own in-house scientific expertise. We have a 
uniquely integrated research program in that we address both human and 
ecological endpoints, conduct research across the risk assessment/risk 
management paradigm, have expertise across scientific disciplines and 
within the different environmental media, and draw from expertise in 
other agencies, organizations, and academia. Lastly, ORD keeps a 
leading edge in research by focusing our efforts and resources on those 
areas where EPA can add the most value toward reducing uncertainty in 
risk assessments and enhancing environmental management.
    The following are a few examples of our more recent 
accomplishments. ORD researchers:

          Collaborated with the Department of Homeland 
        Security, Department of Energy, Department of Defense, and 
        Centers for Disease Control to strengthen water security, 
        develop rapid risk assessment techniques, and develop building 
        decontamination methods.

          Partnered with 24 marine coastal States, four 
        territories, and other federal agencies through the 
        Environmental Monitoring and Assessment Program's National 
        Coastal Assessment, to conduct sampling of estuaries using 
        probabilistic methods.

          Collaborated with EPA's Office of Environmental 
        Information to deliver the draft Report on the Environment, the 
        first-ever national picture of U.S. environmental quality and 
        human health using science-based indicators.

          Developed the Computational Toxicology Program, which 
        has moved EPA to the leading edge in the use of genetics, 
        genomics, and computation for environmental protection.

          Completed an evaluation of Superfund clean-up 
        technologies citing 143 successfully demonstrated technologies 
        and $2.6 billion in total inflation-adjusted cost savings.

          Continued our tradition of leadership in the use of 
        external scientific expertise to enhance the quality and 
        relevance of our scientific products, by relying on the 
        processes of peer participation and peer review.

    I am proud of these accomplishments and the others I will identify 
later. They are the direct result of careful research planning that 
relies on the active involvement of the Agency's program and regional 
offices, as well as outside peer input.

Research Planning

    The President's budget request for FY 2005 will allow us to build 
upon these accomplishments by continuing a research program that 
directly serves EPA's mission. EPA's science and technology efforts are 
aligned with the Agency's strategic goals, and we now have gone a step 
further by including science objectives within each of EPA's five 
strategic goals. ORD created these science objectives in collaboration 
with EPA's program and regional offices, to ensure that we produce the 
right scientific and technical information to meet EPA's programmatic 
needs and thereby advance the Agency's mission.
    The alignment of our science and technology program with EPA's 
strategic goals is carried forward into ORD's planning of our research 
and development program. We have divided our R&D program into topical 
areas, each of which is guided by a multi-year research plan (the plans 
can be found at www.epa.gov/osp/myp.htm). Each multi-year plan contains 
long-term research and development goals for the next 5-10 years that 
tie back to EPA's strategic goals, and are supported by annual 
performance goals and measures. Every multi-year plan, and the goals 
and measures that comprise the plan, is developed in concert with 
colleagues across EPA and in consultation with our stakeholders and the 
broader scientific community. The plans also undergo expert, external 
peer review by EPA's Science Advisory Board (SAB) and ORD's Board of 
Scientific Counselors (BOSC). Both groups have endorsed this research 
planning process.
    The multi-year plans are ``road maps'' that mark the progress our 
research programs have already made, as well as lay out the new 
directions we are taking to adjust as changes occur in the complex 
scientific landscape ahead. Developing this road map requires 
identifying a logical progression of scientific research to be 
contributed by EPA and its partners. This progression is defined in 
each multi-year plan using ``logic models'' that demonstrate how 
research results contribute to EPA's desired long-term outcomes of 
improved human and ecosystem health. By following the logic diagram, 
one can begin to see how each research project contributes to the 
achievement of the long-term outcome. For illustration purposes, I have 
attached the logic diagram from our Particulate Matter (PM) multi-year 
plan. I discuss logic models in greater detail later in this testimony.
    The multi-year research plans help EPA maintain its focus on high-
priority science issues. They also assist in coordinating research 
efforts across the environmental science community, including other 
federal entities; State, tribal, and local governments; international 
organizations; and academia. Such coordination is essential. EPA's 
science and technology budget is only a small fraction of the total 
annual expenditures on environmental research, so leveraging our 
efforts with others--and, most important, identifying the appropriate 
niche for EPA's science and technology programs--is necessary for our 
doing the right science in a fiscally responsible manner.
    Independent scientific bodies have lauded EPA's process for 
planning its research efforts. In its 2000 publication, Strengthening 
Science at the U.S. Environmental Protection Agency, the National 
Research Council stated, ``Our committee expects that ORD's recent 
efforts in multi-year planning will contribute greatly to research 
program continuity and the achievement of strategic goals, and ORD 
merits commendation for these initiatives.'' Four years later, I can 
state with confidence that our research planning process is meeting--
and perhaps exceeding--the NRC's expectations.
    I wish to discuss two of our research programs--airborne 
particulate matter and ecosystem protection--to illustrate how EPA's 
science complements the scientific work of others, to advance 
scientific understanding and inform the decisions that solve 
environmental problems. Both of these research programs were evaluated 
using the Program Assessment Rating Tool (PART). The principles and 
practices applied in the particulate matter and ecosystem protection 
programs are those used in each of EPA's research and development 
programs.

Particulate Matter

    Among the most serious environmental problems affecting the health 
of Americans is exposure to airborne particulate matter. Based on the 
best science available to us, these exposures contribute to the 
premature deaths of tens of thousands of Americans annually, as well as 
the hospitalization of children and adults for diseases such as asthma. 
This has been documented in the Office of Management and Budget's (OMB) 
``Thompson Report'' (68 Fed. Reg. 5492, 5499 (2003) ). To protect the 
public against these effects, the Clean Air Act calls for the 
promulgation and periodic review of National Ambient Air Quality 
Standards, or NAAQS. In the late 1990s, after such a review yielded new 
standards for fine PM (particles less than 2.5 microns in diameter), 
Congress authorized and appropriated funds to EPA for a greatly 
expanded PM research program, to be guided by advice from the National 
Research Council. I would like to describe how we have organized this 
program and share what we have learned.
    To deliver the best science needed to inform sound public policy 
decisions, we have worked with our Agency partners in the Office of Air 
and Radiation and the regions to develop a multi-year plan for PM 
research that looks forward a little more than a decade. This plan, 
which will be peer reviewed by the EPA Science Advisory Board later 
this year, describes research activities in two major areas: (1) PM 
health effects and exposure, to guide future reviews of the NAAQS to 
refine the type and amount of PM that needs to be controlled to protect 
public health; and (2) implementation tools, so that EPA, the States 
and the tribes, and the private sector can ensure that these standards 
are met.
    The PM multi-year plan integrates the strengths of our in-house 
scientists with those of the external scientific community, through the 
extensive use of our STAR research grants program, including the 
support of five PM Research Centers. In addition, EPA's researchers are 
coordinating their efforts with others in the public and private 
sectors, both domestically and internationally. For example, health 
research is being conducted overseas by several organizations, while in 
the United States, studies are being supported by industrial 
organizations including the Electric Power Research Institute and the 
Coordinating Research Council through their support of the Health 
Effects Institute (co-funded by EPA). Recently, EPA, the National 
Institute of Environmental Health Sciences, and the National Heart, 
Lung, and Blood Institute co-sponsored a workshop on the cardiovascular 
effects of environmental pollutants, and planning is now underway to 
develop joint Requests For Applications in the area of cardiovascular 
effects of PM exposure. Through these and other mechanisms, EPA 
contributes to and keeps abreast of the scientific advancements and 
initiatives in the PM area.
    What have we learned since the setting of the 1997 NAAQS? Here are 
a few examples:

          In 1997, questions were raised about the legitimacy 
        of findings showing associations between centrally-monitored PM 
        and health effects. We now understand that these monitors 
        actually do a good job at estimating population exposures, 
        which has lent further credence to the health associations 
        found in epidemiologic studies.

          While we knew of these associations between PM and 
        increased mortality in 1997, we were at something of a loss to 
        explain them biologically. Due to work done by both ORD in-
        house and STAR-supported extramural scientists, we now have 
        several plausible hypotheses for the biological mechanisms 
        leading to those associations, including recent findings 
        showing an effect of PM directly on the heart.

          In 1997, we had a poor understanding of the chemical 
        composition and size distribution of PM that correlated with 
        health effects. Today, we have detailed profiles of the PM 
        associated with many significant sources and geographic areas, 
        and we continue to refine our understanding about the specific 
        types of sources responsible for these public health risks.

    While EPA's PM research program has been a success, there continues 
to be more to learn, as described in the PM multi-year plan. One focus 
of the program in the coming years will be to integrate the methods of 
diverse disciplines to determine the specific types of PM, and their 
sources, that have the greatest effect on public health. This will 
allow future standards and control strategies to focus attention only 
on those sources of pollution that need to be addressed. Another major 
focus will be on understanding the effects of long-term exposures to 
PM, through the funding of a long-term epidemiologic study to be 
conducted as part of our STAR research program. Lastly, EPA will 
evaluate new technologies for reducing air pollution, examining the 
ability of controls to reduce emissions of many pollutants at once. The 
results of these efforts will inform EPA's future PM policies, to 
ensure these policies protect human health in the most effective ways.

Ecological Research

    Current ecological management approaches have made important 
contributions to improved environmental quality through greatly 
reducing emissions of pollutants from point sources and waste disposal 
sites, and reducing the mishandling of toxic or hazardous chemicals and 
pesticides. Future ecological problems, however, will likely be more 
subtle, potentially more far-reaching, and require very different 
solutions. Examples include non-point source pollution control, 
regional-scale effects of air pollutants on aquatic ecosystems, 
dislocations in ecologically and economically important species due to 
invasions by non-indigenous species, and the cumulative effects and 
synergistic interactions of multiple stressors on the health of aquatic 
species and communities.
    To deliver sound science for informed decision-making, EPA has 
focused its ecological research program to assess and compare risks to 
ecosystems, to protect and restore them, and to demonstrate progress in 
terms of ecological outcomes. The ecological research program also 
reflects the growing ethic of environmental stewardship and the 
recognition that the implementation of these ecological management 
approaches will be largely community and sector-based, place-based, and 
performance-based.
    Environment and natural resource research is coordinated 
government-wide through the Committee on Environmental and Natural 
Resources (CENR). EPA is an active member on this committee, whose goal 
is to increase the overall effectiveness and productivity of federal 
research and development in environmental issues. Given the current 
fiscal constraints, EPA believes it is more important than ever for 
federal agencies to collaborate and coordinate research activities. EPA 
has a long history of collaborating ecosystem research with the 
National Science Foundation. EPA plans to continue, and wherever 
appropriate enhance, its coordination with NSF and other agencies.
    ORD's Ecological Research Strategy underwent interagency peer 
review by the CENR in June 1997, and external review by the Science 
Advisory Board's Ecological Processes and Effects Committee in July 
1997. The final Strategy, published in June 1998, formed the basis for 
ORD's Ecological Research Multi-Year Plan, which describes how the 
Agency plans to align its resources to achieve the plan's goals, 
including the integration of ORD's in-house research efforts with those 
conducted by our STAR research grants program.
    The Ecological Research Multi-Year Plan lays out four critical 
scientific questions to be addressed and their associated research 
emphases and programmatic goals. These questions are:

          What is the current condition of ecosystems and what 
        are the trends in their condition over time? (Assessing 
        condition)

          How do natural ecological disturbances and human 
        activities affect ecosystems? How can we most accurately 
        diagnose the causes of ecosystem deterioration? (Diagnosis)

          How can we reliably predict the vulnerability of 
        ecosystems to harm from current resource development and 
        management practices? How can we predict the most probable 
        responses of ecosystems to best management and sustainable 
        development practices? (Forecasting)

          How can we most effectively control risks and manage 
        to protect ecosystems once they have been degraded? 
        (Restoration)

    The PART evaluation on the ecological research program found that 
the program addresses a clear and continuing need and that it is 
generally well-managed, with adequate grantee and resource oversight. 
Its work has lead to accomplishments such as the Environmental 
Monitoring and Assessment Program (EMAP) National Coastal Assessment 
accomplishments that I mentioned earlier. Additional examples include:

          Research methods and findings from ORD's EMAP have 
        enabled State and tribal water monitoring programs to obtain 
        more reliable data on the ecological condition of their streams 
        and rivers, at significantly lower cost than the methods they 
        had been using.

          ORD produced national guidelines on assessing 
        ecological risks. For the first time, these guidelines extend 
        the principles of EPA's risk assessment paradigms to assessing 
        and comparing risks to ecosystems.

          STAR researchers have developed and applied 
        integrated methods to model and evaluate the effect of 
        stressors on water quality. These include development of models 
        to: (1) estimate annual nutrient loading to Lake Tahoe from 
        atmospheric deposition, precipitation, stream discharge, 
        overland runoff, groundwater and shoreline erosion; (2) 
        estimate how ``build-out'' in urbanizing watersheds affects 
        nutrient cycling, water quality, and the ecological health of 
        rivers and streams in Gwynns Falls, Maryland; (3) evaluate the 
        effects of agricultural best management practices on stream 
        flow, sediment, and nitrate loadings in the lower Minnesota 
        River; and (4) contaminant loading and bioaccumulation in Lake 
        Erie.

    As described in the Ecological Research multi-year plan, however, 
we are committed to building upon these achievements, and in the 
future, the ecological research program will focus heavily on 
diagnosis, forecasting, and restoration research. This research will 
enable the Agency to implement performance oriented, place-based 
protection of ecological systems. Our challenge now is to translate 
these successes into performance measures that demonstrate the utility 
of the tools and other protocols that we develop. In particular, long-
term goals are difficult for any environmental program to develop, even 
more so for an environmental research program. I am committed to 
working with OMB and others to create long-term, annual, and efficiency 
measures that capture the important work our program is doing. In the 
end, these measures will help advance our program by demonstrating the 
value of our achievements.

Science Quality Across EPA

    While our comprehensive and collaborative research planning process 
guides EPA to do the right science, as EPA's Science Advisor, I believe 
EPA's integrated approach to scientific quality makes sure that we also 
do the science right, not only in ORD but across the Agency. The three 
pillars of this approach are our Quality System, Information Quality 
Guidelines, and Peer Review Policy.
    EPA's Quality System is the means by which we manage our scientific 
information in a systematic, organized manner. It provides a framework 
for planning, implementing, and assessing the scientific work performed 
by EPA and for carrying out quality assurance and quality control 
activities. Each EPA organization develops a quality management plan 
that describes its quality system in terms of the organizational 
structure, policy and procedures, functional responsibilities of 
management and staff, lines of authority, and necessary interfaces for 
the planning, implementing, documenting, and assessing of all 
activities conducted. At the individual project level, we develop 
quality assurance project plans that describe the necessary quality 
assurance, quality control, and other technical activities that must be 
implemented to ensure that work outputs will satisfy the stated 
performance criteria. The goals of the EPA Quality System are to ensure 
that environmental programs and decisions are supported by data of the 
type and quality needed and expected for their intended use, and that 
decisions involving environmental technology are supported by 
appropriate quality-assured engineering standards and practices.
    EPA recognizes that the Office of Management and Budget's 
Information Quality Guidelines, together with our own Information 
Quality Guidelines issued in October 2002, are an important step 
forward in the quest for quality. The OMB guidelines call for all 
federal agencies to develop quality performance goals, including 
procedures to assure quality before information is disseminated. In 
response to these guidelines, EPA has established a system for 
addressing complaints about the quality of information that the Agency 
has disseminated. We now have more than a year's worth of experience in 
addressing challenges to EPA information under the guidelines, and this 
experience has validated our belief that ensuring the quality of our 
scientific information is paramount to maintaining the integrity of, 
and the public's confidence in, EPA's policies and decisions.
    Consistent Agency-wide application of independent, expert 
scientific peer review has been an EPA priority for many years. Since 
issuing our peer review policy in 1993, we have taken several major 
steps to support and strengthen the policy. But proof of a policy's 
value lies in its implementation, and here also EPA has been very 
active to ensure that our peer review policy is not only understood 
across the Agency, but is applied rigorously across EPA's program and 
regional offices. EPA has in place a strong and extensive program for 
peer reviewing our scientific and technical work products.
    EPA's approach to peer review is articulated in our policy, Peer 
Review and Peer Involvement at the U.S. Environmental Protection 
Agency. In addition to the policy, EPA has published a handbook that 
provides detailed guidance for implementing the policy. The Peer Review 
Handbook can be found at www.epa.gov/osp/spc/2peerrev.htm. We believe 
this is one of the most advanced treatments of peer review for 
intramural research and scientific/technical analysis of any federal 
agency.
    Most of EPA's scientific and technical work products now undergo 
peer review. In 1995, the Agency identified 120 work products for peer 
review. In 2002, of 859 work products generated by or for EPA, only 111 
were deemed, usually because of their repetitive or routine nature, not 
to be candidates for peer review. So, we see that nearly 90 percent of 
our scientific and technical work products receive internal or external 
peer review. And 90 percent of those peer-reviewed products received 
independent, external review.
    We were confident enough in the strength of our peer review program 
that we made it a cornerstone of our Information Quality Guidelines. 
Since issuing our policy ten years ago, peer review has become a part 
of EPA's culture, and its use is widespread across the Agency. Our 
challenge for the future is to continue the significant progress we 
have achieved to date and, not being content with the status quo, to 
look for ways to enhance the use of peer review as a tool for ensuring 
that EPA's decisions are supported by a firm foundation of scientific 
and technical information.
    Doing the right science through forward-looking collaborative 
research planning, and doing the science right by adherence to 
information quality and peer review standards, have given EPA policy-
makers relevant, timely, and credible scientific information to guide 
Agency decisions.

ORD--Making a Difference

    ORD scientists are committed to generating products of the highest 
quality to ensure sound science informs Agency decision-making. Our 
successes have been numerous, and we continue to build upon them. I 
have highlighted below a sampling of such successes, to illustrate the 
depth, breadth, and relevance of our research programs' contributions 
to environmental science generally and to EPA's mission in particular. 
As these examples demonstrate, ORD's research program--as a major part 
of the entire EPA scientific endeavor--plays a critical role in 
protecting human health and safeguarding the environment.

          In July 2003, EPA conducted an important drinking 
        water distribution field study to map the movement of 
        contaminants in a water system. This research is helping water 
        system managers and emergency responders better predict how a 
        biological or chemical contaminant would react in a drinking 
        water system. This study ties directly into EPA's community 
        support and homeland security efforts.

          ORD, working with academia, developed the first air 
        quality model (Models-3/CMAQ) to use a ``one atmosphere'' 
        approach to simulate the interactions among many air 
        pollutants, which is necessary to achieve truly cost-effective 
        air pollution control strategies. This work is critical for 
        local air pollution forecasting, as well as supporting the 
        Agency's multi-pollutant control strategies.

          Working with the Department of Energy and the 
        National Oceanic and Atmospheric Administration, ORD is 
        researching exposures to air pollutants in complex terrains, 
        such as urban canyons created by high-rise buildings and 
        complex traffic patterns. This research combines field 
        monitoring with wind tunnel studies to refine exposure models 
        that can be applied to different U.S. cities.

          ORD developed toxicity methods for determining acute 
        and chronic toxicity to plants, invertebrates, and vertebrates, 
        using several different end points. ORD also participated in 
        the development of the Whole Effluent Toxicity Test Methods 
        Rule, which allows these methods to be used as a basis for 
        decision-making in the National Pollutant Discharge Elimination 
        Systems Program.

          ORD developed analytical methods for Cryptosporidium 
        and evaluated technologies that could be used for removing 
        Cryptosporidium from drinking water sources. ORD worked with 
        the Office of Water to use these results in promulgating the 
        Long-Term 2 Enhanced Surface Water Treatment Rule. This rule 
        will protect drinking water consumers, including sensitive sub-
        populations such as children, by avoiding Cryptosporidium 
        incidents that have resulted in health impacts and even death 
        in the past.

          EPA's cancer risk assessment prompted industry 
        decisions to phase-out the use of chromated copper arsenate 
        (CCA) treated wood in home settings, due to concerns of 
        exposure to kids from decks and play equipment. ORD is working 
        with other EPA scientists to analyze exposures to homeowners 
        and children from CCA-treated decks and play equipment and to 
        evaluate coatings and sealants that can be used to reduce risk 
        from exposure to CCA-treated wood.

          ORD developed a DNA-based system that allows rapid 
        identification and quantification of molds in a matter of 
        hours, as opposed to current methods that require days or even 
        weeks. The new technology can be used to detect the mold 
        Stachybotrys, commonly known as ``black mold,'' and more than 
        50 other possibly harmful molds. The new method has been 
        licensed to 13 companies for use in detecting mold, and four 
        additional licenses are pending.

          EPA chairs the coordination of endocrine disruptor 
        research across federal agencies through an interagency working 
        group under the Committee on Environment and Natural Resources, 
        under the President's National Science and Technology Council. 
        Through this interagency working group, EPA and its partners 
        issued two joint solicitations for research proposals to 
        address the critical data gaps of understanding the impact of 
        endocrine disruptors on humans and wildlife.

Linking Research Results to Outcomes

    EPA recognizes that research findings--no matter how insightful or 
cutting-edge--cannot of their own accord achieve environmental 
outcomes. Achieving environmental outcomes depends on decisions made 
and actions taken by the Agency's program and regional offices, as well 
as by our State and tribal partners. We are working with our Office of 
Inspector General (OIG) to develop better ways to describe the link 
between our research program and environmental and public health 
outcomes. Our efforts are focused on the use of a logic model that was 
developed by the OIG.
    The OIG, in collaboration with the ORD, piloted using the logic 
model to determine if the design of the Pollution Prevention and New 
Technology research program was conducive to achieving desired 
environmental outcomes. The pilot was successful, and we now employ 
logic models to clearly identify the outputs of our research and their 
associated near-term outcomes. Logic model techniques are particularly 
useful for identifying outputs and methods for transferring research 
results to our clients, helping them to achieve environmental outcomes.
    The logic model also emphasizes that there are factors outside the 
realm of science that may help or hinder the success of the program and 
the accomplishment of its results. ORD scientists, EPA program offices, 
and our State, tribal, and local clients each have their respective 
roles for helping to achieve environmental outcomes. In light of this, 
ORD believes that research programs are most appropriately evaluated 
with respect to the soundness of the research strategy, the 
significance of the research findings, and the usefulness of the 
resulting scientific tools or policies for their intended applications. 
We also believe there is an important role for independent, expert peer 
review for accomplishing such evaluations.
    ORD is moving forward with its plans to conduct reviews of its 
research programs by external independent experts. These expert panels 
will review our research in accordance with the Administration's 
investment criteria for research and development; namely, quality, 
relevance, and performance. These reviews will provide valuable input 
for determining that ORD is managing its programs to ensure scientific 
quality, and is providing relevant results for achieving the Agency's 
mission.
    It is a challenging task to relate research, especially inherently 
long-term research, to specific environmental and public health 
outcomes. However, as I mentioned earlier, I am committed to moving ORD 
in that direction. The PARTs conducted last year have provided up with 
valuable experience that will help us demonstrate the value of our 
programs, and we are working with OMB to develop recommendations to 
improve program performance.

Science to Achieve Results Research

    Mr. Chairman, your letter of invitation asked me to specifically 
address the reduction of EPA's STAR grants for research on ecological 
systems, pollution prevention, endocrine disruptors, and mercury. While 
I will address the specific reductions later in my testimony, I want to 
share at this point some of my thoughts and the thoughts of others 
about our STAR program and how it continues to be a vital part of ORD's 
research portfolio.
    In 1995, ORD created the Science To Achieve Results extramural 
research program. This program was created for the purpose of providing 
ORD swift, flexible access to nationally and internationally acclaimed 
scientists who could conduct independent and original research to 
complement the efforts of ORD's intramural research program.
    Since the program's inception, all or parts of the STAR program 
have been reviewed three times by the EPA SAB and twice by the NRC. 
These reviews have been very favorable, but have also noted areas for 
improvement. As the NRC is also a witness today, I will leave it to 
them to describe the findings of their 2003 review of the STAR program, 
The Measure of STAR.
    EPA has developed an in-house staff capability to address 
environmental research needs. In some cases, EPA lacks a critical mass 
of in-house expertise that can devote itself full-time to new research 
issues, and the STAR program enables ORD to quickly deploy resources to 
access nationally and internationally acclaimed scientists to conduct 
independent and original research where the Agency lacks capacity or 
specialized expertise.
    The STAR program remains strong and is aligned to most effectively 
support EPA's priority research needs. For example, STAR research 
efforts will be funded consistent with previous years' investments in 
important areas including children's health, particulate matter, safe 
food, and drinking water. In those areas where STAR will be eliminated 
in FY 2005 (ecological systems, pollution prevention, endocrine 
disruptors, and mercury), EPA will continue to conduct in-house 
research as well as look to increase its ongoing research partnerships 
with university researchers and initiate new ones. STAR currently 
leverages its resources through joint solicitations with 12 federal and 
private sector research partners, enabling EPA to enhance its research 
portfolio by about 30 to 50 additional grants.

FY 2005 President's Budget

    The President's FY 2005 budget request continues the tradition of 
ORD research excellence by emphasizing cutting-edge science and 
technology, collaboration with other agencies, and an orientation on 
results.
    Mr. Chairman, in your letter of invitation you asked me to identify 
what research would not be done as a result of the proposed reductions 
in the STAR grants program in the President's Budget request and the 
associated impacts. The following are areas of decreased STAR research.
Ecological Protection Research Grants (-$22.2M)
    EPA would no longer fund STAR grants in the area of ecological 
protection, a reduction of about 50 grants. In response to PART 
findings, EPA is working to develop long-term, annual, and efficiency 
performance measures for the program. Key areas of research at academic 
institutions across the Nation would no longer be conducted, affecting 
Agency efforts to assess ecosystem condition, diagnose ecosystem 
impairment, and forecast ecosystem health.
Hazardous Substance Research Centers (HSRCs) (-$2.25M)
    Eliminate most of the research in the fifth and final year of 
planned funding for the HSRCs, as well as the technical support and 
outreach efforts of the centers that directly support EPA regional, 
State, and tribal efforts to evaluate and manage risk at clean-up 
sites.
Mercury Research (-$2.0M)
    Eliminate STAR-supported university research in support of 
understanding the atmospheric processes that affect the transport, 
transformation, and deposition of mercury emissions from natural and 
anthropogenic sources.
Endocrine Disruptors (-$4.7M)
    Eliminate funding for the STAR portion of the Endocrine Disruptors 
research program. However, the President's Budget provides a $3.5 
million increase for EPA's computational toxicology program, which uses 
computational chemistry and molecular biology to more accurately 
predict health effects from chemicals, thereby improving linkages 
between potential exposure and disease. Our computational toxicology 
program offers more promising and timely application for our Endocrine 
Disruptors Screening Program.
Pollution Prevention and New Technologies (-$6.0M)
    Transfer funding of the research program to the Office of 
Pesticides, Prevention, and Toxic Substances pollution prevention 
program, which the PART analysis has shown a reduction in the use of 
chemicals and pollution. In response to PART findings, the program is 
working to develop long-term, annual, and efficiency performance 
measures.

Conclusion

    By uniquely combining human health and ecological research in one 
federal agency, ORD has made significant contributions to developing a 
better understanding of environmental risks to both human health and 
ecosystems. The results of this research have consistently and 
effectively informed EPA's environmental decision-making, leading to 
environmental policies based on sound science at the federal, State, 
tribal, and local level.
    The President's FY 2005 budget request for ORD continues this 
tradition of excellence, by emphasizing cutting-edge science and 
technology, collaboration with other agencies, and an orientation on 
results.
    Thank you.


    
    
    
                       Biography for Paul Gilman
    In April 2002, Dr. Gilman was sworn-in to serve as the Assistant 
Administrator for the Office of Research and Development which is the 
scientific and technological arm of the Environmental Protection 
Agency. In May 2002, he was appointed the Agency Science Advisor. In 
this capacity, he will be responsible for working across the Agency to 
ensure that the highest quality science is better integrated into the 
Agency's programs, policies and decisions.
    Before his confirmation, he was Director, Policy Planning for 
Celera Genomics in Rockville, Maryland. Celera Genomics, a bio 
information and drug discovery company, is known for having decoded the 
human genome. In his position Dr. Gilman was responsible for strategic 
planning for corporate development and communications.
    Prior to joining Celera, Dr. Gilman was the Executive Director of 
the life sciences and agriculture divisions of the National Research 
Council of the National Academies of Sciences and Engineering. The 
National Research Council is the operating arm of the National 
Academies which were chartered to provide independent advice to the 
government in matters of science and engineering. Dr. Gilman's 
divisions focused on risks to health and the environment, protection 
and management of biotic resources, and practical applications of 
biology including biotechnology and agriculture.
    Before joining the National Research Council, Gilman was the 
Associate Director of the Office of Management and Budget (OMB) for 
Natural Resources, Energy, and Science. There he coordinated budget 
formulation, regulatory, and legislative activities between agencies 
such as the Environmental Protection Agency, National Science 
Foundation, Agriculture, and Energy with the Executive Office of the 
President.
    Dr. Gilman served as Executive Assistant to the Secretary of Energy 
for technical matters before joining the OMB. His responsibilities 
included participating in policy deliberations and tracking 
implementation of a variety of programs including the Department's 
environmental remediation and basic science research.
    Gilman has 13 years of experience working on the staff of the 
United States Senate. He began that time as a Congressional Science 
Fellow sponsored by the American Association for the Advancement of 
Science in the office of Senator Pete V. Domenici. Later, as the Staff 
Director of the Subcommittee on Energy Research and Development, he was 
involved in the passage of the Nuclear Waste Policy Act of 1982 and 
oversight of energy technology and environmental research. Later he 
served as the chief-of-staff for Senator Domenici.
    Dr. Gilman matriculated at Kenyon College in Ohio and received his 
A.B., M.A., and Ph.D. degrees in ecology and evolutionary biology from 
Johns Hopkins University, Baltimore, Maryland.

    Chairman Ehlers. Thank you.
    Mr. Posner.

  STATEMENT OF MR. PAUL L. POSNER, MANAGING DIRECTOR, FEDERAL 
  BUDGET AND INTERGOVERNMENTAL RELATIONS, GENERAL ACCOUNTING 
                             OFFICE

    Mr. Posner. Thank you, Mr. Chairman.
    My purpose at this hearing is to talk about the report we 
did on the OMB PART and performance budgeting more in general. 
PART is really a new chapter in a long history, some would 
argue checkered history of performance budgeting, trying to 
apply performance to budgeting which is inherently in political 
process. One of the lessons we have learned from the failures 
of PPB, ZBB and OMB is it is a fool's errant. In fact, however, 
the fact that we keep trying suggest we won't accept failure 
for an answer because it is too important to try to link what 
we are trying to get out of these programs with our resource 
allocation. This is really the essence of what government 
should be about. GPRA has, in fact, stayed around much longer 
than its predecessors. We just issued a report today on that, 
showing that in fact for 10 years, it has sustained itself and 
built a credible supply for information and improved planning.
    What PART does is take that supply and attempt to more 
actively prompt decision-makers to use the information. Our 
review of the process in fiscal year 2004 suggests that PART 
succeeded in more explicitly developing linkages between 
performance and budgeting. It more transparently informed 
budget decisions than we have seen before. There is more public 
transparency about the results of the process. Having said 
that, there is not a direct link with funding, nor should there 
be, because budgeting is an exercise in political choice and 
priorities of which performance is but one input. In fact, what 
we really should expect performance budgeting to do is not 
change the answers in some mechanical way, but to change the 
kind of questions we ask. And in fact, this hearing is a very 
good example of what we are talking about, and I would like to 
see this in other areas. The focus we should have in our budget 
debate is what do we know about programs? What more should we 
know? What should we change in the metrics and the kinds of 
goals that we are setting for these programs? This is actually 
exactly what performance budgeting should be all about, what 
you are doing today in this hearing.
    Having said this, doing this is not easy. Developing a 
bottom line of federal programs is not easy. We are not a 
private sector business, where we have one profit and loss 
statement. We have multiple goals for every program, and it is 
really difficult to assign a single rating to various kinds of 
balanced portfolios or programs. So judgment is always 
required, regardless of what the number says. OMB has tried, in 
the development of the PART instrument and the application, to 
provide more consistency among its raters, among OMB staff and 
agency people. Some terms will inherently be subjective and 
require judgment; things like what constitutes an ambitious 
performance goal. We found inconsistencies in such things as 
defining what is an outcome versus an output in evaluating 
agencies.
    The format of the PART tool itself, with the yes, no kinds 
of answers for most questions, force some standardizations for 
areas where ultimately considerable judgment was required to 
balance answers across multiple criteria. The chronic lack of 
performance in evaluation across many programs, which we have 
long identified, remains a considerable barrier in doing this 
exercise, and one thing that we think PART may in fact 
instigate and instill is a greater incentive to get more of 
this done. Particular challenges, as you will note today in 
research and development, in terms of the basic nature of 
research, makes it more difficult. It takes time to really 
assess what we are getting out of research. Research itself, 
particularly basic research, is an uncertain enterprise. But 
these challenges are being addressed in agencies across the 
board, and we think the National Academy of Sciences has 
provided useful criteria.
    One important issue that remains--and when you try to judge 
multiple programs--is defining what a program is. It is 
arguably seemingly something that we should all know, but there 
is no uniform definition of what a program is, and so this 
question of what is the unit analysis for defining what a 
program is, is critical in terms of defining what we are 
getting out of government. This was an issue with GPRA, 
frankly, and the National Academy of Science report identified 
this, and it is clearly an issue with PART. Programs were 
defined one way for GPRA, for example, in more strategic ways. 
Under PART we are redefining programs for budget purposes in a 
more discrete, granular way. These are two different competing 
kinds of accountability mechanisms, and one issue we surfaced 
is the need to better synchronize between the two.
    Ultimately, we recommended continued improvement in the 
guidance and the rating tool by OMB. We recommended a more 
targeted selection process so that, in the future, related 
programs can be grouped together so we can look across 
different programs and see what we are getting. We recommended 
early consultation with the Congress, and most importantly, we 
suggested that Congress develop a process to better identify 
its performance issues and oversight priorities to enable it to 
better communicate with OMB and move this process forward.
    Thank you.
    [The prepared statement of Mr. Posner follows:]
                  Prepared Statement of Paul L. Posner

Mr. Chairman and Members of the Subcommittee:

    I am pleased to be here today to discuss performance budgeting and 
the Office of Management and Budget's (OMB) Program Assessment Rating 
Tool (PART). Since the 1950s, the Federal Government has attempted 
several government-wide initiatives designed to better align spending 
decisions with expected performance--what is commonly referred to as 
``performance budgeting.'' The consensus is that prior efforts--
including the Hoover Commission, the Planning-Programming-Budgeting-
System, Management by Objectives, and Zero-Based Budgeting--did not 
succeed in significantly shifting the focus of the federal budget 
process from its long-standing concentration on the items of government 
spending to the results of its programs. However, the persistent 
attempts reflect a long-standing interest in linking resources to 
results.
    In the 1990s, Congress and the executive branch laid out a 
statutory and management framework that provides the foundation for 
strengthening government performance and accountability, with the 
Government Performance and Results Act of 1993\1\ (GPRA) as its 
centerpiece. GPRA is designed to inform congressional and executive 
decision-making by providing objective information on the relative 
effectiveness and efficiency of federal programs and spending. A key 
purpose of the act is to create closer and clearer links between the 
process of allocating scarce resources and the expected results to be 
achieved with those resources. We have learned that this type of 
integration is critical from prior initiatives that failed in part 
because they did not prove to be relevant to budget decision-makers in 
the executive branch or Congress.\2\ GPRA requires both a connection to 
the structures used in congressional budget presentations and 
consultation between the executive and legislative branches on agency 
strategic plans; this gives Congress an oversight stake in GPRA's 
success.\3\
---------------------------------------------------------------------------
    \1\ Pub. L. No. 103-62 (1993).
    \2\ U.S. General Accounting Office, Performance Budgeting: Past 
Initiatives Offer Insights for GPRA Implementation, GAO/AIMD-97-46 
(Washington, D.C.: Mar. 27, 1997).
    \3\ See Pub. L. No. 103-62 2 (1993), 5 U.S.C. 306 (2003), and 31 
U.S.C. 1115-1116 (2003).
---------------------------------------------------------------------------
    This administration has made the integration of performance and 
budget information one of five government-wide management priorities 
under the President's Management Agenda (PMA).\4\ Central to this 
initiative is PART. OMB developed PART as a diagnostic tool meant to 
provide a consistent approach to evaluating federal programs and 
applied it in formulating the President's fiscal years 2004 and 2005 
budget requests. PART covers four broad topics for all ``programs'' \5\ 
selected for review: (1) program purpose and design, (2) strategic 
planning, (3) program management, and (4) program results (i.e., 
whether a program is meeting its long-term and annual goals) as well as 
additional questions that are specific to one of seven mechanisms or 
approaches used to deliver the program.\6\
---------------------------------------------------------------------------
    \4\ In addition to budget and performance integration, the other 
four priorities under the PMA are strategic management of human 
capital, expanded electronic government, improved financial 
performance, and competitive sourcing.
    \5\ There is no standard definition for the term ``program.'' For 
purposes of PART, OMB described the unit of analysis (program) as (1) 
an activity or set of activities clearly recognized as a program by the 
public, OMB, or Congress; (2) having a discrete level of funding 
clearly associated with it; and (3) corresponding to the level at which 
budget decisions are made.
    \6\ The seven major categories are competitive grants, block/
formula grants, capital assets and service acquisition programs, credit 
programs, regulatory-based programs, direct federal programs, and 
research and development programs.
---------------------------------------------------------------------------
    GPRA expanded the supply of performance information generated by 
federal agencies, although as the PART assessments demonstrate, more 
must be done to develop credible performance information. However, 
improving the supply of performance information is in and of itself 
insufficient to sustain performance management and achieve real 
improvements in management and program results. Rather, it needs to be 
accompanied by a demand for that information by decision-makers and 
managers alike. PART may mark a new chapter in performance-based 
budgeting by more successfully stimulating demand for this 
information--that is, using the performance information generated 
through GPRA's planning and reporting processes to more directly feed 
into executive branch budgetary decisions.
    My statement today focuses on seven points:

          PART helped structure OMB's use of performance 
        information for its internal program and budget analysis, made 
        the use of this information more transparent, and stimulated 
        agency interest in budget and performance integration. 
        Moreover, it illustrated the potential to build on GPRA's 
        foundation to more actively promote the use of performance 
        information in budget decisions.

          The goal of PART is to evaluate programs 
        systematically, consistently, and transparently. OMB went to 
        great lengths to encourage consistent application of PART in 
        the evaluation of government programs, including pilot testing 
        the instrument, issuing detailed guidance, and conducting 
        consistency reviews. Although there is undoubtedly room for 
        continued improvement, any tool is inherently limited in 
        providing a single performance answer or judgment on complex 
        federal programs with multiple goals.

          Performance measurement challenges in evaluating 
        complex federal programs make it difficult to meaningfully 
        interpret a bottom-line rating. The individual section ratings 
        for each PART review provided a better understanding of areas 
        needing improvement than the overall rating alone.

          As is to be expected with any new reform, PART is a 
        work in progress and we have noted in our work where OMB might 
        make improvements. Any tool that is sophisticated enough to 
        take into account the complexity of the U.S. government will 
        require some exercise of judgment. Therefore it is not 
        surprising that we found some inconsistencies in OMB staff 
        interpreting and applying PART.

          PART provides an opportunity to more efficiently use 
        scarce analytic resources, to focus decision-makers' attention 
        on the most pressing policy issues, and to consider comparisons 
        and trade-offs among related programs by more strategically 
        targeting PART assessments based on such factors as the 
        relative priorities, costs, and risks associated with related 
        clusters of programs and activities. The first year PART 
        assessments underscored the long-standing gaps in performance 
        and evaluation information throughout the Federal Government. 
        By reaching agreement on areas in which evaluations are most 
        essential, decision-makers can help ensure that limited 
        resources are applied wisely.

          The relationship between PART and its process and the 
        broader GPRA strategic planning process is still evolving. 
        Although PART can stimulate discussion on program-specific 
        performance measurement issues, it is not a substitute for 
        GPRA's strategic, longer-term focus on thematic goals and 
        department- and government-wide crosscutting comparisons. PART 
        and GPRA serve different but complementary needs, so a strategy 
        for integrating the two could help strengthen both.

          Federal programs are designed and implemented in 
        dynamic environments where competing program priorities and 
        stakeholders' needs must be balanced continually and new needs 
        must be addressed. While PART clearly serves the needs of OMB 
        in budget formulation, questions remain about whether it serves 
        the various needs of other key stakeholders. If the President 
        or OMB wants PART and its results to be considered in the 
        congressional debate, it will be important for OMB to (1) 
        involve congressional stakeholders early in providing input on 
        the focus of the assessments; (2) clarify any significant 
        limitations in the assessments as well as the underlying 
        performance information; and (3) initiate discussions with key 
        congressional committees about how they can best take advantage 
        of and leverage PART information in congressional 
        authorization, appropriations, and oversight processes. 
        Moreover, Congress needs to consider ways it can articulate its 
        oversight priorities and performance agenda.

    My statement is based on our recently published report on OMB's 
PART\7\ in which we reviewed the first year of the PART process--fiscal 
year 2004--and changes in the PART process initiated for fiscal year 
2005. We have not reviewed or analyzed the PART results for the fiscal 
year 2005 budget request. For this testimony, this subcommittee asked 
us to discuss our overall findings and recommendations concerning PART 
to help frame today's hearing. We conducted our work in accordance with 
generally accepted government auditing standards.
---------------------------------------------------------------------------
    \7\ U.S. General Accounting Office, Performance 
Budgeting:Observations on the Use of OMB's Program Assessment Rating 
Tool for the Fiscal Year 2004 Budget, GAO-04-174 (Washington, D.C.: 
Jan. 30, 2004).
---------------------------------------------------------------------------

Strengths and Weaknesses of PART in Its First Year of Implementation

    Through its development and use of PART, OMB has more explicitly 
infused performance information into the budget formulation process; 
increased the attention paid to performance information and program 
evaluations; and ultimately, we hope, increased the value of this 
information to decision-makers and other stakeholders. By linking 
performance information to the budget process, OMB has provided 
agencies with a powerful incentive for improving both the quality and 
availability of performance information. The level of effort and 
involvement by senior OMB officials and staff clearly signals the 
importance of this strategy in meeting the priorities outlined in the 
PMA. OMB should be credited with opening up for scrutiny--and potential 
criticism--its review of key areas of federal program performance and 
then making its assessments available to a potentially wider audience 
through its Web site.
    As OMB and others recognize, performance is not the only factor in 
funding decisions. Determining priorities--including funding 
priorities--is a function of competing values and interests. 
Accordingly, we found that while PART scores were generally positively 
related to proposed funding changes in discretionary programs, the 
scores did not automatically determine funding changes. That is, for 
some programs rated ``effective'' or ``moderately effective'' OMB 
recommended funding decreases, while for several programs judged to be 
``ineffective'' OMB recommended additional funding in the President's 
budget request with which to implement changes. In fact, the more 
important role of PART was not its use in making resource decisions, 
but in its support for recommendations to improve program design, 
assessment, and management. Our analysis of the fiscal year 2004 PART 
found that 82 percent of the recommendations addressed program 
assessment, design, and management issues; only 18 percent of the 
recommendations had a direct link to funding matters.\8\
---------------------------------------------------------------------------
    \8\ The 234 programs assessed for fiscal year 2004 contained a 
total of 612 recommendations.
---------------------------------------------------------------------------
    OMB's ability to use PART to identify and address future program 
improvements and measure progress--a major purpose of PART--depends on 
its ability to oversee the implementation of PART recommendations. As 
OMB has recognized, following through on these recommendations is 
essential for improving program performance and ensuring 
accountability. Currently, OMB plans to assess an additional 20 percent 
of all federal programs annually. As the number of recommendations from 
previous years' evaluations grows, a system for monitoring their 
implementation will become more critical. However, OMB does not have a 
centralized system to oversee the implementation of such 
recommendations or evaluate their effectiveness.
    The goal of PART is to evaluate programs systematically, 
consistently, and transparently. OMB went to great lengths to encourage 
consistent application of PART in the evaluation of government 
programs, including pilot testing the instrument, issuing detailed 
guidance, and conducting consistency reviews. Although there is 
undoubtedly room for continued improvement, any tool is inherently 
limited in providing a single performance answer or judgment on complex 
federal programs with multiple goals.
    OMB recognized the complexity inherent in evaluating federal 
programs by differentiating its rating tool for seven mechanisms or 
approaches used to deliver services, ranging from block grants to 
research and development. However, judgment is involved in classifying 
programs by these categories since many programs fit into more than one 
of these groupings. OMB guidance, for instance, acknowledges that some 
research and development programs can also be evaluated as competitive 
grants and capital assets.
    Performance measurement challenges in evaluating complex federal 
programs make it difficult to meaningfully interpret a bottom-line 
rating. OMB published both a single, bottom-line rating for PART 
results and individual section scores. It is these latter scores that 
are potentially more useful for identifying information gaps and 
program weaknesses. For example, in the fiscal year 2004 PART, one 
program that was rated ``adequate'' overall got high scores for purpose 
(80 percent) and planning (100 percent), but poor scores in being able 
to show results (39 percent) and in program management (46 percent). In 
a case like this, the individual section ratings provided a better 
understanding of areas needing improvement than the overall rating 
alone. In addition, bottom-line ratings may force raters to choose 
among several important but disparate goals and encourage a 
determination of program effectiveness even when performance data are 
unavailable, the quality of those data is uneven, or they convey a 
mixed message on performance.
    Any tool that is sophisticated enough to take into account the 
complexity of the U.S. government will always require some 
interpretation and judgment. Therefore it is not surprising that OMB 
staff were not fully consistent in interpreting complex questions about 
agency goals and results. Many PART questions contain subjective terms 
that are open to interpretation. Examples include terminology such as 
``ambitious'' in describing sought-after performance measures. Because 
the appropriateness of a performance measure depends on the program's 
purpose, and because program purposes can vary immensely, an ambitious 
goal for one program might be unrealistic for a similar but more 
narrowly defined program. Without further guidance, it is unclear how 
OMB staff can be expected to be consistent.
    We also found inconsistencies in how the definition of acceptable 
performance measures was applied. Our review of the fiscal year 2004 
PART surfaced several instances in which OMB staff inconsistently 
defined appropriate measures--outcome versus output--for programs. 
Agency officials also told us that OMB staff used different standards 
to define measures as outcome-oriented. Outputs are the products and 
services delivered by the program whereas outcomes refer to the results 
of outputs. For example, in the employment and training area, OMB 
accepted short-term outcomes, such as obtaining high school diplomas or 
employment, as a proxy for long-term goals for the Department of Health 
and Human Services' Refugee Assistance program, which aims to help 
refugees attain economic self-sufficiency as soon as possible. However, 
OMB did not accept the same employment measure as a proxy for long-term 
goals for the Department of Education's Vocational Rehabilitation 
program because it had not set long-term targets beyond a couple of 
years. In other words, although neither program contained long-term 
outcomes, such as participants gaining economic self-sufficiency, OMB 
accepted short-term outcomes in one instance but not the other.
    The yes/no format employed throughout most of the PART 
questionnaire resulted in oversimplified answers to some questions. 
Although OMB believes it helped standardization, the yes/no format was 
particularly troublesome for questions containing multiple criteria for 
a ``yes'' answer. Agency officials have commented that the yes/no 
format can oversimplify reality, in which progress in planning, 
management, or results is more likely to resemble a continuum than an 
on/off switch. Our review of the fiscal year 2004 PART found several 
instances in which some OMB staff gave a ``yes'' answer for 
successfully achieving some but not all of the multiple criteria, while 
others gave a ``no'' answer when presented with a similar situation. 
For example, OMB judged the Department of the Interior's (DOI) Water 
Reuse and Recycling program ``no'' on whether a program has a limited 
number of ambitious, long-term performance goals, noting that although 
DOI set a long-term goal of 500,000 acre-feet per year of reclaimed 
water, it failed to establish a time frame for when it would reach the 
target. However, OMB judged the Department of Agriculture's and DOI's 
Wildland Fire programs ``yes'' on this question even though the 
programs' long-term goals of improved conditions in high-priority 
forest acres are not accompanied by specific time frames.
    The lack of program performance information also creates challenges 
in effectively assessing program performance. According to OMB, about 
half of the programs assessed for fiscal year 2004 lacked ``specific, 
ambitious long-term performance goals that focus on outcomes'' and 
nearly 40 percent lacked sufficient ``independent, quality 
evaluations.'' Nearly 50 percent of programs assessed for fiscal year 
2004 received ratings of ``results not demonstrated'' because OMB 
decided that program performance information, performance goals, or 
both were insufficient or inadequate. While the validity of these 
assessments may be subject to interpretation and debate, our previous 
work\9\ has raised concerns about the capacity of federal agencies to 
produce evaluations of program effectiveness as well as credible data.
---------------------------------------------------------------------------
    \9\ U.S. General Accounting Office, Program Evaluation: Agencies 
Challenged by New Demand for Information on Program Results, GAO/GGD-
98-53 (Washington, D.C.: Apr. 24, 1998).
---------------------------------------------------------------------------
    In our report on PART, we note that several factors have limited 
the availability of performance data and evaluations of federal 
programs, including the lack of statutory mandates and funding to 
support data collection and analysis. Our work has recognized that 
research programs pose particular and long-standing challenges for 
performance assessments and evaluations.\10\ For instance, in both 
applied and basic research, projects take several years to complete and 
require more time before their meaning for the field can be adequately 
understood and captured in performance reporting systems. These 
challenges can and have been addressed by federal and private research 
organizations. One evaluation approach we have identified in our review 
of leading practices is the use of peer review to evaluate the quality 
of research outcomes.\11\ For example, the National Science Foundation 
(NSF) convenes panels of independent experts as external advisers--a 
Committee of Visitors (COV)--to peer review the technical and 
managerial stewardship of a specific program or cluster of programs 
periodically. The COV compares research plans with progress made, and 
evaluates outcomes to determine whether the research contributes to NSF 
mission and goals.
---------------------------------------------------------------------------
    \10\ U.S. General Accounting Office, Transportation Research: 
Actions Needed to Improve Coordination and Evaluation of Research, GAO-
03-500 (Washington, D.C.: May 1, 2003).
    \11\ U.S. General Accounting Office, Program Evaluation: An 
Evaluation Culture and Collaborative Partnerships Help Build Agency 
Capacity, GAO-03-454 (Washington, D.C.: May 2, 2003).
---------------------------------------------------------------------------

The Relationship between GPRA and PART

    PART was designed for and is used in the executive branch budget 
preparation and review process. As a result, the goals and measures 
used in PART must meet OMB's needs. By comparison, GPRA--the current 
statutory framework for strategic planning and reporting--is a broader 
process involving the development of strategic and performance goals 
and objectives to be reported in strategic and annual plans and 
reports. OMB said that GPRA plans were organized at too high a level to 
be meaningful for program-level budget analysis and management review. 
OMB acknowledges that GPRA was the starting point for PART, but as I 
will explain, it appears that OMB's emphasis is shifting such that over 
time the performance measures developed for PART and used in the budget 
process may also come to drive agencies' strategic planning processes.
    The fiscal year 2004 PART process came to be a parallel competing 
structure to the GPRA framework as a result of OMB's desire to collect 
performance data that better align with budget decision units. OMB's 
most recent Circular A-11 guidance clearly requires both that each 
agency submit a performance budget for fiscal year 2005 and that this 
should replace the annual GPRA performance plan.\12\ These performance 
budgets are to include information from the PART assessments, where 
available, including all performance goals used in the assessment of 
program performance done under the PART process. Until all programs 
have been assessed using PART, the performance budget will also include 
performance goals for agency programs that have not yet been assessed. 
OMB's movement from GPRA to PART is further evident in the fiscal year 
2005 PART guidance stating that while existing GPRA performance goals 
may be a starting point during the development of PART performance 
goals, the GPRA goals in agency GPRA documents are to be revised, as 
needed, to reflect OMB's instructions for developing the PART 
performance goals. Lastly, this same guidance states that GPRA plans 
should be revised to include any new performance measures used in PART 
and that unnecessary measures should be deleted from GPRA plans. In its 
comments to another recently issued GAO report, OMB stated that it will 
revise its guidance for both GPRA and PART to clarify the integrated 
and complementary relationship between the two initiatives.\13\
---------------------------------------------------------------------------
    \12\ OMB Circular A-11, Preparation, Submission, and Execution of 
the Budget.
    \13\ U.S. General Accounting Office, Results-Oriented Government: 
GPRA Has Established a Solid Foundation for Achieving Greater Results, 
GAO-04-38 (Washington, D.C.: March 10, 2004).
---------------------------------------------------------------------------
    Although there is potential for complementary approaches to GPRA 
and PART, the following examples clearly illustrate the importance of 
carefully considering the implications of selecting a unit of analysis, 
including its impact on the availability of performance data. They also 
reveal some of the unresolved tensions between the President's budget 
and performance initiative--a detailed budget perspective--and GPRA--a 
more strategic planning view. Experience with PART highlighted the fact 
that defining a ``unit of analysis'' useful for both program-level 
budget analysis and agency planning purposes can be difficult. For 
example, disaggregating programs for PART purposes could ignore the 
interdependence of programs recognized by GPRA by artificially 
isolating programs from the larger contexts in which they operate. 
Agency officials described one program assessed with the fiscal year 
2004 PART--Projects for Assistance in Transition from Homelessness--
that was aimed at a specific aspect of homelessness, that is, referring 
persons with emergency needs to other agencies for housing and needed 
services. OMB staff wanted the agency to produce long-term outcome 
measures for this program to support the PART review process. Agency 
officials argued that chronically homeless people require many services 
and that this federal program often supports only some of the services 
needed at the initial stages of intervention. GPRA--with its focus on 
assessing the relative contributions of related programs to broader 
goals--is better designed to consider crosscutting strategies to 
achieve common goals. Federal programs cannot be assessed in isolation. 
Performance also needs to be examined from an integrated, strategic 
perspective.
    One way of improving the links between PART and GPRA would be to 
develop a more strategic approach to selecting and prioritizing areas 
for assessment under the PART process. Targeting PART assessments based 
on such factors as the relative priorities, costs, and risks associated 
with related clusters of programs and activities addressing common 
strategic and performance goals not only could help ration scarce 
analytic resources but also could focus decision-makers' attention on 
the most pressing policy and program issues. Moreover, such an approach 
could facilitate the use of PART assessments to review the relative 
contributions of similar programs to common or crosscutting goals and 
outcomes established through the GPRA process.

The Importance of Congressional and Other Stakeholder Involvement

    We have previously reported\14\ that stakeholder involvement 
appears critical for getting consensus on goals and measures. In fact, 
GPRA requires agencies to consult with Congress and solicit the views 
of other stakeholders as they develop their strategic plans.\15\ 
Stakeholder involvement can be particularly important for federal 
agencies because they operate in a complex political environment in 
which legislative mandates are often broadly stated and some 
stakeholders may strongly disagree about the agency's mission and 
goals.
---------------------------------------------------------------------------
    \14\ U.S. General Accounting Office, Agencies' Strategic Plans 
Under GPRA: Key Questions to Facilitate Congressional Review (Version 
1), GAO/GGD-10.1.16 (Washington, D.C.: May 1997).
    \15\ 5 U.S.C. 306(d) (2003).
---------------------------------------------------------------------------
    The relationship between PART and its process and the broader GPRA 
strategic planning process is still evolving. As part of the executive 
branch budget formulation process, PART must clearly serve the 
President's interests. Some tension about the amount of stakeholder 
involvement in the internal deliberations surrounding the development 
of PART measures and the broader consultations more common to the GPRA 
strategic planning process is inevitable. Compared to the relatively 
open-ended GPRA process, any budget formulation process is likely to 
seem closed.
    Yet, we must ask whether the broad range of congressional officials 
with a stake in how programs perform will use PART assessments unless 
they believe the reviews reflect a consensus about performance goals 
among a community of interests, target performance issues that are 
important to them as well as the administration, and are based on an 
evaluation process in which they have confidence. Similarly, the 
measures used to demonstrate progress toward a goal, no matter how 
worthwhile, cannot serve the interests of a single stakeholder or 
purpose without potentially discouraging use of this information by 
others.
    Congress has a number of opportunities to provide its perspective 
on performance issues and performance goals, such as when it 
establishes or reauthorizes a new program, during the annual 
appropriations process, and in its oversight of federal operations. In 
fact, these processes already reflect GPRA's influence. Reviews of 
language in public laws and committee reports show an increasing number 
of references to GPRA-related provisions. What is missing is a 
mechanism to systematically coordinate a congressional perspective and 
promote a dialogue between Congress and the President in the PART 
review process.
    In our report, we have suggested steps for both OMB and the 
Congress to take to strengthen the dialogue between executive officials 
and congressional stakeholders. We have recommended that OMB reach out 
to key congressional committees early in the PART selection process to 
gain insight about which program areas and performance issues 
congressional officials believe warrant PART review. Engaging Congress 
early in the process may help target reviews with an eye toward those 
areas most likely to be on the agenda of Congress, thereby better 
ensuring the use of performance assessments in resource allocation 
processes throughout government. We have also suggested that Congress 
consider the need to develop a more systematic vehicle for 
communicating its top performance concerns and priorities; develop a 
more structured oversight agenda to prompt a more coordinated 
congressional perspective on crosscutting performance issues; and use 
this agenda to inform its authorization, appropriations, and oversight 
processes.

Concluding Observations

    The PART process is the latest initiative in a long-standing series 
of reforms undertaken to improve the link between performance 
information and budget decisions. Although each of the initiatives of 
the past appears to have met with an early demise, in fact, subsequent 
reforms were strengthened by building on the legacy left by their 
predecessors. Prior reforms often failed because they were not relevant 
to resource allocation and other decision-making processes, thereby 
eroding the incentives for federal agencies to improve their planning, 
data, and evaluations.
    Unlike many of those past initiatives, GPRA has been sustained 
since its passage 10 years ago, and evidence exists that it has become 
more relevant than its predecessors. PART offers the potential to build 
on the infrastructure of performance plans and information ushered in 
by GPRA and the law's intent to promote the use of these plans in 
resource allocation decision-making. GPRA improved the supply of plans 
and information, while PART can prompt greater demand for this 
information by decision-makers. Enhancing interest and use may bring 
about greater incentives for agencies to devote scarce resources to 
improving their information and evaluations of federal programs as 
well.
    Increasing the use and usefulness of performance data is not only 
important to sustain performance management reforms, but to improve the 
processes of decision-making and governance. Many in the United States 
believe there is a need to establish a comprehensive portfolio of key 
national performance indicators. This will raise complex issues ranging 
from agreement on performance areas and indicators to getting and 
sharing reliable information for public planning, decision-making, and 
accountability. In this regard, the entire agenda of management reform 
at the federal level has been focused on shifting the attention of 
decision-makers and agency management from process to results. Although 
PART is based on changing the orientation of budgeting, other 
initiatives championed by Congress and embodied in the PMA are also 
devoted to improving the accountability for performance goals in agency 
human capital management, financial management, competitive sourcing, 
and other key management areas.
    In particular, we have reported that human capital--or people--is 
at the center of any serious change management initiative. Thus, 
strategic human capital management is at the heart of government 
transformation. High-performing organizations strengthen the alignment 
of their GPRA strategic and performance goals with their daily 
operations. In that regard, performance management systems can be a 
vital tool for aligning an organization's operations with individual 
day-to-day activities, but they are currently largely unused. As we 
move forward to strengthen government performance and accountability, 
effective performance management systems can be a strategic tool to 
drive internal change and achieve desired results.
    The question now is how to enhance the credibility and use of the 
PART process as a tool to focus decisions on performance. In our 
report, we make seven recommendations to OMB and a suggestion to 
Congress to better support the kind of collaborative approach to 
performance budgeting that very well may be essential in a separation 
of powers system like ours. Our suggestions cover several key issues 
that need to be addressed to strengthen and help sustain the PART 
process. We recommend that the OMB Director take the following actions:

          Centrally monitor agency implementation and progress 
        on PART recommendations and report such progress in OMB's 
        budget submission to Congress. Government-wide councils may be 
        effective vehicles for assisting OMB in these efforts.

          Continue to improve the PART guidance by (1) 
        expanding the discussion of how the unit of analysis is to be 
        determined to include trade-offs made when defining a unit of 
        analysis, implications of how the unit of analysis is defined, 
        or both; (2) clarifying when output versus outcome measures are 
        acceptable; and (3) better defining an ``independent, quality 
        evaluation.''

          Clarify OMB's expectations to agencies regarding the 
        allocation of scarce evaluation resources among programs, the 
        timing of such evaluations, as well as the evaluation 
        strategies it wants for PART, and consider using internal 
        agency evaluations as evidence on a case-by-case basis--whether 
        conducted by agencies, contractors, or other parties.

          Reconsider plans for 100 percent coverage of federal 
        programs and, instead, target for review a significant 
        percentage of major and meaningful government programs based on 
        such factors as the relative priorities, costs, and risks 
        associated with related clusters of programs and activities.

          Maximize the opportunity to review similar programs 
        or activities in the same year to facilitate comparisons and 
        trade-offs.

          Attempt to generate, early in the PART process, an 
        ongoing, meaningful dialogue with congressional appropriations, 
        authorization, and oversight committees about what they 
        consider to be the most important performance issues and 
        program areas that warrant review.

          Seek to achieve the greatest benefit from both GPRA 
        and PART by articulating and implementing an integrated, 
        complementary relationship between the two.

    In its comments on our report, OMB outlined actions it is taking to 
address several of these recommendations, including refining the 
process for monitoring agencies' progress in implementing the PART 
recommendations, seeking opportunities for dialogue with Congress on 
agencies' performance, and continuing to improve executive branch 
implementation of GPRA plans and reports.
    Our recommendations to OMB are partly directed at fortifying and 
enhancing the credibility of PART itself and the underlying data used 
to make the judgments. Decision makers across government are more 
likely to rely on PART data and assessments if the underlying 
information and the rating process are perceived as being credible, 
systematic, and consistent. Enhanced OMB guidance and improved 
strategies for obtaining and evaluating program performance data are 
vital elements.
    The PART process can be made more sustainable if the use of 
analytic resources at OMB and the agencies is rationalized by 
reconsidering the goal of 100 percent coverage of all federal programs. 
Instead, we suggest a more strategic approach to target assessments on 
related clusters of programs and activities. A more targeted approach 
stands a better chance of capturing the interest of decision-makers 
throughout the process by focusing their attention on the most pressing 
policy and program issues and on how related programs and tools affect 
broader crosscutting outcomes and goals. Unfortunately, the government-
wide performance plan required by GPRA has never been engaged to drive 
budgeting in this way.
    Improving the integration of inherently separate but interrelated 
strategic planning and performance budgeting processes can help support 
a more strategic focus for PART assessments. GPRA's strategic planning 
goals could be used to anchor the selection and review of programs by 
providing a foundation to assess the relative contribution of related 
programs and tools to broader performance goals and outcomes.
    Finally, refining the PART questionnaire and review process and 
improving the quality of data are important, but the question of whose 
interests drive the process is perhaps paramount in our system. 
Ultimately, the impact of PART on decision-making will be a function 
not only of the President's decisions, but of congressional decisions 
as well.
    Much is at stake in the development of a collaborative performance 
budgeting process. Not only might the PART reviews ultimately come to 
be disregarded absent congressional involvement, but more important, 
Congress will lose an opportunity to use the PART process to improve 
its own decision-making and oversight processes.
    This is an opportune time for the executive branch and Congress to 
carefully consider how agencies and committees can best take advantage 
of and leverage the new information and perspectives coming from the 
reform agenda under way in the executive branch. Ultimately, the 
specific approach or process is not important. We face a long-term 
fiscal imbalance, which will require us to re-examine our existing 
policies and programs. It is all too easy to accept ``the base'' as 
given and to subject only new proposals to scrutiny and analysis. The 
norm should be to reconsider the relevance or ``fit'' of any federal 
program, policy, or activity in today's world and for the future.
    Mr. Chairman, this concludes my prepared statement. I would be 
pleased to answer any questions you or the other Members of the 
Subcommittee may have at this time.




    Chairman Ehlers. Dr. Matanoski.

STATEMENT OF DR. GENEVIEVE MATANOSKI, PROFESSOR, DEPARTMENT OF 
  EPIDEMIOLOGY, JOHNS HOPKINS UNIVERSITY; CHAIR, EPA SCIENCE 
ADVISORY BOARD REVIEW OF EPA'S FISCAL YEAR 2005 BUDGET; FORMER 
               CHAIR, EPA SCIENCE ADVISORY BOARD

    Dr. Matanoski. Good morning, Mr. Chairman and Members of 
the Subcommittee on Environment, Technology and Standards.
    My name is Genevieve Matanoski. I am honored to appear 
before you today to present the views of the Science Advisory 
Board for EPA regarding their Fiscal Year 2005 Science and 
Research Budget Request. The Board will complete its full 
report by March 19, 2004, and with the permission of the 
Chairman, we will submit that report for the record then.
    Over the last year, the Board, under the leadership of its 
Chairman, Dr. William Glaze, has reorganized to make its 
advisory function as apparent as its peer-review function. With 
this reorganization, the Board moved the EPA Science and 
Research Budget Review from a small standing committee to the 
entire larger Board, and this has provided great visibility to 
the review, and it also has increased the expertise available 
to conduct the evaluation. This Board, in the past years, has 
noted its deep concern over the constant erosion of EPA's 
research budget programs. The fiscal year 2005 budget we now 
observe not only continued the erosion but actually cut funding 
for research. This continuous decrease in research funding 
severely constrains the ability of the agency to provide the 
necessary science to guide informed decision-making, and to 
assist in reaching the Nation's goals for human health and 
environmental risk reduction.
    In particular, the Board notes, as did the Chairman, 
substantial cuts in their fiscal year 2005 budget request for 
EPA's Science to Achieve Results program, STAR. STAR is 
recognized by this Board as the science program of major 
importance to the agency. That view is consistent with the 
views of the National Academy of Science in its review of STAR. 
STAR provides many benefits to EPA, one of which is the 
necessary flexibility to obtain critical scientific expertise 
in a wide range of disciplines that are essential in addressing 
emerging issues that are outside EPA's current areas of 
expertise. EPA could never maintain the same large base of 
scientific expertise that is available on an as-needed basis to 
carry out specific research. STAR also enhances EPA's 
collaborations with outside researchers and academic 
institutions, and in the process actually stimulates additional 
resources for the agency and its science needs.
    STAR also benefits and strengthens scientific research 
throughout the United States by providing training for graduate 
students who will reinforce the declining base of engineers and 
scientists in the United States. Even though STAR is largely 
focused on EPA's core research, it has begun to accrue a record 
of early success. Serious budget cuts in STAR are as follows. 
Ecosystems protection research got a $22 million loss, with a 
loss of some 50 STAR grants based in many states. That is a 
serious reduction. Endocrine disruptors' research was cut by 
five million. Pollution prevention research, which is--a focus 
of that research is to avoid future problems. That was cut by 
five million. And mercury research was cut by another two 
million. Even the STAR Graduate Fellowship Program, despite its 
increases, is still $4 million short of what Congress 
recommended for fiscal year 2003.
    The Board believes that these cuts will have a negative 
impact on the balance research portfolio that EPA has developed 
over the last decade. In that time, EPA has developed a program 
that balances its problem-driving, short-term applied research, 
whatever you want to call it, with its core long-term basic 
research. Here, the STAR program helps EPA balance its internal 
and its extramural research portfolios. The result is that 
science from many different institutions is integrated into a 
total research program that compliments the scientific niche 
filled by EPA's own excellent scientists. This provides a more 
nimble resource that is available to work on existing and 
emerging environmental issues than would be available with only 
an intramural program.
    Another surprising change as already noted in the budget 
was EPA's decrease in its commitment to Homeland Security. The 
Board believes that EPA must play a continuing role in Homeland 
Security in several areas, and one of these is building 
contamination, which was cut.
    I have run out of time. There are several other issues that 
we have in our written testimony, one regarding PART and the 
other regarding the future of research at EPA.
    [The prepared statement of Dr. Matanoski follows:]
               Prepared Statement of Genevieve Matanoski
    Good morning Mr. Chairman and Members of the Subcommittee on 
Environment, Technology, and Standards. My name is Genevieve Matanoski. 
I am honored to appear before you today to present the conclusions of 
the EPA Science Advisory Board (SAB) on the EPA's FY 2005 Science and 
Research Budget Request. My comments today will summarize the views of 
the Board about that request for funding EPA's science and research 
programs during FY 2005. The Board will complete its full report on 
this issue by March 19, 2004 and with the permission of the Chairman, 
will submit that report for the record then.

1. Introductory Remarks

    Over the last year the Board, under the leadership of its Chairman, 
Dr. William Glaze, has reorganized to make its advisory function as 
apparent as its peer review function. To take advantage of the benefits 
of this reorganization, the Board moved the EPA Science and Research 
advisory function from a small standing committee to the larger Board. 
This enhances the visibility of the review and increases the expertise 
available to conduct the evaluation. We will also be working with EPA 
to obtain information on the Agency's science and research programs on 
a systematic and continuous basis. The Agency now presents information 
to the Board in ways that correlate with their new Strategic Plan and 
which will ultimately include all science and research programs 
regardless of their funding source or where they are conducted within 
EPA.

2. Specific Comments on EPA's FY 2005 Science and Research Program and 
                    Budgets

a. Erosion of the EPA Research Budget
    First, I want to mention that in past years, the Board has noted 
its deep concern with the constant erosion of EPA's research program 
budgets. For the FY 2005 budget, we now observe not just continued 
erosion, but a substantial cut to research funding. This continuous 
decrease in science and research funding severely constrains the 
ability of the Agency to provide the necessary science to guide 
informed decision-making and to assist in reaching the Nation's goals 
for human health and environmental risk reduction. The EPA has an 
outstanding group of scientists who conduct excellent basic and applied 
research focused on reducing both recognized problems such as air 
pollution, and the continually emerging problems arising from the past 
misuse of our environment.
b. Cuts to the STAR Program
    In particular, the Board notes the substantial cuts in the FY 2005 
budget request for EPA's Science to Achieve Results program (STAR). 
STAR is recognized by this Board as a science program of major 
importance to the Agency. That view is consistent with the views of the 
National Academy of Science in its 2003 review entitled ``The Measure 
of STAR.'' STAR provides many benefits to EPA, including the necessary 
flexibility to obtain critical scientific expertise in the wide range 
of disciplines that are essential in addressing emerging issues that 
are outside EPA's current areas of expertise. EPA could never maintain 
the same large base of scientific expertise that is available on an as 
needed basis to carry out specific research. STAR enhances EPA's 
collaboration with outside researchers and academic institutions and in 
the process actually stimulates additional resources for Agency science 
needs. STAR also benefits and strengthens scientific research 
throughout the United States by providing training for graduate 
students who will reinforce the declining base of engineers and 
scientists in the U.S. Even though STAR is largely focused on EPA's 
core research, it has begun to accrue a record of early success. 
Evidence of this can be found in the NAS's ``Measure of STAR'' report 
in which it states that STAR research on endocrine disruptors, and 
ecologic assessment have already resulted in ``. . .peer-reviewed 
publications that are of immediate use in understanding causes, 
exposures, and effects of environmental pollution.'' By any measure, 
STAR is an excellent investment.
    To emphasize the seriousness of this situation, we note a number of 
cuts to STAR research which are a part of the FY 2005 budget request:

          i.  Ecosystems Protection Research is reduced by over $22 
        million with a loss of some 50 STAR grants based in many 
        states. The Board is particularly concerned about this cut 
        given the critical need for ecosystems research, which the 
        Board feels is generally under-funded across EPA.

         ii.  Endocrine Disruptors Research is reduced by about $5 
        million. This is an area of research that investigates the 
        effects that could be associated with use of many chemicals 
        used in large quantities in our society.

        iii.  Pollution Prevention Research is reduced by $5 million 
        even though the focus of this research is on avoiding future 
        problems and reducing the expensive cleanup costs that we face 
        today.

         iv.  Mercury Research is reduced by $2 million, just at the 
        time when more information is needed on this ubiquitous 
        contaminant.

    In addition, even though the STAR Graduate Fellowship program 
increases by $1.2 million over the FY 2004 level, it is still nearly $4 
million below the level enacted by Congress for FY 2003 ($9.8 million). 
This program's aim is to educate the future environmental scientists 
that will be needed to replace the currently aging population of such 
scientists. Thus, adequate funding of this program continues to be 
essential.
    The Board believes that these cuts will have a negative impact on 
the balanced research portfolio that EPA, especially ORD, has developed 
over the last decade. In that time, EPA has developed a program that 
balances its problem-driven (shorter-term, applied) research with its 
core (longer-term, basic) research. Though components of the core 
program are not always easy to identify in the budget, EPA appears to 
have a balanced research program, in this regard, with about half being 
``core'' and half being ``problem-driven.'' This seems to be 
appropriate.
    Further, the STAR program helps EPA balance its internal research 
portfolio with its extramural research portfolio. The result is that 
science from many different institutions (government, academia, non-
governmental organizations, and industry) is integrated into a total 
research program that complements the scientific niche filled by EPA's 
own scientists. This provides a more nimble resource that is available 
to work on existing and emerging environmental issues than would be 
available with only an intramural or an extramural program. Changes in 
this budget, especially to STAR, will significantly impair the balance 
of this integrated research program in both core vs. problem driven and 
intramural vs. extramural research dimensions.
c. Building Decontamination and Homeland Security
    Another surprising change for which no satisfying explanation was 
offered was EPA's decrease in its commitment to Homeland Security. The 
Board believes that EPA must play a continuing role in Homeland 
Security in several areas. EPA's building decontamination research is 
one of EPA's contributions to Homeland Security and it is being 
eliminated in an $8.3 million dollar cut prior to its completion. The 
Board is unsure as to why research on this important issue is being cut 
when it appears that further research is needed to provide rapid, on-
target responses to contamination episodes. The Board is aware that 
other agencies have substantial resources devoted to Homeland Security, 
so perhaps other groups have taken on this role for the future. EPA has 
the special expertise to carry out this research. However, if it is 
judged that this is not a research direction for EPA, it is still 
important to ensure that this work be conducted somewhere.

2. Program Planning and Measurement

a. Program Assessment
    Each year, the Board tries to evaluate EPA's research priorities 
and their role in meeting the Agency's goals. As part of the current 
review, the Board was given information resulting from the application 
of a new survey tool, the Program Assessment Rating Tool (PART) that 
was used to evaluate selected EPA programs. The Board is concerned that 
decisions are being made about research program funding on the basis of 
the application of this new tool.
    To be clear, the Board did not receive or review information on the 
rating instrument itself; however, after evaluating PART summaries for 
several research programs, our conclusion is that PART may, at this 
time, have a limited capacity to inform budget decisions on research 
programs. The Board's is concerned with the manner in which the 
weighting formula in PART seems to influence the full analysis and thus 
favor programs with short-run results over those having long-term 
results. There is also concern that an evaluator's subjective 
considerations might be able to bias those weights and the rating 
itself.
    Specifically, it appears that the weighting formula in the PART 
favors programs with near-term benefits at the expense of programs with 
long-term benefits. Since research inevitably involves more long-term 
benefits and fewer short-term benefits, PART ratings serve to bias the 
decision-making process against programs such as STAR ecosystem 
research, global climate change research, and other important subjects. 
The PART seems to be intended as a formula for predictions about likely 
program success. However, the weights that the PART assigns to 
different program characteristics do not seem to have been validated 
systematically against the contribution of each program characteristic 
to any independent objective measure of program success. If the weights 
in the tool are arbitrarily assigned, the PART may have characteristics 
that could lead to biases in evaluation that are related to the 
subjective judgments of its designers. We believe that the tool should 
be reviewed to determine its adequacy for its use in supporting budget 
decisions.
    As the Board observed significant decreases in science and research 
funding, it also noted a substantial resource increase in the State and 
Tribal Assistance Grant account (STAG) for an initiative for 
retrofitting school busses. The Board does not challenge the worthiness 
of this program, rather it notes that it has no information on the 
science supporting this initiative. The Board trusts that the benefits 
of this program have been rigorously reviewed.
    The real issue here is how research programs (and others) are to be 
evaluated and whether a different metric is necessary for basic vs. 
applied research programs. Also, of interest is whether research 
results should be evaluated separately from the outcomes of programs 
they are intended to support? Although the Board did not directly 
evaluate the PART itself, it is of obvious difficulty to conceive of a 
simple quantitative metric that could be applied across the broad areas 
of ecosystem quality, human health effects, endocrine effects, and 
technology development. The question is even more complex when you 
consider that some research is intended to develop limited data in the 
short-run to fill a specific knowledge gap and other research is 
intended to provide an understanding of whole systems in the long-term. 
Research program measurement is even more difficult because the 
knowledge and methods developed by EPA, especially ORD's researchers, 
are not usually directly applied by ORD, rather they are often used by 
others to support decisions on a broad suite of diverse statutory 
mandates. Thus, we believe that evaluations of the performance of 
research programs will need to consider the specific factors of each 
program that the research is intended to support. Further, it is 
unlikely that simple formulas will be able to handle this task well. It 
is more likely that realistic research program performance assessment 
will need to be a combination of quantitative metrics and other 
information and analyses which is then evaluated by groups of experts 
with relevant knowledge.
    I note that the NAS, in its review of STAR, also had concerns with 
quantitative routines used in performance assessments and noted that 
``The Committee judges that expert review by a group of people with 
appropriate expertise is the best method of evaluating broad research 
programs, such as the STAR program.''
b. Multi-Year Plans (MYPs)
    Multi-Year Plans (MYPs) are an important innovation in EPA's 
research planning process. The SAB has reviewed a limited number of 
these plans and the process used in their development and we believe 
that they will become more useful to the Board's evaluation of EPA's 
science and research, and its funding, in the future. MYP's are tools 
that identify knowledge and methodology gaps needed to support EPA's 
mission areas and the body of research that would address those needs. 
Further, they provide a basis for identifying annual performance goals 
and measures for efforts that become a part of EPA's research budget. 
Finally, MYPs are very useful in providing focus on long-term progress 
toward research goals, especially on cross-cutting subjects such as 
pollution prevention where coordination across the Agency is essential. 
The Board supports the continued refinement of Multi-Year plans and is 
available to continue its review of EPA's progress in this regard.

3. EPA Science

    My final comment will address an issue of great concern to the 
Board. Our analyses of EPA's science and research budgets, today and in 
the past, convince us that the Agency is in danger of underestimating 
the pace of large scale changes that are now occurring in our society. 
If so, EPA and the Nation are at risk of repeating the mistakes of the 
past that force us to spend huge sums of public funds to reduce and to 
clean up the pollution brought on by the first industrial revolution.
    The evidence before us suggests that we are now in a new, high 
velocity technological revolution that will yield great economic gains, 
but at the same time, will offer new environmental challenges. 
Nanotechnology and biotechnology, to name only two innovations, are 
proceeding with breathtaking speed, and are compounded by forces such 
as global transfer of pollution and disease, and possible climate 
change. EPA must carefully examine all of its science and research 
programs and ask whether the Agency is conducting research that will 
help us protect human health and the environment while encouraging 
innovation and growth.
    This is not to say that EPA should neglect the ``legacy'' issues of 
the past; rather, the Agency must continue to resolve those problems, 
and at the same time, work with citizen's groups, industry, and 
academia in creative ways to ensure that the Nation avoids a new legacy 
of human health and environmental problems.
    The Board would be pleased to work with EPA to explicitly address 
ways in which EPA science and research might be focused to help EPA 
develop, and use to the fullest, knowledge that will be instrumental in 
avoiding a negative legacy.
    I want to express my gratitude to the Members of the Subcommittee 
for inviting me to testify about EPA's science and research. I would be 
pleased to answer your questions.
    Thank you.

                   Biography for Genevieve Matanoski
    Dr. Matanoski is a Professor of Epidemiology at the Johns Hopkins 
University School of Hygiene and Public Health in Baltimore, MD. For a 
time after medical school she pursued a career in pediatrics and 
general preventive medicine. After earning a Doctor of Public Health 
Degree, she was appointed to the faculty of Johns Hopkins University 
and has been a professor since 1976. In addition to teaching and 
research, Dr. Matanoski has had appointments in a number of teaching 
and training programs in the U.S. and abroad and is a frequent advisor 
to legislative and policy-making groups. She is a member of several 
scientific advisory bodies both for governmental agencies and for 
industry. She is a past Chair of the EPA Science Advisory Board, as 
well as a past Chair of the SAB Radiation Advisory Committee. She now 
serves as a member of the Board of the EPA SAB. During her tenure on 
the EPA SAB, Dr. Matanoski has been involved in the writing of several 
documents produced to provide advice to EPA including the ``Beyond the 
Horizon: Using Foresight to Protect the Environmental Future'' document 
and the Integrated Risk Project report ``Toward Integrated 
Environmental Decision-making,'' and was Chair of the latter Committee. 
She is the author or co-author of over 80 publications.
    Dr. Matanoski's work has focused on the epidemiology of cancer, 
including bladder, lung, skin and uterine cancers, and leukemia. Her 
research studies have examined the risks associated with occupational 
and environmental exposures to such agents as radiation, 
electromagnetic fields, and chemical substances as styrene, butadiene, 
arsenic and environmental tobacco smoke. Her recent research has 
emphasized reproductive effects and congenital malformations from 
environmental exposures. Her early work involved infectious diseases 
and illnesses in infants and children. Dr. Matanoski received a B.A. 
degree in chemistry at Radcliffe College and a M.D. at the Johns 
Hopkins School of Medicine. She also earned a Doctor of Public Health 
Degree from the Johns Hopkins University School of Hygiene and Public 
Health (11/2003).

    Chairman Ehlers. Thank you for your testimony.
    Dr. Denson.

  STATEMENT OF DR. COSTEL D. DENSON, PROFESSOR, DEPARTMENT OF 
  ENGINEERING, UNIVERSITY OF DELAWARE; MEMBER OF THE NATIONAL 
 RESEARCH COUNCIL PANEL REVIEW, THE MEASURE OF STAR: REVIEW OF 
  THE EPA'S SCIENCE TO ACHIEVE RESULTS (STAR) RESEARCH GRANTS 
                            PROGRAM

    Dr. Denson. Good morning, Mr. Chairman and Members of the 
Committee.
    My name is Costel Denson. I am a Professor of Chemical 
Engineering at the University of Delaware, where I have been 
employed since 1977. I have also served as Vice Provost for 
Research. In that position, I had oversight of and was 
responsible for all aspects of the research enterprise at the 
University. Recently, I served as a member of the National 
Research Council, the NRC Committee, to review EPA's Research 
Grants Program. The NRC is the operating arm of the National 
Academies of Sciences and Engineering. I am pleased to be here 
to discuss the unanimous findings and recommendations of that 
committee.
    The Environmental Protection Agency is a mission agency 
established to protect human health and to safeguard the 
natural environment. EPA's regulatory and decision-making role 
requires that the agency have access to the best available 
science that is relevant to its mission. In an effort to 
improve the scientific foundation of its decision-making 
process, the agency's Office of Research and Development 
established the Science to Achieve Results program, the STAR 
program, as part of its Research Grants Program in 1995.
    STAR is a competitive, peer-reviewed, extramural research 
grants program created to encourage interagency collaboration 
and to increase EPA's access to the Nation's best scientists 
and engineers in academic and other non-profit research 
institutions. It supports research pertaining to human health 
and the environment and it is designed to maximize the 
independence of the researchers it supports and to provide an 
equal opportunity for all researchers to qualify for that 
support.
    Over the past six years, a number of occasions have arisen 
where I have had the opportunity to review the STAR program. 
When I was chair of EPA's Office of Research and Development's 
Board of Scientific Counselors, I oversaw the review that BOSC 
conducted in 1997 of EPA's National Center for Environmental 
Research and Quality Assurance, and the STAR program, which was 
operated by that center. Again, in 1999/2000, while still chair 
of BOSC, I oversaw a review of the STAR program that BOSC and 
EPA's Science Advisory Board conducted jointly. Now as I 
mentioned earlier, I served on the NRC Committee to Review 
EPA's Research Grants Program. The findings and conclusions 
from this most recent report, along with those from the 
previous BOSC reports, lead me to conclude the following.
    First, environmental regulatory decisions must be informed 
by the best science. The STAR program is judged to be the best 
mechanism that we have for providing the very best science 
through extramural sources. Second, research in STAR is focused 
on EPA's and the country's greatest environmental needs. And 
finally, the STAR program has an exceptional process for the 
peer-review of proposals. The NRC Committee stated in its 
conclusions that the STAR program ``compares favorably with and 
in some cases exceeds that in place at other agencies that have 
extramural research programs, such as the National Science 
Foundation and the National Institute of Environmental Sciences 
Health,'' which is part of NIH.
    The Committee, in its deliberations, developed its own 
metrics, and in that case, developed a series of nine 
questions, and for each question, there was a finding and a 
recommendation. Those are delineated in my written testimony, 
and also as a part of this report, which is called The Measure 
of STAR: Review of the U.S. Environmental Protection Agency's 
Science to Achieve Results Research Grants Program.
    Mr. Chairman, that concludes my testimony. I would be 
pleased to answer any questions. Thank you and the Members of 
your Committee for the opportunity to participate in this 
hearing.
    [The prepared statement of Dr. Denson follows:]
                 Prepared Statement of Costel D. Denson
    Good morning, Mr. Chairman and Members of the Committee. My name is 
Costel Denson. I am a Professor of Chemical Engineering at the 
University of Delaware where I have been employed since 1977. I have 
also served there as Vice Provost for Research. In that position I had 
oversight of, and was responsible for, all aspects of the research 
enterprise at the university. Recently, I served as a member of the 
National Research Council (NRC) Committee to Review EPA's Research 
Grants Program. The NRC is the operating arm of the National Academies 
of Sciences and Engineering. I am pleased to be here to discuss the 
unanimous findings and recommendations of that committee.
    The Environmental Protection Agency (EPA) is a mission agency 
established to protect human health and to safeguard the natural 
environment. EPA's regulatory and decision-making role requires that 
the agency have access to the best available science that is relevant 
to its mission. In an effort to improve the scientific foundation of 
its decision-making process, the agency's Office of Research and 
Development established the Science to Achieve Results (STAR) research 
grants program in 1995.
    STAR is a competitive, peer-reviewed, extramural research grants 
program created to encourage interagency collaboration and to increase 
EPA's access to the Nation's best scientists and engineers in academic 
and other nonprofit research institutions. It supports research 
pertaining to human health and the environment and is designed to 
maximize the independence of the researchers it supports and to provide 
an equal opportunity for all researchers to qualify for support.
    Over the past six years a number of occasions have arisen where I 
have had the opportunity to review the STAR Program. When I was chair 
of EPA's Office of Research and Development's Board of Scientific 
Counselors (BOSC), I oversaw the review that BOSC conducted in 1997 of 
EPA's National Center for Environmental Research and Quality Assurance, 
and the STAR Program, which was operated by that center. Again, in 
1999/2000, while still chair of BOSC, I oversaw a review of the STAR 
program that BOSC and EPA's Science Advisory Board conducted jointly. 
And now, as mentioned earlier, I served on the NRC Committee to Review 
EPA's Research Grants Program.
    The findings and conclusions from this most recent report, along 
with those from the previous BOSC reports, lead me to conclude the 
following:

        1.  Environmental regulatory decisions must be informed by the 
        best science: the STAR program is judged to be the best 
        mechanism that we have for providing the very best science 
        through extramural sources.

        2.  Research in STAR is focused on EPA's and the country's 
        greatest environmental needs.

        3.  The STAR Program has an exceptional process for the peer-
        review of proposals. The NRC committee stated in its 
        conclusions that the STAR program ``compares favorably with and 
        in some cases exceeds that in place at other agencies that have 
        extramural research programs, such as NSF and NIEHS (NIH).''

    In 2000, EPA asked the NRC to conduct an independent assessment of 
the STAR program. In response, the NRC established a committee and gave 
it the following tasks:

         Assess the program's scientific merit, its demonstrated or 
        potential impact on the agency's policies and decisions, and 
        other program benefits that are relevant to EPA's mission;

         Recommend ways to enhance the program's scientific merit, 
        impact, and other benefits; and

         In the context of other relevant research conducted or funded 
        by EPA, and in comparison with other basic and applied research 
        grants programs, address the STAR program's research 
        priorities, research solicitations, peer-review process, 
        ongoing research projects, and results and dissemination of 
        completed research.

    In 2003, the committee completed its report, The Measure of STAR, 
Review of the U.S. Environmental Protection Agency's Science to Achieve 
Results (STAR) Research Grants Program.
    In its evaluation, the committee developed a series of nine 
specific questions that it considered were of greatest importance to 
the research program. I will present each of these questions, along 
with the committee's overall findings and recommendations for the STAR 
program.

1.  Should the STAR program continue to be part of EPA's research 
                    program?

Finding. EPA requires a strong and balanced science and technology 
research program to fulfill its mission properly. The STAR program is 
an important part of the overall EPA research program.
    The STAR program is EPA's pre-eminent program that solicits 
independent scientific and technologic research from the Nation's best 
academic and other nonprofit research institutions. The program has 
established and maintains a high degree of scientific excellence. 
Through broadly advertised, competitively awarded, peer-reviewed 
grants, the STAR program provides the agency access to independent 
information, analyses, and perspectives.
    The STAR program provides the agency access to a broad community of 
researchers, allows it to fund research at the cutting edge of science, 
and assists it in addressing information gaps that it does not have the 
internal resources to address properly. The STAR program also 
encourages its grantees to disseminate their research results widely in 
peer-reviewed scientific journals.

Recommendation. The STAR program should continue to be an important 
part of EPA's research program.

2.  What is the unique contribution of the STAR program?

Finding. The STAR program funds important research that is not 
conducted or funded by other agencies.
    For instance, the STAR ecologic-indicators program is the primary 
source of support of research on the development of water-quality 
indicators for biologic monitoring. The interdisciplinary centers that 
STAR has funded also represent an innovative approach to supporting 
research that will be most relevant for environmental decision-making.
    The STAR program has also made commendable efforts to leverage 
funds by establishing research partnerships with other agencies and 
organizations that have similar or complementary research interests, 
including the National Science Foundation, the Department of Energy, 
the Office of Naval Research, the National Aeronautics and Space 
Administration, the National Oceanic and Atmospheric Administration, 
the U.S. Department of Agriculture, the National Institute of 
Environmental Health Sciences, the Department of Interior, and the 
American Waterworks Research Foundation.

Recommendation. STAR should continue to partner with other government 
and non-government organizations to support research of mutual interest 
and of relevance to EPA's mission, explore innovative approaches for 
carrying out this research, and sponsor a diverse portfolio of research 
that alerts the agency to emerging issues and provides independent 
analyses of issues that the agency is currently addressing.

3.  Does the STAR program have adequate processes to ensure that it is 
                    sponsoring high-quality and relevant research?

Finding. The STAR program has developed a grant-award process that 
compares favorably with and in some ways exceeds that in place at other 
agencies that have extramural research programs, such as the National 
Science Foundation (NSF) and the National Institute of Environmental 
Health Sciences. An unusually high degree of planning goes into 
identifying the specific research subjects to be supported. The agency 
also puts considerable time and thought into preparing effective 
research solicitations and into funding projects that are relevant to 
its mission and program needs. Furthermore, the STAR program has 
established a rigorous peer-review process.

Recommendation. The STAR program should continue to improve the focus 
of its Request for Applications, and when the agency does not have the 
capacity internally to adequately define the state of the science in a 
particular research field, STAR should consider greater use of external 
experts to assist in identifying the highest-priority research and data 
gaps.

4.  Is the STAR program producing high-quality research results?

Finding. Although it is still too early for comprehensive evaluations 
of the research results of the STAR program, some STAR research efforts 
have already substantially improved the scientific foundation for 
decision-making.
    Many STAR projects have resulted in articles in highly respected, 
peer-reviewed journals a traditional measure of research quality. These 
STAR research results have already helped to improve our understanding 
of the causes, exposures, and effects of environmental pollution 
information critical to improving the scientific foundation for 
decision-making. For instance, STAR-funded research on particulate 
matter has helped to improve our understanding of the biologic 
mechanisms by which inhaled ambient particles cause health effects and 
the nature of some of these effects. These data are critical to future 
regulatory decisions regarding our nation's ambient air quality.

Recommendation. EPA should continue its efforts to attract ``the best 
and the brightest'' researchers to compete for STAR funding.

5.  Are the STAR program results useful for EPA decisions and 
                    processes?

Finding. The STAR portfolio effectively supports EPA's mission and 
research and development strategic plans and GPRA goals. Specific STAR 
research projects have yielded significant new findings and knowledge 
critical for regulatory decision-making.
    The STAR program is too young to be able to document fully the 
extent to which its research results are being used to inform 
development of new regulations and environmental-management decisions. 
However, some STAR projects have already yielded information important 
for environmental decision-making. For example, STAR-sponsored research 
in endocrine disruptors, particulate matter, and ecologic assessment 
has resulted in groups of peer-reviewed publications of immediate use 
in understanding causes, exposures, and effects of environmental 
pollution. Those results are directly relevant to EPA's mission to 
``protect human health and to safeguard the natural environment--air, 
water, and land--upon which life depends.'' For instance, research on 
ecologic indicators has led to the development of a dynamic, 
economically linked model to evaluate the driving forces and ecologic 
consequences of land-use change.

Recommendation. The STAR program and EPA's Office of Research and 
Development should develop mechanisms for documenting the extent to 
which its research is being used to support the agency's environmental 
decision-making, should consider using outside experts to help document 
systematically the ``state of the science'' before research is 
initiated, and should synthesize the results of the research when it is 
completed to identify the specific contributions that STAR and other 
EPA research has made to providing critical information.

6.  Is the STAR program effective in providing results relevant to the 
                    appropriate audiences?

Finding. The STAR program has been commendably aggressive in 
experimenting with innovative approaches to communicating the results 
of its funded research to a wide variety of users and audiences, but 
its success in these efforts has been uneven.
    The STAR program supports research of potential value to a variety 
of users and audiences, both in and outside EPA. Much of the research 
is aimed primarily at the scientific community and those responsible 
for providing technical support for environmental-management decisions.
    The program, however, also has other potential users, at least for 
some of its research results, including other federal agencies; 
industry; State, tribal, and local governments; nonprofit environmental 
organizations; and international environmental agencies.
    The STAR program has experimented aggressively with a wide variety 
of communication mechanisms, including EPA's web site and publication 
of reports. Through these efforts, the program appears to substantially 
exceed the dissemination efforts of most other research-sponsoring 
organizations, both in and outside the Federal Government. 
Nevertheless, the STAR program could substantially improve its 
dissemination efforts by directing them more effectively to specific 
users and audiences.

Recommendation. The STAR program should clearly identify the intended 
audiences for proposed research results as early in the process as 
possible and indicate them in the Request for Applications. When 
appropriate, EPA should consider involving representatives of the 
intended audiences from outside the agency to help define the relevant 
research results and a strategy for their dissemination.

7.  Should the fellowship program continue to be part of EPA's research 
                    program?

Finding. The STAR fellowship program is a valuable mechanism for 
enabling a continuing supply of graduate students in environmental 
sciences and engineering to help build a stronger scientific foundation 
for the Nation's environmental research and management efforts.
    It is the only federal fellowship program exclusively designed for 
students pursuing advanced degrees in environmental sciences.

Recommendation. Given the Nation's continuing need for highly qualified 
scientists and engineers in environmental research and management, the 
STAR fellowship program should be continued and funded.

8.  Are the STAR program's funds adequate to achieve its objectives?

Finding. STAR is only able to fund less than 15 percent of the 
proposals received for its individual investigator and center grants, 
and its funding has not kept pace with the rate of inflation.
    NIH and NSF strive to fund, on the average, 25-30 percent of the 
proposals received. STAR's budget allows it to fund only 10-15 percent 
of the proposals it receives and only about 60 percent of those rated 
``excellent'' or ``very good'' by its independent quality peer-review 
panels. By that measure, STAR does not have sufficient funds to 
recognize all the best proposals received.
    Although the STAR program's budget grew rapidly in its first three 
years, it has not kept pace with general inflation in the last few 
years. That is particularly true of the STAR fellowship program. The 
effect of that budgetary situation is exacerbated by the fact that 
costs of research have outpaced general inflation for more than a 
decade. Therefore, at present, STAR funds buy less research than the 
same amount of money could have bought several years ago.
    It is appropriate to consider the funding of the STAR program in 
the context of the overall funding for all of EPA's Office of Research 
and Development, which also has not kept pace with inflation. STAR 
currently represents about 18% of EPA's Office of Research and 
Development total funding. The committee considers that percentage to 
be a reasonable recognition of the value of independent peer-reviewed 
research to the agency.

Recommendation. STAR program funding should be maintained at 15-20 
percent of the overall research and development budget, even in budget-
constrained times. However, budget planners should clearly recognize 
the constraints of not having inflation escalators to maintain the 
level of effort of the entire program.

9.  How should the STAR program be evaluated?

Finding. There are no easy answers when it comes to identifying metrics 
for evaluating research programs, and the best approach for evaluating 
the STAR program is to establish a structured system of reviews by 
panels of experts.
    The committee assessed the quality, relevance, and performance of 
the STAR program, as set forth in recent OMB research and development 
criteria, by using qualitative and quantitative metrics. That is one 
approach for reviewing the STAR program and similar programs. Several 
examples of qualitative and quantitative metrics that were used for 
evaluating the STAR program are: Does the STAR program have a clearly 
defined plan for regular, external reviews of its research quality, and 
has this plan been effectively carried out? Has the program made 
significant contributions to advancing the state of the science in 
particular research topics? Does the program award grants 
expeditiously? Does the program have a schedule for the products it 
intends to produce and how well is it adhering to the schedule?
    The committee's judgment is that quantitative metrics, although 
outwardly simpler to use, are not necessarily more informative than 
qualitative metrics. In some cases, quantitative metrics can be 
misleading, and emphasizing inappropriate metrics can distort the 
research outputs of a program. Qualitative metrics are less likely to 
have such effects, but they need to be interpreted carefully.
    The committee judges that expert review by a group of people with 
appropriate expertise is the best method of evaluating broad research 
programs, such as the STAR program. Expert review is appropriate for 
evaluating both the processes and the products of the STAR program. 
Both qualitative and quantitative metrics can provide valuable support 
for such expert reviews.

Recommendation. STAR and EPA's Office of Research and Development 
should establish a structured program of reviews by panels of 
independent experts and should collect the appropriate information to 
support these reviews.
    Mr. Chairman, that concludes my testimony. I would be pleased to 
answer any questions. Thank you and the Members of your committee for 
the opportunity to participate in this hearing.

                     Biography for Costel D. Denson
    Costel Denson is a professor of chemical engineering at the 
University of Delaware. He received a B.S. from Lehigh University, a 
Master's from Rensselaer Polytechnic Institute and his Ph.D. from the 
University of Utah, all in chemical engineering. His research has 
focused on the rheology and processing of polymeric materials and he is 
a fellow of the Society of Plastics Engineers. Dr. Denson has served as 
Vice Provost for Research at the University of Delaware where he was 
responsible for the administration of all aspects of the research 
enterprise. He has served as chair of EPA's Board of Scientific 
Counselors and is a past chair of the Materials Task Force at the 
Military Engineering Center of Excellence. Dr. Denson has served as a 
member of the National Science Foundation Advisory Committee for 
Environmental Research and Education and on the National Research 
Council's Committee on Air Quality Management in the United States. He 
has also served on the Ford Foundation Minority Predoctoral Review 
Panel on Engineering.

                               Discussion

    Chairman Ehlers. Thank you very much, and thank you to all 
of the witnesses for being here and for your testimony.
    We will now begin with the questions. The Chair will 
recognize himself first for five minutes, and then each Member, 
in turn, will receive a five-minute question period. If there 
are sufficient questions, we will continue with a second round, 
as well.

    Rationale for Science to Achieve Results (STAR) Research Grants 
                          Program Budget Cuts

    The first question is for Mr. Johnson and Dr. Gilman. You 
have--been interesting in my career in the Congress. Because I 
am a scientist, Members are constantly coming to me to ask my 
opinion on scientific issues and questions, and probably the 
most often repeated phrase that I have heard is I want some 
good science on this. Frequently, they will also say I want 
sound science, in which case I say I don't know anything about 
acoustics. But I do know something about good science, and I 
have been actively involved in trying to improve the scientific 
effort of the Nation in every agency, including the EPA, and I 
am very pleased the EPA has made considerable progress.
    It is very disappointing to see the recommendations for the 
big cuts that we are seeing recommended here. The 92 percent I 
referred to earlier and the 35 percent in STAR, and the 
question for Mr. Johnson and Dr. Gilman is why the cuts? Where 
does this come from? This is not just a reallocation. This is 
really a cut, unless you can show me that you are transferring 
money to some other agency to do the same type of research. 
Particularly in the environment, Members of Congress are very 
interested in good science. They want it done right. They want 
to know the right answer so they can make the right decision, 
so I would appreciate comments from both of you on this issue.
    Mr. Johnson. The more substantive answer will come from Dr. 
Gilman, I suspect, but let me just give you my general view of 
this.
    As we mentioned, we think the focus should be on what we 
are getting, what the result is, what we are learning and 
what--how we are advancing our usual pursuit of a cleaner, 
healthier environment. The feeling was that the 20-some odd 
million dollars would be better spent in other programs than in 
the Ecological Research and Pollution Prevention Programs 
because we believe that there are more specific deliverables 
that could be resulting from those programs. We think those 
programs are well-funded now, and we believe that there would 
be little detriment to the output of those research programs 
with those cuts. We believe we could produce a greater return 
for the 20-some odd million dollars if we put those monies in 
the Prevention of Pesticides and Toxics Program.
    I think this President has demonstrated his commitment to 
all the things that EPA stands for and all of these 
environmental objectives that we all share. This is--should not 
be considered to be a back-off in the President's or the 
Administration's commitment to a healthy, clean environment. It 
should be considered to be a focus on what we should be--what 
we are trying to get for our money, not how much money we are 
trying to spend. Now for the real answer, Dr. Gilman.
    Dr. Gilman. Well, there is no question there is competition 
for resources. The Agency is putting forward a number of 
aggressive efforts proposing a $150 million increase in 
Superfund, proposing a School Bus Diesel Retro-Fit Program on 
the order of $60, $65 million. There is also an initiative in 
leaking underground storage tanks. You know, the list goes on. 
These are the kinds of things that create a competition for 
funds within the Agency. So in terms of where do we place the 
dollars, that is certainly an element for consideration. For 
the ecological research, the program still has a substantial 
effort beyond the cuts, as is the case in pollution prevention. 
So we think we can still drive those programs to have good work 
done and good outcomes from that work, even with these 
reductions.
    Chairman Ehlers. But I am really puzzled because science is 
the basis for what you do, and it seems to me you are 
transferring money out of scientific research and into 
programs, and I just can't believe that the science has been 
done. That you don't have to do it anymore, and that it should 
be cut by that. I am particularly concerned about the STAR 
program, which has received high ratings. You heard two 
panelists here today make good comments about it, good reviews, 
and it is generally acknowledged as being one of the strongest 
environmental programs. Why cut STAR by 35 percent? And I know 
there is always a tendency of managers to say well, we have got 
to protect our own employees, and therefore, we will cut the 
outside. But you get more for your money with STAR than I think 
you get almost anywhere else.
    What is the justification for cutting STAR?
    Dr. Gilman. Again, the decision wasn't made on the basis 
that STAR is a bad program. Decisions were made on the basis of 
the program reviews and the elements of the ecological and 
pollution prevention programs.
    Chairman Ehlers. What are these program reviews you are 
referring to?
    Dr. Gilman. The program reviews that we did in 
collaboration with the Office of Management and Budget's PART 
exercise.
    Chairman Ehlers. So you are saying the cuts are a result of 
applying PART?
    Dr. Gilman. Yes, for these two programs, that is the case, 
but not as it relates to performance of the STAR grant program 
per se. The reductions were made in the STAR grant programs 
because, just as those programs are the place where we can 
easily ramp up, it is also the place where, in terms of 
managing the entire research enterprise for the agency, we look 
to ramp down when we have to make a reduction. It is the place 
where we do the least disruption to our infrastructure for 
research when we have to make those reductions.
    Chairman Ehlers. But that implies that you are doing it 
because it is easier to ramp down, even though some of the best 
science is done there.
    Dr. Gilman. Well, I wouldn't say it necessarily is better 
than, though it sometimes is different than, the work that is 
going on inside the Agency. But the alternative of cutting back 
on the resources for the Intramural Program or reducing the 
Intramural Program is for the longer run. When we are very 
hopeful of re-engineering these programs and getting them their 
better measures and their better performance--to begin to 
reduce the internal infrastructure when we hope to be able to 
turn these things around doesn't make sense in our opinion.
    Chairman Ehlers. Well, it doesn't make sense to me when the 
President is trying to freeze the discretionary non-defense, 
non-security budget that we are cutting a major effort on your 
part. I mean, all of EPA is being cut, and that also doesn't 
make sense to me. If we are freezing, let us freeze everything 
where it is, but let us not cut to this extent.
    My time has expired. Let me just add one point, and after 
spending a fair amount of my life in science and part of it in 
managing science, I can assure you that managing science is one 
of the most difficult enterprises around because it is very 
difficult to develop measures for measuring science because you 
really literally do not know what the outcome is going to be. 
And it takes a special type of expertise and I am very, very 
suspicious of instruments and methodologies that are applied 
across the board by any agency, whether OMB or something else, 
that just says okay, fill in the blanks and then we will make a 
decision. That does not do justice to the enterprise, and I 
think we have to be very cautious about that.
    Mr. Johnson, did you want to respond before I go on?
    Mr. Johnson. Yes, I would. I think we recognize that R&D 
needs to be looked at differently than an operating program, 
and so the questions are different. I have gotten to know Eli 
Zerhouni in the last year plus, and he talks about the 
opportunity to better manage our investments in research better 
than we are now, and he understands way better than I and 
comparable to your understanding as to how difficult it is to 
measure this, but these can be managed. Shame on us if we don't 
continually try to deal with all of its difficulty, try to get 
a better understanding of what we are getting and are we on 
track, and is there accountability? And we understand that the 
end product, especially of basic research, is by definition an 
open question. But like every dollar we spend, we must continue 
to try to find better ways to make sure that the money is well 
spent.
    Chairman Ehlers. I don't agree with your goal. I disagree 
with the result you have come up with. I have--my time has 
expired. I would like to--who came first? All right. Zoe, you 
were first. Please recognize Congresswoman Lofgren from 
California.

     Incorporation of STAR Scientific Research into Decision-making

    Ms. Lofgren. Thank you, Mr. Chairman. I thank you for 
holding this hearing today. I think it is an important one, and 
although the testimony has been a little bit dense, I think we 
can really distill what is going on here so that it is easily 
understood.
    I remember years ago a Member of the House giving me a 
phrase about OMB. That they know the cost of everything and the 
value of nothing, and I think we have got that kind of fight 
brewing here--I don't mean disrespect, but it is a word we use 
often in the House, the bean-counters versus the scientists, 
and when it comes down to good science, I am going to rely more 
on the scientists than I am on the CPAs. And it does disturb me 
that the basic research that we are--have gotten through the 
STAR program, which has been evaluated by the scientists, and 
Dr. Denson, I found your testimony, written testimony 
particularly helpful because as you went through the analysis 
that has been undertaken of the program, it is not without 
rigor, and it--but it is not mechanistic. It has been an 
analysis that actually is thoughtful. It is not, you know, 
check off the boxes, and I think that is the kind of analysis 
that we need to have.
    And that doesn't mean that changes won't sometimes result 
because I know--I am not a scientist, as the Chairman is, but I 
have certainly--I come from Silicone Valley. I live surrounded 
by scientists, and there is not a more vicious critic of 
science than another scientist. So that can certainly result. 
But I am just sort of wondering, when you look at these nine 
specific results, on page eight of your testimony, really the--
one of the things that you discovered was not that the science 
that was being developed was flawed or defective, but there 
were big questions about the extent to which the research was 
actually being used, I am quoting you, ``to support the 
agency's environmental decision-making.'' So the question, as I 
understood it, was really not about the science delivered 
through the STAR program, but the agency's inability or 
questioning their ability to actually incorporate the data in 
their decision-making.
    Is that a fair analysis of that testimony? I don't want to 
misquote you, but that is the way it seemed to me.
    Dr. Denson.I think that is an accurate assessment. I would 
add one other thing, and that is that it takes a while for the 
research results to percolate through the system and to apply 
the results to some known situation. These results, the 
research results in many ways are autocatalytic in that they 
feed on each other, and so once you have a set of results, a 
scientist someplace reads those results and has another idea 
that yet takes it to another level. And finally, it gets to a 
critical mass where it becomes very obvious how these results 
can be used to implement a particular program.
    My own sense, and this is my personal reflection since I 
have been involved with it since nearly the start, is that the 
STAR and the agency have made great strides in improving the 
program and improving the way it implements the results. They 
have a way to go yet. I think the STAR program has some 
improvements to make, and we have mentioned what those are. But 
in every step along the way that I have been involved in the 
evaluation of STAR, when those comments have been made, the 
agency has responded very promptly and forthrightly in trying 
to make the changes and the improvements.

      Effect of Ecological Research Programs Budget Cuts on Human 
                        Health Research Programs

    Ms. Lofgren. Well, listening to your testimony and really 
the testimony of all the witnesses, it seems that the 
extramural portions of EPA's research efforts, including the 
research on ecological systems, I am particularly interested in 
the mercury analysis, are basically just not going to happen if 
the STAR program sustains the kind of cuts that are 
recommended. You are a prominent scientist, and I thank you 
very much for volunteering to do this for so many years. Is 
that your assessment as well?
    Dr. Denson. I believe that--I believe personally that it 
would be detrimental to cut the ecological programs for several 
reasons. One is that you have to recognize that EPA and ORD is 
a regulatory agency, and conducting research in a regulatory 
agency is different than conducting research in other kinds of 
agencies. One feeds on the other. It is a coupling effect. What 
happens in research on human health informs what happens in 
research and the results on the ecology, and vice-versa. So 
they are intimately coupled. And if you cut one, then in an 
agency of this sort, I think you seriously undercut the ability 
to do research across the board.
    My second point is this, is that the National Science 
Foundation conducts, in the environmental area, research on 
ecological problems. I mean, they don't do human health 
research.
    Ms. Lofgren. Right.
    Dr. Denson. The National Institute of Health does human 
health research. So we have two compartments. And what EPA, ORD 
is doing is to bring those two together because they are in a 
regulatory agency and they have to conduct research, one arm 
which informs the other. So to cut the ecological part, impairs 
EPA's and ORD's ability to carry out its mission. See, because 
that is part of their mission, and they need that, as we said 
in this report, they need a strong research arm.
    Ms. Lofgren. I know my time is over, Mr. Chairman, but I 
would just note that at a time when there has been a wide 
concern about proposals to increase the amount of mercury that 
can be permitted in the environment, I think the American 
people are going to be very suspicious of a proposal to cut 
research into the mercury element, while, you know, polluters 
are being promised relief for more mercury in the environment. 
I think it is a terrible message to give to the American people 
who I don't think ever asked for more pollution in their 
children.
    Chairman Ehlers. The gentlewoman's time has expired. 
Pleased to recognize the gentleman from Texas, Mr. Burgess.

     Immediate Effects of Science to Achieve Results (STAR) Grants 
                  Program Budget Cuts on EPA Research

    Mr. Burgess. Thank you, Mr. Chairman, and I also thank you 
for calling this hearing today.
    Let me first ask Dr. Gilman if the Science to Achieve 
Results, if the cuts in that program are implemented as 
written, what do we lose? What research are we going to--what 
is going to be the immediate effect of that?
    Dr. Gilman. There are a set of requests for proposals that 
we would have otherwise put out on the street that we will not 
put out on the street, but none of the cuts that we are 
proposing to the Extramural Program would, in and of 
themselves, terminate research overall. So there is still a 
substantial ecological research effort underway. It behooves us 
to do more towards collaborating with some of our other federal 
agencies to stretch the dollars we have there. In the case of 
pollution prevention, we have a substantial Pollution 
Prevention Program remaining. We are in the process of re-
engineering that program, and there is a true connection 
between the results of the PART process and our efforts to re-
engineer that process and change its focus. So none of the cuts 
to the program will terminate initiatives in either ecological 
research or in the pollution prevention research.
    Mr. Burgess. Well, then taking the last point from the 
gentlelady from California, will the mercury research and the 
research on human endocrine disruptors, will that continue?
    Dr. Gilman. That research does continue. The mercury 
research is a good example. Actually, we received a cut from 
the Congress in our proposed budget for 2004, which made us cut 
back on an initiative we had proposed in mercury-related 
research. In effect, the proposal for 2005 would take that off 
the table, as the Congress did as well, but we still have an 
intramural program. The big focus of our intramural program at 
this point is trying to better understand how we can do 
continuous monitoring of emissions for mercury so that we can 
put in place the regulatory program that has been proposed. The 
regulatory program that has been proposed for mercury reduction 
does constitute the largest reduction that has ever been 
proposed, and it is the first time that power companies will be 
required to reduce their mercury emissions.
    Mr. Burgess. Okay. Let me be clear on that then, that 
program is going to continue to go forward. Is that correct?
    Dr. Gilman. Yes.
    Mr. Burgess. All right. Then just to give us a flavor of 
what is on the shelf in those requests for proposals, what are 
some of the things that will not be brought forth as a result 
of the reduction in funds?
    Dr. Gilman. In substance, there is no one particular body 
of research that won't go forward. It is the supplemental work 
that would be done by extramural researchers. There is no 
question that that supplemental research would be of value to 
us, but it is not anything that we can't make up for in the 
future, in my opinion.
    Mr. Burgess. Very well. Dr. Denson, and too, I thank you 
for providing such a comprehensive but readable paper for us. 
You allude in here to that you do research that is not funded 
by other agencies, but there is also some discussion about 
leveraging your funding and partnering either with I guess 
other agencies or with private, nongovernmental organizations. 
Can you talk to us about that just a little bit? If the 
proposed cuts go through, how are you prepared to carry forward 
to make sure to maximize your research dollars and to leverage 
those dollars so that they are spent effectively?
    Dr. Denson. I would speak to that question, sir, as a 
Member of the NRC Committee. If I understand the question 
correctly, my response would be this. That one of the areas 
where we gave STAR program very high marks was in the area of 
partnerships and collaboration. The partnerships allowed STAR 
program to greatly leverage the funds that they have, and in 
particular, in terms of the ecology, I am thinking of 
leveraging the funds to the National Science Foundation and the 
Program for Technology for Sustainable Environment. So areas 
where there are not a great deal of resources in the STAR 
program, they have been very diligent with the money they do 
have in leveraging it with other organizations. As I said, NSF 
is a good example because they only work on ecological 
problems, on environmental problems.
    And the STAR program has been diligent enough to take what 
resources they have to leverage those with NSF. If those monies 
are cut, then one would have to look at the leveraging effect 
because not only do you lose the money that is cut, you lose 
what you could have gotten in that leveraging effect.
    Mr. Burgess. Very well. I see my time has expired. I take 
your point that we will lose the multiplier effect. I want to 
thank the panel for doing this today, being so well-prepared. 
It has been very informative to me, and I will yield back.

     Application of the Program Assessment Rating Tool (PART) and 
    Alternative Review Methods Within EPA's Office of Research and 
                              Development

    Chairman Ehlers. The gentleman's time has expired. Mr. 
Davis has stepped out of the room temporarily, so I will 
proceed with my next questions and we will recognize him, 
should he return.
    I just wanted to note, Dr. Matanoski, you in your testimony 
said that PART appears to favor short-term R&D programs with 
near-term benefits at the expense of programs of long-term 
benefits. I also noticed Mr. Posner has also had some 
criticisms of PART, and the real question is what is a good 
evaluation program? Mr. Gilman, you said that EPA is committed 
to developing good standards by which to measure EPA's R&D 
programs. A question, Mr. Johnson, is what standards would OMB 
find acceptable? It seems to be the thinking of the scientists 
here and GAO that PART is not applicable as it has been used to 
the scientific programs. I am wondering, and I am sympathetic 
to good management of science. I always have been.
    The question is can we do it right? I remember we worked 
very hard in this Committee when GPRA came out to modify GPRA 
as it is applied to science so that it made sense, and after 
some stumbles, I think that has begun to work. The question is, 
Mr. Johnson, what standards would OMB find acceptable if we are 
going to improve this? How can we make PART fit science, rather 
than the present situation where we have at least expressed 
opinions here that PART doesn't really fit in this situation? 
The Floor is yours.
    Mr. Johnson. I can't speak, maybe Dr. Gilman can, on what 
specific opportunities we have to improve the R&D criteria that 
are addressed in the PART R&D, R&D PART. But I do know Dr. 
Zerhouni is supportive of use of the PART in assessing various 
research programs, and the PART will get better and better each 
time. There is nothing magic about the PART, these 25 
questions. We think these are good questions now, and they will 
get better every year as we get smarter and smarter, and the 
goal here is to look at, on a regular basis, everything we 
spend money on, including science, and ask ourselves is this 
consistent with our priorities? Do we think we are getting 
something for our money? Is the research money of these long-
term projects being well-managed, and if not, what can we do 
about it?
    But I don't know the answer to your specific question.
    Chairman Ehlers. Okay. I will make a comment, but let me 
let the others comment first, and we will go through the whole 
panel. Dr. Gillman?
    Dr. Gilman. Okay. Mr. Chairman, EPA is not GPRA-phobic. We 
are not afraid to be measured. We, the science arm have tried 
hard. We need to do better. One of the things we have done is a 
pilot with the Office of the Inspector General, trying to 
develop a way of planning our program and measuring our program 
that really does look to long-term outcomes. That is a pilot 
effort that we have brought before the Office of Management and 
Budget and hope to continue the discussions whether we can look 
at the program slightly differently? To try and sum it up for 
you, when you are developing research in science in a mission-
oriented Agency like our own, it is difficult to look at the 
results of that work in the absence of asking yourself how well 
has the program integrated the results that were to achieve the 
long-term outcomes?
    So what we are, in short, proposing is an integrated 
process, looking at both the program and the research arm to 
ask the question how is the Agency doing in getting the 
outcomes it wants with the research it is engaged in?
    Chairman Ehlers. As you know, we have had a number of 
discussions about integration and I am very interested in 
strengthening that effort within the EPA by increasing the 
scientific component of it. Mr. Posner.
    Mr. Posner. Thank you, Mr. Chairman.
    I want to make sure we are not saying that PART shouldn't 
be applied to science programs. I think recognizing that there 
are particularly unique challenges, given the long-term nature 
of how results unfold and the uncertainties with science 
programs, means you have to be careful, and OMB does have 
particular questions for R&D. That may be one area that needs 
to be focused on. I want to add, thanks to the National Academy 
of Sciences, the National Science Foundation and others, there 
are developed criteria for doing this.
    One of the things that we typically find when we look at 
research, such highway research, is it the success story model, 
which is really not the way necessarily to go, where agencies 
reach for individual good examples to justify their programs. 
But what we want to see and what has been developed in the 
National Science Foundation and others is a way for scientists 
to get peer-reviewed retrospective evaluations of portfolios of 
research. This is something that has been developed in other 
agencies and it is the kind of thing that I think OMB has 
endorsed in their research criteria. The National Academies of 
Science really laid this out as a model.
    So when we are talking about having leading scientists 
review the results of science programs, that is the model of 
accountability that makes sense here.
    Chairman Ehlers. Well, thank you, and what we are 
specifically interested in is what specific measure should OMB 
use to evaluate EPA's R&D programs, or for that matter, other 
R&D programs? So we will continue with Dr. Matanoski.
    Dr. Matanoski. I think our committee, the Science Advisory 
Board, in looking at the actual outcomes, suggested that the 
short-term outcome would fare much better in the PART review 
than would any long-term outcome. Now if you add on top of the 
long-term outcome something like ecological research, which is 
so complex, it has so many ramifications to it that when you go 
back and start looking at one basic piece of it and then carry 
it all the way to its completion at the end, which we don't 
even do to be truthful--we don't even know what is the best 
ecological end for most of what we are looking at, take that 
and try to produce an outcome with it becomes extremely 
difficult.
    It is easy to say I have to model something and I go in and 
I model that, like whether jellyfish are doing well in the 
Chesapeake Bay, and if you can do that, you can do that very 
well and you can do certain pieces of research and figure it 
out. But when you try to put the whole thing together, the 
entire fish population, the entire growth population, the 
entire water systems of all of these, it becomes a very complex 
not only research arm, but a very complex outcome arm. And so 
the long-term outcome is you want something better in your 
coastal waters, for example. But all the pieces that have to go 
into the research to get there and then the application become 
too complicated to measure simply with a tool.
    I agree wholeheartedly that everybody has to be accountable 
for their money, in science as well as anywhere else. It is 
just the same as if we were in a business. But by the same 
token, when you get a complex program like that, it is very 
difficult, and our recommendation was that they begin to think 
about instead looking at a committee that looked at the tool 
along with an actual science review group, and then gradually, 
you could work aware, as you began to figure out what was the 
best way to do this. You could begin to work your way back and 
evaluate each piece. Right now, it is such a complex issue; it 
would be very difficult to do that.
    Chairman Ehlers. Dr. Denson.
    Dr. Denson. Yes, sir. I am going to answer the question in 
two ways, from two perspectives. One is that from the 
perspective of the Committee, the NRC Committee. That Committee 
attempted to do guidelines at that time, those guidelines 
required the Committee to look at metrics of performance, 
quality and relevance of the program. And the Committee judged 
the programs on those bases. The metrics they used were both 
product and process metrics. The Committee discarded 
bibliometric types of analysis, quantitative types of analysis 
in evaluating the program, and instead concluded that complex 
research programs, and I use the word programs very carefully, 
require a different kind of evaluation. And that evaluation 
means bringing in a panel of experts to peer-review the program 
in all of its aspects. To bring in a panel, peer-review all of 
the aspects of the program.
    And as we point out in the report, we call this a level 
three or a level four evaluation. Level one evaluations, which 
look at individual grants, the progress of an individual 
research project can be graded quantitatively, and you can use 
bibliometric analysis for that. But the Committee concluded 
that that was inappropriate for a program as complex as the 
STAR program, which we consider to be a level three program. 
The ORD program in general would be a level four program, and 
you would need a much bigger, much broader panel of experts to 
review that.
    I am going to change for just a moment, sir, and bring out 
the fact that for four years, I served on the National Science 
Foundation's Advisory Committee for the Engineering 
Directorate. I also served for two years on the National 
Science Foundation's Directorate Advisory Committee for 
Environmental Research and Education. So I have a perspective 
of how it is done in one of the very best research agencies. 
And there, while they follow the GPRA and have the tools, ideas 
and people concern, there it is also difficult to do a metric 
on some of the programs until they have had a chance to fully 
develop. And my point about the STAR program is that it just 
needs a little bit more time, and I think that given that time 
and more results and having a review of a complex program by a 
peer-review panel would be the metric to use, and we say that 
in our report.
    Chairman Ehlers. My time has expired. We will recognize the 
gentlelady from California, Ms. Lofgren.

     Termination of EPA's Building Decontamination Research Program

    Ms. Lofgren. I wanted to explore a specific program and--
because I serve on the Homeland Security Committee as well, and 
I am on the Cybersecurity and Science Subcommittee, and we are 
obviously spending a lot of time in that area. And I was 
interested in the termination of the Building Decontamination 
Research Implementation Plan, and I was wondering, Dr. Gilman, 
it is my understanding that last year, the National Academy of 
Sciences, at EPA's request, reviewed that plan. They indicated, 
the National Academy did, that it was not possible really for 
EPA to get the job done in the timeframe remaining. I am 
interested in why EPA wants to terminate the program, rather 
than extending it and is that your judgment that building 
decontamination is now a lesser threat, or is that because this 
decision was made in conjunction with the Department of 
Homeland Security, Mr. Ridge, and is it going to be transferred 
to Homeland Security? Because we haven't seen it surface yet in 
that department.
    Could you give me some report on----
    Dr. Gilman. Sure.
    Ms. Lofgren.--that?
    Dr. Gilman. Let me start by giving you a little background 
of the research effort at EPA, which we are very proud of. We 
set out to establish a center there. It is something of a 
virtual center. We gave it a time, a sunset in creating the 
center, a three-year window to do the work. Why did we do that? 
We did that because we wanted to really drive the folks 
involved in delivering products to first responders and to 
planners. So we created the center. Its mission was to work 
heavily through partnerships with Homeland Security, the Army, 
Air Force, Food and Drug Administration, all of the different 
groups where we might find technologies and approaches to 
utilize in the mission areas for the EPA--they include water 
infrastructure, both drinking and sanitary. They include 
buildings, and it includes the use of tools for rapidly 
assessing risks for different factors.
    Those are the three areas the research center is focused 
on. We are in the process right now of working with those 
various agencies who view themselves as our customers to 
determine what we should do about that three-year sunset, 
whether there should be a further research effort within a 
center embedded elsewhere in the Agency. The decontamination 
program, that again was meant to be a three-year program. That 
is what the National Research Council reviewed for us. We are 
in the process now of trying to identify those things that were 
in the third year of our program that we would like to move up 
into the second year. We haven't expended all the resources 
that have been appropriated for the first two years yet, so we 
do have the flexibility to change our priorities, and as I 
said, we are in consultation with Department of Homeland 
Security and others now about what their desires are for our 
future work.
    I think the products that we have turned out so far are 
quite important and we are very, very proud of the productivity 
of that group to this point in time.
    Ms. Lofgren. It looks to me that the sun set on the thing. 
It is not really a work in progress, and it may be that we want 
to transfer the whole thing over to Homeland, but it is just I 
don't think that has happened, and with so many things in the 
Homeland Security area, not just environmental issues but where 
you have an activity going on by another agency and it never 
really gets picked up and it leaves the United States exposed 
and bare, I mean, in many respects, not just in the 
environmental arena, I am concerned that this is another one of 
them.
    My time is just about up, Mr. Chairman. I would just say 
that I think this has been a very useful hearing. One thing I 
think has become very clear in the questioning is that the 
mechanistic application of PART two science is not going to 
serve the country well, and I really appreciate your 
leadership, not only as Chairman, but as a scientist in helping 
this committee, and then later, I think the Congress to 
understand that that will not well serve our country and our 
future. So I yield back the time, and thank you once again.

               Peer Review and Evaluation of R&D Programs

    Chairman Ehlers. The gentlelady yields back her time. Let 
us continue with some of the previous discussion. There seems 
to be a general agreement among three of the panel that peer-
review is the best way to review R&D programs. I just want to 
establish, Mr. Johnson, Dr. Gilman, do you agree that peer-
review is the best method to review the science programs, the 
R&D programs?
    Mr. Johnson. Well, I will let, again, Dr. Gilman answer in 
a more detailed fashion, but I believe the PART has a very 
substantial role in the assessment of R&D programs. There is 
nothing mechanistic, not one thing that is mechanistic about 
the use of PART. Nothing happens automatically as a result of a 
PART score. I would like to just add one other comment--to 
elaborate on something Dr. Gilman said about the building 
contamination research. The balance--the majority of the funds 
appropriated for this research in the last two years are un-
obligated. They have not been spent or committed. There is 
already plenty of money available for this effort. Additional 
monies are not needed in the fiscal year 2005 to continue this 
effort in an aggressive fashion.
    Chairman Ehlers. All right. Thank you. Dr. Gilman.
    Dr. Gilman. The pilot that we worked with the Inspector 
General and the work that we would hope to do for the future 
does involve the use of peer-reviewers in evaluating both the 
research outputs and the long-term outcomes. The PART process 
envisions the use of that kind of peer-review. The National 
Science Foundation has availed itself of that. Those are the 
kinds of things we hope to be discussing with the Office of 
Management and Budget for the future.
    Chairman Ehlers. Let me just comment. There are really two 
aspects, and I have seen these get interwoven here, but not 
clearly distinguished. It seems to me part of it is simply 
evaluating the science per se. Is the EPA doing good science? 
Is it judged by peers, it is really good work and should it be 
done? That is one aspect. The other aspect is, in a sense, a 
management aspect. Does the research fit with the mission and 
the program of the EPA? And I think it is important to clearly 
distinguish those. Both of those are difficult to do, but they 
are of a different nature, and what is most difficult is to 
mesh the two. But isn't that what you are trying to do, Dr. 
Gilman, with your work and Mr. Johnson, your primary interest, 
I believe, is does the science fit the program? Does this make 
sense for the government to do? Is that a fair statement?
    Mr. Johnson. Well, I would say more than that. Does it make 
sense for what our priorities are, is it promising fruit? Is it 
on track? Is the money being well-spent?
    Chairman Ehlers. Right.
    Mr. Johnson. Is it being spent in a responsible fashion? 
But I would also say that with regard to this--we are talking 
about generalities, and we also need to talk about specifically 
this program, the Ecologic Research Program. I am not a 
scientist, but as I understand it, Ecologic Research, the 
primary opportunity in this area of inquiry has to do with the 
opportunity for us as a country is better monitoring and better 
diagnosing. It is a pretty specific area. Therefore, the 
primary benefit or the opportunities to be pursued with 
research is to develop tools and databases to help informed 
decision-making.
    There is, for this kind of research, a much more specific 
kind of end product, if you will, which is something you 
don't--it is not the right term for research, but it can--it is 
more purposeful than most research programs, and therefore, I 
think it lends itself more to PART assessment than basic 
research.
    Chairman Ehlers. Absolutely, and I don't argue that. 
However, that is fairly low-level science, which makes it easy 
to analyze. I mean, monitoring and diagnosing is--well, I don't 
want to get into Pasteur's Quadrant here. Some of the 
scientists will know what this is. But that is in the lowest 
quadrant. The difficult part comes in, as one of the scientists 
mentioned here, synthesizing the information after you have 
monitored it, after you have diagnosed it, and then 
synthesizing it with other results from elsewhere.
    So you are looking at organism X and organism Y and 
organism Z and many others and seeing how they are interacting 
and what the net effect is, and you find some very surprising 
things there, which you don't anticipate, some of which create 
greater environmental problems, some of which actually solve 
your problems. For example, depending--developing microbes that 
will consume toxic materials or oil. That is very basic 
research, but yet it is also related to the program. If you are 
trying to clean up underground storage tanks, as we have been 
trying to do for over a decade, that is a very important issue, 
if you can clean those up with microbial action rather than 
digging it up, hauling it somewhere, treating it at a very high 
cost.
    So it all interrelates, and that is where my concern is. 
How are we evaluating how everything interrelates? And just 
without really knowing as much as I should about the evaluation 
program, it seems to me the PART program is ideal for what you 
just described, Mr. Johnson. But I don't think it is--from what 
I have seen, it is not the appropriate instrument for the 
broader picture, and the question is can we all, working 
together, develop something that is more appropriate for the 
broader picture? We are not here to condemn EPA or OMB. We are 
all part of the same team. We are trying to help develop 
something that really does the job and does it in the best way 
for the country, and I hope everyone, both panelists and 
audience, understand this. Our goal is, as I often tell my 
people back home when I give a speech, I am here from the 
Federal Government, and I am here to help you----
    Mr. Johnson. Yeah.
    Chairman Ehlers.--which usually brings a laugh. But that--I 
am serious about that. We really want to try to find out what 
the problems are and what we can do to help. Mr. Johnson.
    Mr. Johnson. But as Dr. Poser said, or Mr. Poser said, one 
thing I believe with all kinds of programs, including research 
and development, basic and applied, the PART is causing us to 
ask really good questions, and that is more important than the 
kind of answers that we are getting, but it is causing us 
questions that we weren't asking before.
    Chairman Ehlers. And Mr. Johnson, I have no objection to 
that. I think that is an important function. The question is 
how those answers are evaluated and how they are used, and what 
the appropriate use is. Did you have additional questions, Ms. 
Lofgren.

              Evaluating Short-term vs. Long-term Research

    Ms. Lofgren. I think I do, but I think we are going to be 
having a vote soon, and the only thing I would note is Dr. 
Matanoski, in your testimony, you describe research that is 
targeted to fill a research gap in the short run versus other 
types of research intended to provide an understanding of large 
systems in the long run, and I am just wondering if it is 
possible to have programs that have only short-term or long-
term goals, and not both. And as I look at the PART questions 
themselves, it appears that really, you are going to get 
penalized, and you can't get an approval if you have done that, 
and if your understanding is similar to mine.
    Dr. Matanoski. I think it is very similar to what you have 
said. You have hit it right on the nose. If you have something 
that is very simple that I am going to evaluate, an operative 
procedure or something, and I know what the long--what the 
outcome is going to be and they are going to come off the table 
alive or dead, that is my outcome. That is easy. And I can tell 
you how many operations I do and how many are going to take 
place, you know, and who is going to do them.
    When you get into a program as complicated and integrated 
as ecological research, then you really begin to have to look 
at the independent parts, part of which was STAR. And so STAR 
was at the very basic end of it, and some of the more applied 
and the monitoring, which is fairly easy, you know. You put how 
many monitors out there. How much do they show us, and whether 
they have improved the quality of the water. But when you begin 
to look at the whole picture that is a very difficult thing to 
do. So what do we need to do? Separate it out so I have a 
product that I can say comes out next week, and if I get it out 
next week, great. I have just met my goal. And, you know, that 
is--if you get that simple, like putting a car on the line and 
coming out at the other end, then you can easily use a----
    Ms. Lofgren. Right.
    Dr. Matanoski.--very simple metric system. The more 
complicated you get, the more complicated any metric system 
would become, and right now, we are probably stuck with not 
having a very good metric system for a complicated problem.
    Ms. Lofgren. Right. In just looking at the questions, it is 
really biased against basic, complex research, it seems to me. 
I remember, Mr. Chairman, when I was first elected to the House 
and the late Congressman Steve Schiff was on the Science 
Committee and I was new to the Committee, and he actually gave 
an education to me and other new members about the value of 
basic research, and really its role in not just solving 
problems that are immediately before us, but really in building 
the economy of the country. And I think we would be ill-advised 
to forget Steve's lessons to us, and I yield back the balance 
of my time.

                           Concluding Remarks

    Chairman Ehlers. I thank the Congresswoman for her 
comments, and I think we have kept you here long enough and 
tortured you enough, but I certainly want to thank the 
panelists for being here. You have been very, very helpful to 
me in helping me learn more about it from both sides of the 
issue, and I appreciate you being here. I would like to just 
summarize a few thoughts I have. So far, I have not heard any 
convincing reason why the STAR program should be cut so 
drastically, by all accounts. Not just what we have heard here, 
but other accounts I have heard. It is a well-run, competitive 
peer-reviewed program that produces high-quality research, and 
I don't think the proposed reduction should take place, and so 
we will work with the appropriators on that and see what we can 
achieve.
    I also learned this morning the PART program was at least 
partially responsible for the funds being cut from Ecological 
and Pollution Prevention Research Programs, but it doesn't 
appear to me that PART is related at all directly to the cuts 
in the STAR program, and again, I don't quite understand why 
the cut, but we will have further discussions. And finally, I 
believe that PART has been a valuable tool for OMB in a number 
of different areas, and I am just not yet convinced that it is 
the appropriate one as it is structured to use for EPA and 
particularly I would be interested in how conclusions are drawn 
from the PART review and how they are used. It seems to me that 
the results of the PART program were not applied uniformly 
across the agency. That certain areas were cut more than they 
should have been, others were cut less than they should have 
been, if you look strictly at the PART review. And that is 
something else that we would like to investigate. Thank you 
again for being here. If there is no objection, the record will 
remain open for additional statements from the Members, and for 
answers to any follow-up questions the Subcommittee may ask of 
the panelists, and I hope you will be willing to answer in 
writing if any Subcommittee Members wish to send you 
communications. Without objection, so ordered.
    Hearing is now adjourned.
    [Whereupon, at 11:30 a.m., the Subcommittee was adjourned.]
                               Appendix:

                              ----------                              


                   Answers to Post-Hearing Questions




                   Answers to Post-Hearing Questions
Responses by Clay Johnson, III, Deputy Director of Management, Office 
        of Management and Budget

Questions submitted by Democratic Members

Q1.  GAO's review of the PART program indicated that PART and 
Government Performance and Results Act are not well integrated. GAO 
recommends these programs be integrated to be more complementary. Does 
OMB intend to adopt this recommendation? Will OMB alter PART to conform 
to GPRA or will agencies be instructed to alter their GPRA performance 
plans to conform to PART? Since GPRA mandated an open process for the 
development of agency strategic plans and performance measures, what 
process will OMB use to include Congress and program stakeholders in 
the effort to conform PART and GPRA performance plans?

A1. Responding in part to GAO's recommendation that PART and GPRA be 
better integrated, OMB clarified its PART guidance so agencies 
understood that PART is simply a tool to ensure GPRA is implemented as 
intended. The guidance now includes the following:

         RELATIONSHIP OF THE PART TO THE GOVERNMENT PERFORMANCE AND 
        RESULTS ACT (GPRA)

         The GPRA statute provides a framework under which agencies 
        prepare strategic plans, performance plans, and performance 
        reports that set goals and report on the extent to which they 
        are achieved. The PART is a systematic method of assessing the 
        performance of program activities, focusing on their 
        contribution to an agency's achievement of its strategic and 
        program performance goals.

         The PART strengthens and reinforces performance measurement 
        under GPRA by encouraging careful development of performance 
        measures according to the outcome-oriented standards of the law 
        and by requiring that agency goals be appropriately ambitious. 
        Therefore, performance measures included in GPRA plans and 
        reports and those developed or revised through the PART process 
        must be consistent.

         The PART also extends the usefulness of performance 
        information by defining programs that comprise activities about 
        which management and budget decisions are made. As a matter of 
        sound manage practice, agencies will integrate operational 
        decisions with strategic and performance planning by:

                  improving performance measures over time 
                through the PART review,

                  aligning budgets with programs, and

                  aligning programs and measures with GPRA 
                goals.

    If agency goals are revised as a result of the PART process, 
agencies must engage in whatever consultation with stakeholders, 
including Congress, is required by GPRA.

Q2.  In cases where OMB and the Agency disagree about the performance 
measure used to evaluate a program with PART, how is the disagreement 
resolved? Please provide documentation of the EPA and OMB positions on 
the PART evaluation of the Ecosystem Research Program. Please include 
documentation of the negotiations that led to the final resolution of 
the differing positions and the awarding of the final PART evaluation 
score for this program.

A2. OMB and agencies complete PART assessments together. We work hard 
to come to agreement on every answer to PART questions. Agencies can 
``appeal'' OMB answers to PART questions to OMB policy officials, but 
this process is rarely used. The give and take between OMB and agencies 
in PART assessments is considered pre-decisional and therefore cannot 
be disclosed. However, OMB and EPA would be happy to discuss with you 
and/or your staff the process EPA and OMB went through to complete its 
PART assessments and, more importantly, what we are doing subsequently 
to improve the programs we assessed.

Q3.  Under GPRA, program goals and performance matrices for programs 
are supposed to be adopted in consultation with Congress and outside 
stakeholders. However, it appears that PART-defined program goals and 
performance metrics are superseding those established under GPRA. 
Please provide a side-by-side comparison of the PART goals and 
performance metrics for the EPA programs that have been assessed under 
PART in the 2004 and 2005 budget and the goals and performance metrics 
for these same programs developed under GPRA. Please provide an 
explanation of all instances where PART and GPRA program goals and 
performance metrics differ and OMB's rationale for deviating from GPRA 
to develop PART.

A3. As discussed above, the PART is simply a tool to ensure GPRA is 
implemented as intended. The PART is used to ensure, among other 
things, that program goals meet the high standards of GPRA. Where we 
find that existing goals do not meet those standards, agencies may be 
required to revise the goals associated with the programs being 
assessed.
    I am assured that EPA conducts extensive outreach to Congress and 
consults with its stakeholders when developing goals, objectives, and 
performance measures under GPRA. In its Strategic Plan and Annual 
Performance Plans, EPA seeks to use the highest-quality and most 
outcome-oriented goals and measures for which valid performance 
information is expected to be available. OMB and EPA consider EPA's 
GPRA goals and measures to be the definitive set for Congress and the 
public to use to understand the agency's budget in performance terms 
and to hold the agency accountable for results.
    The PART is not a substitute for GPRA. It is an assessment tool 
that informs management decisions, enhances organizational learning, 
and promotes effective strategies and program results. The PART is used 
to ensure, among other things, that program goals meet the high 
standards of GPRA. The PART process has been one of several drivers for 
improving the agency's performance measures. Some of the new measures 
identified during a PART assessment will become GPRA measures. In many 
cases, newly identified measures are not ready immediately for 
inclusion in GPRA documents, since it will take some time for the 
agency to conduct appropriate monitoring and data collection, to 
establish a baseline, and to set relevant targets. In these cases the 
agency develops specific plans, with milestones, for putting the 
necessary elements in place to use it as a GPRA measure. Each of the 
agency's program offices vets these measures with stakeholders, just as 
they do with any newly proposed measures.
    Some performance measures, notably efficiency measures, which are 
identified under the PART process, are more appropriately used as 
management tools. In other cases, PART assessments have been conducted 
on a relatively small set of agency activities that are not readily 
identifiable as a distinguishable unit in GPRA documents. In these 
cases, new measures identified under the PART process may complement 
GPRA measures, but they are not necessarily in GPRA documents.
    EPA included a discussion about the PART and performance measures 
in its most recent Annual Report, which we are including with this 
package (see pp. 12-13, Using Program Evaluation and Improving 
Environmental Indicators, Performance Measurement, and Data Quality and 
pp. 109-110, Support for Environmental Results, Fiscal Year 2003 Annual 
Report, U.S. Environmental Protection Agency). Also included with this 
package are summaries of all EPA programs assessed with the PART. EPA, 
in full consultation with stakeholders, has increased the percentage of 
externally reported annual performance measures based on environmental 
outcomes (as opposed to activity-based outputs) from seven percent in 
the FY 1999 Annual Performance Plan to 36 percent in the FY 2004 Annual 
Performance Plan.

Q4.  OMB established two groups to consult with regarding the 
development of PART, the Performance Measurement Advisory Council and 
an Interagency Review Panel. What recommendations of these groups were 
adopted as revisions to PART for the 2005 budget presentation? What is 
the current status of these two groups and what role will they play in 
the making adjustments to PART for the 2006 budget? Please provide 
documentation of the date of meetings held between OMB and the PMAC and 
the IRP, a list of persons attending these meetings, and a brief 
description of the issues discussed.

A4. The Performance Measurement Advisory Council (PMAC) was established 
in June 2002 pursuant to the Federal Advisory Committee Act (FACA) to 
provide OMB with independent, expert advice relating to performance 
measurement. Pursuant to the statute, the PMAC's duration was limited 
to nine months and all the meetings were public. Minutes from those 
meetings are available on the OMB website at http://www.whitehouse.gov/
omb/budintegration/pmac-index.html. The PMAC's 
recommendations are detailed in the meeting minutes.
    In fall 2002, OMB established an interagency panel to review a 
sample of draft PARTS completed the first year, judge their consistency 
with the PART guidance, and make recommendations for how consistency 
could be improved. The panel also reviewed a selected number of 
disagreements between agency and OMB staff on how particular questions 
in the PART had been answered. The panel's work reviewing draft PARTs 
is considered pre-decisional and therefore cannot be disclosed. In the 
two subsequent years of PART implementation, similar quality control 
exercises have been performed. In summer 2003, the National Academy of 
Public Administration convened a working group to assist OMB with the 
PART consistency check. This summer, OMB conducted the consistency 
check on its own. We would be happy to brief you in greater detail on 
the substance of the groups' work and the improvements that resulted 
from it.
                   Answers to Post-Hearing Questions
Responses by Paul Gilman, Assistant Administrator for Research and 
        Development, United States Environmental Protection Agency

Questions submitted by Representative Zoe Lofgren

Q1.  In relation to the proposed rule on mercury emissions from coal-
fired electric utilities (Fed. Reg. Vol. 69, No. 2: Friday, January 30, 
2004; pp. 4651-4752):

Q1a.  Provide a list and a brief description of the studies relied upon 
by EPA to develop the proposed mercury cap-and-trade program prior to 
the release of the proposed rule in January. Please indicate the 
author/s of these studies and indicate where EPA has made the studies 
available for examination by the general public and Members of 
Congress.

A1a. EPA places all studies relied upon by EPA to develop the proposed 
and final rules in the docket. The Clean Air Mercury Rule docket can be 
found at http://docket.epa.gov/edkpub/index.jsp by searching for the 
docket ID OAR-2002-0056.

Q1b.  Administrator Leavitt has recently announced his intention to 
initiate and/or obtain additional analysis on the proposed mercury 
regulation. What additional analyses will be undertaken? What aspects 
of the proposed mercury cap-and-trade program for electric utilities 
will be examined by these new analyses? Will the analyses be undertaken 
by EPA's Office of Air and Radiation or by EPA's Office of Research and 
Development? What is the anticipated date of completion for the 
additional analyses?

A1b. As Administrator Leavitt has clearly stated, EPA will make sure 
that we have all the analysis necessary to make the right decision 
about how to address mercury emissions from power plants.
    During the comment period on the proposal, EPA received a number of 
relevant analyses from various groups, including both industry and 
environmental groups. These analyses address many of the key issues 
faced by EPA and will help inform the Agency's ongoing mercury work. 
They are available in EPA's docket, and the Agency is seeking public 
comment on these analyses in its Notice of Data Availability (NODA) 
released on November 30, 2004. The NODA is part of the EPA process 
toward delivering a final mercury rule by March 15, 2005.
    EPA received a number of modeling analyses from various groups, 
including both industry and environmental groups. In some cases, EPA 
and commenters modeled the same or similar policy scenarios, sometimes 
using the same model, but obtained substantially different results due 
to differences in the assumptions employed. In these cases, model-input 
assumptions can be better understood by comparing and contrasting the 
modeling performed. The NODA shares these analyses and seeks additional 
comment on the models and assumptions used.
    Administrator Mike Leavitt has outlined five guiding principles 
that provide context for additional inquiry and that narrows the focus 
of the Agency's deliberations. The five principles will ensure that the 
final mercury rule: (I) concentrates on the need to protect children 
and pregnant women from the health impacts of mercury; (2) stimulates 
and encourages early adopters of new technology that can be adequately 
tested and widely deployed across the full fleet of U.S. power plants 
utilizing various coal types; (3) significantly reduces total emissions 
by leveraging the $50 billion investment that the Clean Air Interstate 
Rule (CAIR) will require; (4) considers the need to maintain America's 
competitiveness; and (5) comprises one of many agency actions to reduce 
mercury emissions.

Q1c.  What study or combination of studies led to EPA's assertion that 
the cap-and-trade mercury control program would reduce mercury 
emissions by 70 percent by 2015? What do EPA's analyses show in regard 
to the anticipated change in the geographic distribution of U.S. 
mercury emissions from coal-fired electric utilities if the cap-and-
trade mercury control program were adopted as compared to the 
reductions anticipated if the MACT mercury control program were 
adopted?

A1c. The proposed mercury cap-and-trade program would place an 
emissions cap on mercury emissions from coal-fired electricity 
generating units. This cap would be implemented in two phases. The 
second phase of the program would begin in 2018, with a cap of 15 tons 
for emissions from these units. When this cap is fully implemented, 
emissions from affected units would be reduced by approximately 70 
percent. The 15 ton cap represents a 70 percent reduction from the 
estimated 1999 level of 48 tons of mercury emissions from coal-fired 
electricity generating units. Once the 15 tons cap is fully 
implemented, emissions would not exceed that level, because the cap and 
trade mechanism places an absolute limit on allowable emissions. 
Because the cap is lowered in 2018, cumulative emissions reductions 
under the proposed cap-and-trade program would exceed cumulative 
emissions reductions under the proposed MACT. Furthermore, EPA analyses 
suggest that the proposed cap-and-trade program attains more reductions 
from larger and higher emitting facilities than the proposed MACT.
    An important feature of the proposed mercury cap-and-trade program 
is the ability for sources to bank emissions allowances. Sources can 
over-comply with the program in one period and bank remaining 
allowances for use in a later period. Banking of allowances provides 
flexibility to sources, encourages earlier or greater reductions than 
required, stimulates the market, and encourages efficiency. By 
encouraging early reductions, banking provides early human health and 
environmental benefits relative to what would occur otherwise, though 
it results in extending the time until the cap is reached. We expect 
that sources will take advantage of the banking provision under the 
mercury cap-and-trade program by beginning to reduce emissions soon 
after the program takes effect. Allowing banking should not affect the 
cumulative mercury reductions achieved under the program.

Questions submitted by Democratic Members

Q1.  EPA's budget presentation includes program funding request changes 
for FY 2005 relative to the Administration's FY 2004 budget request. 
Please provide the changes for the following programs relative to the 
FY 2004 enacted appropriations for EPA including the funding level 
received in the FY 2004 enacted appropriations law and the requested 
funding level for FY 2005. In cases where a decrease in funding is 
proposed include a description of the activities proposed for 
termination. In cases where and increase in funding is proposed, 
include a description of the new or expanded activities that will be 
undertaken.

Q1a.  Office of Research and Development (ORD) research and development 
funding for mercury.

A1a. Mercury Research, EPA's Office of Research and Development.




    EPA will maintain in-house research in this area, but plans to 
award five fewer Science to Achieve Results (STAR) grants for mercury 
research in FY 2005.

Q1b.  Office of Air and Radiation (OAR) research and development 
funding for mercury

A1b. In the Science and Technology Appropriation in the FY 2004 
Operating Plan, OAR has $361,700 for mercury activities; OAR has 
requested $361,700 in the FY 2005 President's Budget. This funding 
supports the regulatory and guidance programs within OAR for mercury.

Q1c.  ORD research and development funding on fine particulates.

A1c. Particulate Matter Research, EPA's Office of Research and 
Development




    The increase in resources will provide funding for a recently 
awarded 10-year, $30 million Science to Achieve Results (STAR) grant to 
study the connection between long-term exposure to air pollution and 
cardiovascular disease. The balance of the resources will support STAR 
research examining emerging biological mechanisms that can explain 
observed health effects associated with exposure to PM.
    The findings from these studies will assist policy-makers in 
understanding the public health consequences of long-term exposure to 
air pollutants, and will contribute important information for PM 
standard setting by increasing our understanding of the biological 
plausibility of reported effects; the effects of PM and co-pollutants; 
who is most susceptible to these effects; and the effects of long-term 
exposure to PM. All these subjects are highlighted in the 2001 NRC 
Report IV on PM research priorities.

Q1d.  Office of Air and Radiation (OAR) research and development 
funding for fine particulates.

A1d. In the Science and Technology Appropriation in the FY 2004 
Operating Plan, OAR has $33,009,800 for fine particulates activities; 
OAR has requested $42,762,500 in the FY 20005 President's Budget, This 
funding supports the regulatory and guidance programs within OAR, 
particularly the Mobile Source program.

Q1e.  ORD research and development funding on ozone.

A1e. Tropospheric Ozone Research, EPA's Office of Research and 
Development




    The decrease is due to a realignment of research support resources, 
and there is no programmatic impact to this research effort.

Q1f.  Office of Air and Radiation (OAR) research and development 
funding for ozone

A1f. In the Science and Technology Appropriation in the FY 2004 
Operating Plan, OAR has $34,167,3000 for ozone activities; OAR has 
requested $36,224,400 in the FY 2005 President's Budget. This funding 
supports the regulatory and guidance programs within OAR, particularly 
the Mobile Source program.

Q2.  The FY 2005 Budget Request describes the Green Suppliers Network 
(GSN) activity within the pollution Prevention Program. How much 
funding does EPA intend to provide in FY 2005 to support this 
partnership program and meet the goals of expanding the GSN 
partnerships to include four additional industry sectors? How much 
funding did EPA supply to support the GSN pilot effort in 2002? What 
were the funding levels for this activity in 2003 and 2004?

A2. The promise of the Green Suppliers Network (GSN) is best 
illustrated by the response to it by industry. The program is not yet a 
year old and already countless Original Equipment Manufacturers and 
their suppliers have joined the program. It is truly a win-win for all 
involved. The Original Equipment Manufacturers benefit because the 
program is designed to improve their supply chains both economically 
and environmentally. Small businesses benefit because they are being 
shown opportunities to lean their operations and improve their 
environmental footprints, while at the same time achieving significant 
cost savings. The states benefit because the small businesses and their 
clients located within their jurisdictions remain competitive, leading 
to job retention and viability within the state's economy. EPA and the 
environment benefit because the GSN outlines direct opportunities for 
environmental improvement from energy savings to waste minimization to 
the reduction in the use and emissions of toxic chemicals.
    When the program was launched, EPA did not expect the response from 
industry to be so strong and so immediate. Currently, The Green 
Suppliers Network is partnering with the Automotive Industry, the 
Aerospace Industry, the Office Furniture Industry and the Healthcare/
Pharmaceutical Industry. However, EPA has been approached by several 
other industries which have shown an interest in participating in the 
program. The Transportation Industry represented initially by AMTRAK is 
already engaged in a pilot program with GSN. The Farm Machinery 
Industry, represented by John Deere and others have written EPA with an 
interest in participating. The Trucks and Buses industries have also 
registered interest and most recently, the Retail Industry is opening a 
dialogue with EPA on this program. Additional resources are needed to 
ensure that the technical information the GSN program offers is well 
packaged and effectively delivered to these new industries. Each 
industry that participates in this program is different and must first 
identify a list of suppliers within their supply chains which will 
benefit most from the program. Consequently, for each of these four 
industrial sectors, we could be addressing as many as 100 small to 
medium sized enterprises located across the United States. EPA, in 
partnership with NIST and the State Technical Assistance Providers, 
must make sure that they can deliver the GSN program effectively and 
consistently to all of these suppliers. This will require cross-
training, more extensive communication and planning and the tailoring 
of existing lean and clean tools to these new suppliers. Additional 
resources are needed to ensure that EPA can successfully merge lean 
technologies and environmental technical assistance for delivery to 
this extended supply chain. Moreover, GSN has attracted the attention 
of foreign governments such as Canada and China, arid resources will be 
used to build a foundation for GSN implementation there.
    With additional resources one should expect to see measurable 
results from numerous small to medium sized enterprises and their 
Original Equipment Manufacturers that are aggregated and ready to be 
reported in the near-tern. The results are collected from individual 
suppliers by the Manufacturing Extension Partnership program of NIST 
and aggregated for EPA. The results are measured through the tools that 
the NIST MEPs use in conducting an assessment. This includes, for 
example, the use of a tool called Value Stream Mapping which creates a 
material and information flow map of a product or process. This 
powerful tool allows companies to map the flow of products and 
information from order to cash as well as through the supply chain. 
Kaizen is another tool that is used to monitor and measure continuous 
improvement. These lean manufacturing tools and the environmental 
considerations that are being built into them through GSN are designed 
to measure improvements as the process changes they recommend are 
implemented. Results measured by these systems include pollution 
reductions, crater and energy savings, and company cost savings 
associated with those environmental benefits. These results will 
contribute to achievement of new results-oriented Pollution Prevention 
annual performance measures incorporated into EPA's FY 2003 President's 
Budget Request anal related Goal 5 Strategic Targets in EPA's Strategic 
Plan for 2003-2008.
    Please see the table below for the requested funding levels.


    
    

Q3.  Do the goals, program definitions, and performance measures for 
research programs that EPA has developed and included in its strategic 
plan align with the goals, program definitions, and performance 
measures associated with OMB's PART analysis? How will the 
discrepancies be addressed?

A3. The three EPA research programs that were reviewed through OMB's 
Program Assessment Rating Tool (PART) analysis for the FY 2005 budget 
represent research programs as defined by EPA, each with its own multi-
year research plan. These three programs (Particulate Matter Research, 
Ecosystems Protection Research, and Pollution Prevention Research) 
received PART scores of ``Results Not Demonstrated'' because the Agency 
and OMB were unable to reach agreement regarding appropriate outcome-
oriented goals and measures. For this reason, the PART worksheets for 
these programs note that goals and measures are under development. EPA 
is continuing to work with OMB to develop and identify appropriate 
outcomes for environmental research.
