Environmental Protection: Collaborative EPA-State Effort Needed to
Improve New Performance Partnership System (Chapter Report, 06/21/1999,
GAO/RCED-99-171).

The Environmental Protection Agency (EPA) has had long-standing problems
in building effective partnerships with the states, which have the lead
responsibility for implementing many environmental programs. Among the
key issues affecting the EPA-state relationship have been concerns that
EPA is inconsistent in its oversight across regions, sometimes
micromanages state programs, does not provide enough technical support
for state programs, and often does not adequately consult with states
before making key decisions affecting them. In 1995, EPA established the
National Environmental Performance Partnership System, a key element of
which was EPA's commitment to give states with strong environmental
performance greater flexibility and autonomy in running their
environmental programs. This report (1) identifies the status of grants
and agreements made under the system between EPA and the states, (2)
examines progress by EPA and the states in developing results-oriented
performance measures to be incorporated into system agreements and
grants to the states, (3) examines how EPA oversight may or may not be
changing in states that are participating in the system, and (4)
discusses the extent to which the use of these performance partnership
agreements and grants has yielded expected benefits.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  RCED-99-171
     TITLE:  Environmental Protection: Collaborative EPA-State Effort
	     Needed to Improve New Performance Partnership System
      DATE:  06/21/1999
   SUBJECT:  Grants to states
	     Environmental monitoring
	     Environmental policies
	     Federal/state relations
	     Performance measures
	     State-administered programs
IDENTIFIER:  EPA National Environmental Performance Partnership System
	     Maine
	     Florida
	     Georgia
	     Minnesota
	     Connecticut
	     Oregon
	     EPA Performance Partnership Grant
	     EPA Core Performance Measures

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  This text was extracted from a PDF file.        **
** Delineations within the text indicating chapter titles,      **
** headings, and bullets have not been preserved, and in some   **
** cases heading text has been incorrectly merged into          **
** body text in the adjacent column.  Graphic images have       **
** not been reproduced, but figure captions are included.       **
** Tables are included, but column deliniations have not been   **
** preserved.                                                   **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************

    United States General Accounting Office GAO                Report
    to the Subcommittee on VA, HUD, and Independent Agencies,
    Committee on Appropriations, House of Representatives June 1999
    ENVIRONMENTAL PROTECTION Collaborative EPA-State Effort Needed to
    Improve New Performance Partnership System GAO/RCED-99-171 GAO
    United States General Accounting Office Washington, D.C. 20548
    Resources, Community, and Economic Development Division B-282588
    June 21, 1999 The Honorable James T. Walsh Chairman, Subcommittee
    on VA, HUD, and Independent Agencies Committee on Appropriations
    House of Representatives Dear Mr. Chairman: As requested, we are
    reporting on the Environmental Protection Agency's (EPA) and the
    States' progress in implementing the National Environmental
    Performance Partnership System. As arranged with your office,
    unless you publicly announce its contents earlier, we plan no
    further distribution of this report until 7 days from the date of
    this letter. At that time, we will send copies to the appropriate
    congressional committees; the Honorable Carol Browner,
    Administrator, EPA, and the Honorable Jacob Lew, Director, Office
    of Management and Budget. We will also make copies available to
    others upon request. Please call me at (202) 512-6111 if you or
    your staff have any questions. Major contributors to this report
    are listed in appendix II. Sincerely yours, Peter F. Guerrero
    Director, Environmental Protection Issues Executive Summary
    Purpose       The Environmental Protection Agency (EPA) has had
    long-standing difficulties in establishing effective partnerships
    with the states, which generally have the lead responsibility in
    implementing many environmental programs. Among the key issues
    affecting EPA-state relationships have been concerns that EPA (1)
    is inconsistent in its oversight across regions, (2) sometimes
    micromanages state programs, (3) does not provide sufficient
    technical support for state programs' increasingly complex
    requirements, and (4) often does not adequately consult the states
    before making key decisions affecting them. To address these
    problems and improve the effectiveness of environmental program
    implementation, EPA's Administrator and leaders of state
    environmental programs established the National Environmental
    Performance Partnership System (NEPPS) in May 1995. In signing the
    agreement that established NEPPS, EPA and state leaders said that
    the system is designed to strengthen protection of public health
    and the environment by directing scarce public resources toward
    improving environmental results, allowing states greater
    flexibility to achieve those results, and enhancing accountability
    to the public and taxpayers. A key element of NEPPS was EPA's
    commitment to give states with strong environmental performance
    greater flexibility and autonomy in running their environmental
    programs. Given the expectation among participants that NEPPS
    could deal with many of the issues that have long impeded EPA-
    state relationships, the Chairman, Subcommittee on VA, HUD, and
    Independent Agencies, House Committee on Appropriations, asked GAO
    to examine the progress made by EPA and the states since the 1995
    agreement. Specifically, as agreed with the Chairman's office,
    this report (1) identifies the status of grants and agreements
    made under NEPPS between EPA and participating states, (2)
    examines the progress that EPA and the states have made in
    developing results-oriented performance measures to be
    incorporated into NEPPS agreements and grants to the states, (3)
    examines how EPA oversight may or may not be changing in states
    that are participating in NEPPS, and (4) discusses the extent to
    which the use of these Performance Partnership Agreements and
    Grants has achieved the benefits envisioned for the states and the
    public. Background    Under NEPPS, states may voluntarily enter
    into "Performance Partnership Agreements" with their EPA regional
    offices. While there is considerable flexibility in how the
    agreements may be designed, they typically provide a means for EPA
    and the states to negotiate such matters as (1) which problems
    will receive priority attention within the state programs, Page 2
    GAO/RCED-99-171 Performance Partnership System Executive Summary
    (2) what EPA's and the states' respective roles will be, and (3)
    how the states' progress in achieving clearly defined program
    objectives will be assessed. An important component of the
    Partnership Agreements is the use of a common set of national
    environmental indicators (called "Core Performance Measures") to
    measure the effectiveness and success of states' environmental
    programs. In their efforts to develop these performance measures,
    EPA and state officials have sought to move beyond counting the
    number of actions (such as the number of inspections conducted or
    environmental enforcement actions taken) and increasingly toward
    evaluating the impact of programs on the environment. While NEPPS
    provides the overarching framework for developing Partnership
    Agreements, the Performance Partnership Grants Program, authorized
    by the Congress in April 1996, is used by many states as a major
    tool to implement them. This program allows states to request that
    funds from 2 or more of the 15 eligible categorical grants be
    combined to give governmental entities greater flexibility in
    targeting limited resources to their most pressing environmental
    needs. These grants are also intended to be used to better
    coordinate existing activities across environmental media and to
    develop multimedia programs. While the Partnership Agreements are
    designed to complement the Partnership Grants, states are free to
    negotiate agreements and/or grants or to decline participation in
    NEPPS altogether. Results in Brief    State participation in the
    National Environmental Performance Partnership System grew from 6
    pilot states in its initial year in fiscal year 1996 to 45 states
    by the end of fiscal year 1998. Of that number, 31 states had both
    Performance Partnership Agreements and Performance Partnership
    Grants with EPA in 1998; 12 states had grants only; 2 states had
    agreements only; and 5 states did not participate at all.
    Nationwide, for that year, $217 million of $745 million in state
    environmental program grants was consolidated into Performance
    Partnership Grants-an increase of 28 percent from the previous
    year. EPA and the states agree on the importance of measuring the
    outcomes of environmental activities rather than just the
    activities themselves. However, the development of these measures
    has been impeded by a number of technical challenges, including
    (1) an absence of baseline data against which environmental
    improvements could be measured, (2) the inherent difficulty in
    quantifying certain results, (3) the difficulty of linking program
    activities to environmental results, and (4) the considerable Page
    3                             GAO/RCED-99-171 Performance
    Partnership System Executive Summary resources needed for high-
    quality performance measurement. In addition, EPA and the states
    have had to resolve fundamental disagreements over a number of
    issues, including (1) the degree to which states should be
    permitted to vary from the national core measures and (2) the
    composition of the measures-particularly regarding the degree to
    which preexisting output measures are to be retained as newer
    outcome measures are added. Despite these barriers, EPA and state
    leaders have managed to agree on a set of core measures for fiscal
    year 2000 that are widely regarded by EPA and state officials as
    significantly improved from those negotiated in previous years.
    The initial expectation that participation in NEPPS would be
    accompanied by reduced federal oversight of states has thus far
    been realized to a limited degree. A number of instances were
    identified among the six participating states GAO visited where
    oversight reduction did accompany participation in the system.1
    However, in other cases cited by both state and EPA regional
    officials, (1) decreased oversight could either not be linked
    directly to NEPPS participation or (2) oversight had either
    remained the same or increased. Among the factors cited by these
    officials as complicating reduced EPA oversight were (1) statutory
    and/or regulatory requirements that in some cases prescribe the
    kind of oversight required of states by EPA; (2) reluctance by EPA
    regulators to reduce oversight without the measures in place to
    ensure that environmental quality would not be compromised; (3)
    the inherent difficulty in "letting go" on the part of some
    regulators that have implemented the existing EPA-state oversight
    arrangement for several decades; and (4) EPA's multi-level
    organizational structure, which complicates efforts to identify
    whether all key agency decisionmakers among the agency's
    headquarters and regional offices are in agreement on key
    oversight-related questions. EPA and state participants
    nonetheless cited a number of benefits associated with NEPPS,
    noting in particular that participation (1) provided a means of
    getting buy-in for innovative and/or unique projects, (2) allowed
    states the option to shift resources and funds under the
    Performance Partnership Grants Program, (3) served as a tool to
    divide an often-burdensome workload more efficiently between
    federal and state regulators, and (4) improved communication and
    increased understanding among EPA and state program participants
    about program priorities and other key matters. Yet while
    participants from each state indicated that their participation in
    the voluntary program would probably continue, they also
    consistently expressed the view that the benefits of the program
    1The six states were Connecticut, Florida, Georgia, Maine,
    Minnesota, and Oregon. Page 4
    GAO/RCED-99-171 Performance Partnership System Executive Summary
    should be greater; that the program has yet to achieve its
    potential; and that improvements are needed. The 1995 agreement
    anticipated the appropriateness of such reflection in calling for
    "a joint evaluation system for EPA and the states to review the
    results of their efforts to ensure continuous improvement." GAO
    recommends in this report that such a joint evaluation process be
    initiated and suggests a number of issues to be considered for
    attention during such a process. Principal Findings Growth of
    State           NEPPS was initially tested on a pilot basis in
    fiscal year 1996 with six Participation in NEPPS    participating
    states. This first year was viewed by EPA and the states as a time
    to experiment with the new system and various ways to implement
    it. The number of participating states has increased since that
    time to 45 states in fiscal year 1998, although the extent of
    their participation has varied widely. For example, half the
    states have negotiated both Partnership Agreements and Partnership
    Grants through their lead environmental agencies that cover most
    EPA programs; other states have thus far limited their
    participation to a Partnership Grant, such as one administered by
    their agriculture agency that, for example, addresses only
    pesticide programs. States have also varied considerably in the
    detail and content of their agreements. Senior officials in EPA's
    Office of State and Local Relations explained that the agency has
    not attempted to impose uniformity on the development of
    Partnership Agreements at this early stage of the NEPPS process
    and has, therefore, refrained from issuing guidance on how the
    agreements should be structured. Hence, the agreements vary widely
    in content and emphasis, reflecting individual states' conditions
    and priorities and reflecting the results of negotiations with
    their respective EPA regional offices. While Performance
    Partnership Grants allow eligible states to request that funds
    from two or more categorical grants (such as those authorized
    under the Clean Water Act or those used to implement the Clean Air
    Act) be combined to allow for greater flexibility in targeting
    limited resources to states' most pressing environmental needs,
    the percentage of eligible grant funds consolidated under these
    Grants is less than one-third. For fiscal year 1998, $217 million
    (29 percent) of eligible grants was consolidated among the
    participating states, while $528 million (71 percent) remained as
    categorical grants. This level of consolidation Page 5
    GAO/RCED-99-171 Performance Partnership System Executive Summary
    represents an increase of 28 percent over the $169 million that
    was consolidated the previous year. Progress in Developing
    Both EPA and individual states have a number of efforts under way
    to Results-Oriented Measures    develop effective performance
    measures to better understand whether their programs are achieving
    their intended results. Their collective effort to develop such
    measures for NEPPS has centered on the "Core Performance Measures"
    that have been negotiated between EPA and the Environmental
    Council of the States during the past several years.2 The effort
    has faced a number of technical challenges inherent in developing
    defensible results-oriented measures. The results of activities
    designed to improve water quality, for example, can take years to
    appear, and the capability of many states to monitor a significant
    share of their waters is limited. Moreover, even if environmental
    conditions could be reliably and consistently measured, it may be
    particularly difficult to demonstrate the extent to which a
    government program affected that condition. Officials from Florida
    (a state that has made a significant commitment to measuring
    compliance rates and environmental indicators), for example,
    explained that factors outside their control, such as economic
    activity and weather conditions, make it particularly difficult to
    link program activities with changes in environmental conditions.
    In addition to these technical challenges in developing results-
    oriented measures, the effort has also been challenged by
    disagreements between EPA and the states on a number of issues,
    including (1) the degree to which states should be permitted to
    vary from the national core measures and (2) the composition of
    the measures, particularly regarding the degree to which
    preexisting output measures are to be retained as newer outcome
    measures are added. Overall, however, the states and EPA have made
    progress in meeting these challenges. For example, officials in
    four of the six states whose programs GAO examined have developed
    and implemented their own measures to address their own
    priorities. At the same time, program officials in each of the six
    states have also agreed to report information required for the
    national core measures agreed upon between the Environmental
    Council of the States and EPA. In addition, while they maintain
    that further refinement will still be needed, EPA and state
    officials have agreed on a set of fiscal year 2000 measures for
    use in negotiating EPA-state partnership agreements that, by most
    accounts, are a substantial improvement over those negotiated from
    previous years in that 2The Environmental Council of the States is
    a national nonpartisan, nonprofit association of state and
    territorial environmental commissioners. Page 6
    GAO/RCED-99-171 Performance Partnership System Executive Summary
    they are fewer in number (i.e., better targeted to address key
    goals) and generally more outcome-oriented. Reductions in
    Oversight    Instances of greater state flexibility and reduced
    EPA oversight tended to Attributable to NEPPS      focus on
    reducing the frequency of reporting and, in some cases, the Have
    Thus Far Been         frequency of on-site reviews. Maine
    environmental officials, for example, Modest
    noted that more frequent, and less formal, dialogue between the
    program staff and regional staff had replaced written reports,
    saving time and improving the level of cooperation between EPA and
    state staff. While Maine program officials attributed the
    reductions in part to the assignment by EPA's Boston Regional
    Office of a liaison for each state's delegated programs, they
    credited NEPPS with formalizing or legitimizing the changes.
    Florida program officials identified sizable reporting reductions
    in its waste program as a result of a joint state-EPA effort
    included in the Partnership Agreement. Other instances were cited
    by officials in Georgia and Minnesota. Yet aside from such
    individual instances of streamlining reporting requirements and
    similar tracking efforts, the large majority of the state
    officials GAO contacted generally maintained that participation in
    NEPPS has not yet brought about significant reductions in
    reporting and other oversight activities by EPA staff, nor has it
    resulted in significant opportunities for them to focus on other
    priorities or to shift resources to weaker program areas. EPA
    officials generally acknowledged this point, but provided specific
    reasons why oversight of state programs has not significantly
    decreased as a result of NEPPS-and in some cases has actually
    increased. Some headquarters and regional officials, for example,
    noted that environmental statutes or regulations sometimes
    prescribe the level of oversight required of EPA, leaving little
    room for EPA to scale it back. The officials also pointed to (1)
    audits that identified problems in some state enforcement programs
    (such as the underreporting by states of significant violations
    and precipitous decreases in the number of state enforcement
    actions taken) that they believed called for greater oversight and
    (2) the difficulty in scaling back oversight without measurable
    assurances indicating that state programs experimenting with
    alternative compliance strategies are achieving their desired
    results. At the same time, EPA officials cited a number of
    barriers preventing greater state flexibility that could be more
    readily addressed. For example, senior EPA officials in three of
    the four regional offices that GAO visited acknowledged that
    support for NEPPS within EPA varies. One senior Page 7
    GAO/RCED-99-171 Performance Partnership System Executive Summary
    regional official explained that many regional managers and staff
    are often more comfortable with preexisting ways of doing business
    and are unsure as to how they can accomplish their work in the
    context of the partnership approach under NEPPS. The official also
    said that there may be a need for training EPA regional staff in
    NEPPS implementation. Another senior regional official said that
    some agency staff will only take NEPPS seriously when their reward
    system is more closely tied to their performance in implementing
    the program. Headquarters officials also acknowledged another
    problem cited by many of the state officials GAO contacted-that
    headquarters' guidance, initiatives, and special requests
    sometimes arrive at the regions too late to be used effectively in
    regional-state Partnership Agreement negotiations and that they
    have taken steps to address the problem. Benefits of NEPPS
    Despite their disappointment at the rate of progress in achieving
    greater Participation Cited,      autonomy and greater emphasis on
    state priorities, senior officials and program managers from each
    of the six states in GAO's review agreed that but Full Potential
    Has    NEPPS has provided their programs with worthwhile benefits,
    and that its Yet to Be Realized        potential for achieving a
    more effective partnership between EPA and the states is still
    worth pursuing. Among the examples cited were instances in which
    Partnership Agreements were used to more efficiently divide a
    heavy workload between regional and state staff, and in which
    states were able to take at least limited advantage of the
    flexibility in their Performance Partnership Grant agreements to
    shift resources among their media programs. Overall, however, the
    most frequently cited benefit among both state and EPA regional
    participants was that the two-way negotiation process inherent in
    the program has fostered more frequent and effective communication
    between regional and state participants and improved their overall
    working relationship. At the same time, state officials almost
    unanimously expressed the view that the benefits from their
    investment of time and resources into NEPPS should be greater;
    that the program has yet to achieve its potential; and that
    improvements are needed. Of particular note, almost all of the
    state officials GAO interviewed cited progress in achieving
    reduced oversight and greater autonomy as critical to the future
    success of the program. Also cited was the need to continue
    improving performance measures; addressing the barriers impeding
    greater acceptance of NEPPS among staff within both EPA and state
    agencies; determining how to make greater use of the flexibility
    under Performance Partnership Grants to shift resources and
    funding to address higher priorities; and improving the manner in
    Page 8                             GAO/RCED-99-171 Performance
    Partnership System Executive Summary which headquarters offices
    provide their input into regional-state NEPPS negotiations. These
    concerns pose challenges for the future of NEPPS-challenges that
    were anticipated by the 1995 agreement that launched the program,
    which called for a joint evaluation system for EPA and the states
    to review the results of their efforts to ensure continuous
    improvement. On the basis of the considerable information that can
    be learned from the experiences to date of participating states
    and regional offices, GAO believes that it is now appropriate to
    undertake such a joint evaluation process, with the goals of (1)
    identifying best practices among participating states for dealing
    with the most challenging problems facing the program and (2)
    eventually obtaining agreement on actions that will improve and
    expand the program. EPA officials and representatives of the
    Environmental Council of the States have, in fact, recently agreed
    upon the basic outline of such a joint evaluation process. Further
    progress (including decisions on the specific issues to address
    and a timetable for addressing them) would be important steps in
    expanding both the participation in, and effectiveness of, this
    important program. Recommendations      GAO recommends that the
    Administrator, EPA, work with senior-level state officials to
    initiate a joint evaluation process that (1) seeks agreement on
    the key issues impeding progress in developing a more effective
    National Environmental Performance Partnership System and (2)
    develops mutually agreeable remedies for these issues. Among the
    issues such a process could focus on are these: * Developing a set
    of flexible guidelines, to be used as a tool by state and EPA
    regional NEPPS negotiators, that could help to clarify the
    appropriate performance expectations and other conditions that
    states must meet to achieve reduced oversight in carrying out
    their environmental programs and the type of reduced oversight
    (e.g., reduced frequency of reporting, greater autonomy in setting
    program priorities) that could be achieved. * Identifying what
    additional work is needed to improve the Core Performance Measures
    recently negotiated by EPA and state representatives for fiscal
    year 2000. * Alleviating the resistance among some staff (both
    within EPA offices and among participating state agencies) toward
    implementing NEPPS, through training efforts and other strategies.
* Determining what appropriate steps should be taken by EPA and
    the states to allow for greater use by states of the flexibility
    envisioned under the Page 9                            GAO/RCED-
    99-171 Performance Partnership System Executive Summary
    Performance Partnership Grant system to shift resources and
    funding among their media programs. * Determining how effective
    public participation in the NEPPS process can best be ensured. *
    Developing ways to improve communication among EPA's headquarters
    and regional offices and participating states to ensure that
    states are given clear and timely information on whether key
    elements of their NEPPS-related agreements have the full buy-in of
    key EPA offices. Agency Comments      GAO provided a draft of this
    report for review and comment to EPA and the Environmental Council
    of the States. EPA said that "the Report describes, in a fair and
    balanced manner, the progress EPA and the States have made through
    performance partnerships." EPA also agreed with the report's
    recommendation that agency and state efforts to improve NEPPS
    should include training and other efforts to achieve the cultural
    change necessary for greater success. EPA also commented on GAO's
    recommendation that EPA and state environmental leaders should
    agree on guidelines that would help to clarify, for EPA and state
    negotiators, the appropriate performance expectations that states
    must meet to achieve reduced oversight in carrying out their
    environmental programs and the type of reduced oversight that
    could be achieved. EPA noted that while it agreed with this
    recommendation in principle, EPA and the states believe that each
    state's Performance Partnership Agreement should specify the
    degree of oversight necessary to accommodate the unique
    environmental problems and varied program capabilities of that
    state. GAO agrees that oversight arrangements should be negotiated
    between each state and its corresponding regional office in a
    manner that accounts for that state's unique circumstances, and
    that these arrangements should be specified in the Performance
    Partnership Agreement. GAO continues to believe, however, that
    nonbinding national guidance-to be agreed upon in advance by EPA
    and state environmental leaders-would be useful in introducing
    objective parameters to be considered by regional and state
    negotiators as they seek agreement over this sensitive issue. In
    addition to these comments, EPA provided updated information and
    comments on several other issues (discussed at the end of chs. 3,
    4, and 5). EPA's comments, together with GAO's detailed responses,
    are included in appendix I. Page 10
    GAO/RCED-99-171 Performance Partnership System Executive Summary
    Representatives of the Council provided a number of suggested
    clarifications. They cautioned that since their comments had not
    been reviewed by the Council's membership, they should be viewed
    as informal suggestions to enhance the accuracy and completeness
    of the report. GAO made revisions as appropriate to incorporate
    these comments. Page 11                         GAO/RCED-99-171
    Performance Partnership System Contents Executive Summary
    2 Chapter 1
    14 Introduction                   NEPPS Was Designed to Improve
    the Effectiveness of the                      15 EPA-State Working
    Relationship Objectives, Scope, and Methodology
    19 Chapter 2
    21 Growth of State                Initial Implementation Was
    Devoted to Experimentation                       21 State
    Participation Expanded Rapidly Since Initial
    22 Participation in                  Implementation NEPPS Chapter
    3
    25 EPA and States Have            Developing and Agreeing on Core
    Performance Measures Has                    25 Been Difficult Made
    Progress in               Status of Core Performance Measures
    34 Developing                     Conclusions
    35 Results-Oriented               Agency Comments
    36 Performance Measures Chapter 4
    37 Reductions in EPA's            Initial Expectations Concerning
    EPA Oversight of Participating              37 States' Programs
    Oversight Attributable States and Regional Offices Report Limited
    Oversight Reduction                      39 to NEPPS Have Thus
    Thus Far Directly Attributable to NEPPS Far Been Modest
    Factors Affecting Potential to Reduce Oversight Under NEPPS
    41 Agency Comments
    49 Chapter 5
    51 Benefits of NEPPS              Program Improvements Attributed
    to NEPPS                                    51 Future Prospects
    for Success Depend on Further Progress                     58
    Participation Cited,           Conclusions
    60 but Full Potential Has         Recommendations
    61 Yet to Be Realized             Agency Comments
    62 Page 12                          GAO/RCED-99-171 Performance
    Partnership System Contents Appendixes    Appendix I: Comments
    From the Environmental Protection                      64 Agency
    and Our Evaluation Appendix II: GAO Contacts and Staff
    Acknowledgments                         68 Tables        Table
    1.1: Categories of Environmental Performance
    18 Measurement Table 3.1: Number of Core Performance Measures,
    Fiscal Years                35 1998 Through 2000 Figures
    Figure 2.1: State Participation in Performance Partnership
    22 Agreements and Grants, Fiscal Year 1998 Figure 2.2: State
    Environmental Agencies Participating in                   23
    Performance Partnership Agreements and Grants, Fiscal Year 1998
    Abbreviations EPA          Environmental Protection Agency GAO
    General Accounting Office NEPPS        National Environmental
    Performance Partnership System Page 13
    GAO/RCED-99-171 Performance Partnership System Chapter 1
    Introduction The Environmental Protection Agency (EPA) has had
    long-standing difficulties in establishing effective partnerships
    with the states. Among the key issues affecting EPA-state
    relationships have been concerns that EPA (1) is inconsistent in
    its oversight across regions, (2) sometimes micromanages state
    programs, (3) does not provide sufficient technical support for
    state programs' increasingly complex requirements, and (4) often
    does not adequately consult the states before making key decisions
    affecting them. In an effort to address these problems and improve
    the effectiveness of environmental program implementation, EPA and
    state environmental agencies established the National
    Environmental Performance Partnership System (NEPPS). Under this
    system, strong state programs were to be given more leeway to set
    environmental priorities, design new strategies for addressing
    these priorities, and manage their own programs-allowing EPA to
    concentrate more effort, oversight, and technical assistance on
    weaker programs. A major component of the system is the
    development of Performance Partnership Agreements. These
    agreements are to provide a means for EPA and the states to
    negotiate such matters as (1) which problems will receive priority
    attention within state programs, (2) what EPA's and the states'
    respective roles will be, and (3) how the states' progress in
    achieving clearly defined program objectives will be assessed.
    States may also establish Performance Partnership Grants, which
    allow them to consolidate grants as a way of providing more
    flexibility in managing their environmental grant funds, and to
    cut paperwork and simplify financial management. For example, a
    state that would otherwise have separate water, air, and pesticide
    grants can now combine the funds from some or all of these grants
    into one or more performance partnership grants. Given the
    expectation among participants that NEPPS could deal with many of
    the issues that have long impeded the EPA-state relationship, the
    Chairman, Subcommittee on VA, HUD, and Independent Agencies, House
    Committee on Appropriations, asked us to examine the progress made
    by EPA and the states since the 1995 agreement. Specifically, as
    agreed with the Chairman's office, this report (1) identifies the
    status of grants and agreements made under NEPPS between EPA and
    participating states, (2) examines the progress that EPA and the
    states have made in developing results-oriented performance
    measures to be incorporated into NEPPS agreements and grants to
    the states, (3) examines how EPA oversight may or may not be
    changing in states that are participating in NEPPS, and (4)
    discusses the extent to which the use of these performance
    partnership Page 14                           GAO/RCED-99-171
    Performance Partnership System Chapter 1 Introduction agreements
    and grants has achieved the benefits envisioned for the states and
    the public. NEPPS Was Designed        Most of the nation's
    environmental statutes envision a strong role for the to Improve
    the            states in implementing and managing environmental
    programs. Toward this end, in 1993, a joint State/EPA task force
    recommended that EPA and Effectiveness of the      the states
    adopt a more systematic approach to manage environmental programs
    in a way that allows each level of government to contribute EPA-
    State Working         according to its respective strengths. In
    May 1993, the EPA Administrator Relationship
    established a State/EPA Steering Committee to oversee the
    implementation of the task force's recommendations. Subcommittees
    were established to pursue work on oversight reform, with the goal
    of increasing state participation in EPA decision-making,
    developing national environmental goals and measures, allowing
    flexible funding across programs, and improving communications
    between EPA and states. As a result of these efforts, on May 17,
    1995, the EPA Administrator and the leaders of state environmental
    programs formally agreed to implement a new environmental
    partnership entitled the National Environmental Performance
    Partnership System. This agreement, entitled the Joint Commitment
    to Reform Oversight and Create a National Environmental
    Performance Partnership System, stated that the long-range goal of
    NEPPS was "to provide strong public health and environmental
    protection by developing a system where EPA and the states work
    together for continuous gains in environmental quality and
    productivity." In establishing NEPPS, EPA and the leaders of state
    environmental programs indicated the system is designed to
    strengthen protection of public health and the environment by
    directing scarce resources toward improving environmental results,
    allowing states greater flexibility to achieve those results, and
    enhancing accountability to the public and taxpayers. The seven
    principle components of NEPPS are * increased use of environmental
    goals and indicators in order to measure the effectiveness and
    success of environmental programs; * a new approach for conducting
    assessments of environmental programs, which will include a
    greater reliance on annual environmental and programmatic self-
    assessments conducted by each state and sharing with the public
    information about environmental conditions, goals, priorities, and
    achievements; * the development of environmental performance
    agreements that outline environmental priorities and goals agreed
    to jointly by EPA and the states; Page 15
    GAO/RCED-99-171 Performance Partnership System Chapter 1
    Introduction * a reduction in oversight for those states with
    strong environmental programs, which will enable EPA to focus
    resources on states that need more assistance; * the designation
    of strong state environmental programs as "leadership programs"
    that are afforded minimal oversight; * increased opportunity for
    constructive public involvement in the management of environmental
    programs through a program that encourages regulated entities and
    the general public to review and comment on environmental issues;
    and * the development of a joint system evaluation for EPA and the
    states to review the results of their efforts to ensure continuous
    improvement. As we reported in May 1998, NEPPS is intended to
    strengthen the effectiveness of the nation's environmental
    programs by redefining the federal and state roles to ensure that
    public resources are used efficiently to address the most
    important environmental problems.1 According to EPA, NEPPS is
    based on a shared recognition that continued environmental
    progress can be achieved most effectively by working together as
    partners. Accordingly, the effort is designed to promote joint
    planning and joint priority-setting, which takes into account each
    state's environmental conditions and objectives. A key element of
    this program is EPA's commitment to give states with strong
    environmental performance greater flexibility and autonomy in
    running their environmental programs. To help document this
    capability, a primary objective of the program is the measuring
    and reporting of EPA's and states' progress toward achieving their
    environmental and programmatic goals. Negotiation of
    Under NEPPS, states and their corresponding EPA regional offices
    are Performance Partnership      expected to reach an
    understanding of the state's environmental Agreements and
    conditions and to agree on appropriate environmental goals and
    priorities Performance Partnership      and on program performance
    indicators to measure progress. The results Grants
    of these negotiations are documented in Performance Partnership
    Agreements and/or Performance Partnership Grants. Partnership
    Agreements are comprehensive agreements that are expected to be
    used as the principal mechanism for implementing NEPPS. According
    to EPA, the agreements are derived from joint discussions by EPA
    and the state on their 1Environmental Protection: EPA's and
    States' Efforts to Focus State Enforcement Programs on Results
    (GAO/RCED-98-113, May 27, 1998). Page 16
    GAO/RCED-99-171 Performance Partnership System Chapter 1
    Introduction interests, concerns, choices, and commitments for
    sound environmental performance. While NEPPS provides the
    overarching framework for developing partnership agreements, the
    Performance Partnership Grants Program serves as a major tool to
    implement them. Performance Partnership Grants are intended to
    allow states greater flexibility in deciding how federal grant
    funds can best be spent to achieve their environmental goals.
    Under these grants, which were authorized by the Congress in April
    1996, eligible states and tribes may request that funds from two
    or more categorical grants (such as those authorized under the
    Clean Water Act or those used to implement the Clean Air Act) be
    combined into one or more grants to give governmental agencies
    greater flexibility in targeting limited resources to their most
    pressing environmental needs. These grants are also intended to be
    used to better coordinate existing activities across environmental
    media and to develop multimedia programs. Importantly, state
    participation in NEPPS is voluntary. In particular, while
    Partnership Agreements are designed to complement Partnership
    Grants, states are free to negotiate both agreements and grants or
    to decline participation in NEPPS altogether. Development of
    A key component of the 1995 NEPPS agreement was the commitment by
    EPA Performance Measures Is a    and the Environmental Council of
    the States to identify a common set of Key Component of NEPPS
    national environmental indicators to measure the effectiveness and
    success of states' environmental programs.2 In an effort to
    fulfill this commitment, on August 20, 1997, EPA and the Council
    agreed on a set of "Core Performance Measures" for EPA and states
    to use in measuring progress toward the achievement of
    environmental and program goals. This first set was used to
    measure progress in fiscal year 1998 and, with some minor
    revisions, was used again in fiscal year 1999. In their efforts to
    develop these performance measures, EPA and state officials have
    sought to move beyond counting the number of actions and
    increasingly toward evaluating the impact of programs on the
    environment. Traditionally, performance measures have focused on
    tracking "outputs," such as the number of inspections conducted
    and enforcement actions taken. Such actions are easiest to count,
    and they provide a useful measure of the level of agency activity.
    On the other hand, 2The Environmental Council of the States is a
    national nonpartisan, nonprofit association of state and
    territorial environmental commissioners. Page 17
    GAO/RCED-99-171 Performance Partnership System Chapter 1
    Introduction measuring the actual results a program is intended to
    achieve, such as the degree to which progress is made in achieving
    air or water quality standards, is more difficult but provides
    information on whether the goals of the regulatory program are
    being achieved.3 In order to strike a better balance between
    output measures and measures of program results, EPA and the
    Council developed a tiered approach, shown in table 1.1, to better
    account for program results. As the table indicates, an output
    measure considers numbers of actions taken, demonstrating the
    level of a particular activity or how resources are used. An
    outcome, on the other hand, can measure the results associated
    with a particular policy, such as the percent of facilities in
    environmental compliance. Finally, environmental indicators
    demonstrate whether overall, long-term agency objectives are being
    achieved, such as the trend in the number of bodies of water
    meeting clean water standards. Table 1.1: Categories of
    Environmental Performance Measurement                   Measure
    Characteristic         Examples                Purpose Output
    Numbers of actions Number of penalty Demonstrates level dollars
    collected;      of activity; number of               demonstrates
    how violations              resources are used discovered Outcome
    Environmental or       Tons of pollution       Demonstrates
    programmatic           reduced or              results of specific
    results associated percent of facilities initiatives or with a
    particular      in environmental        policies program or policy
    compliance Environmental indicator         Indicators
    Trend in number of Demonstrates associated with        bodies of
    water         whether overall, overall                meeting
    clean           long-term agency environmental or       water
    standards         objectives are program objectives
    being achieved Note: In its efforts to develop overall performance
    measures for the Government Performance and Results Act of 1993,
    EPA uses slightly different terms: "outputs," "intermediate
    outcomes," and "long-term outcomes." In its guide to implementing
    the act, the Office of Management and Budget distinguishes between
    "output goals" and "outcome goals" and calls on federal agencies
    to measure progress toward both. Other experts in the field of
    government performance measurement labeled the three tiers
    "outputs," "policy or behavioral outcomes," and "program
    outcomes." See for example, Sparrow, Malcolm, "Regulatory
    Agencies, Searching for Performance Measures That Count," and
    Greiner, John M., "Positioning Performance Measurement for the
    Twenty-first Century, "Organization Performance and Measurement in
    the Public Sector, Quorum Books, (1996). Source: Environmental
    Protection: EPA's and States' Efforts to Focus State Enforcement
    Programs on Results (GAO/RCED-98-113, May 27,1998). 3Thus, for
    example, one outcome-oriented core measure in the air program
    tracks overall emission reductions for key pollutants over time.
    Page 18                                       GAO/RCED-99-171
    Performance Partnership System Chapter 1 Introduction NEPPS'
    emphasis on performance measurement also provides a critical link
    to the Congress' intent in passing the Government Performance and
    Results Act of 1993. The Results Act requires agencies to clearly
    define their missions, establish long-term strategic goals (and
    annual goals linked to them), measure their performance against
    the goals they have set, and report this information to the
    Congress. Importantly, rather than focusing on the performance of
    prescribed tasks and processes, the statute emphasizes the need
    for agencies to focus on and achieve measurable program results.
    Objectives, Scope,    Our objectives in this review were to (1)
    identify the status of grants and and Methodology       agreements
    made pursuant to NEPPS between EPA and participating states, (2)
    examine the progress that EPA and the states have made in
    developing results-oriented performance measures to be
    incorporated into NEPPS agreements and grants to the states, (3)
    examine how EPA oversight may or may not be changing in states
    that are participating in NEPPS, and (4) discuss the extent to
    which the use of performance partnership agreements and grants has
    achieved the benefits envisioned for the states and the public.
    For the first objective, we reviewed EPA documents describing the
    overall status of performance partnership grants and agreements
    made between EPA and states. We also interviewed officials from
    EPA's Office of State and Local Relations to obtain the latest
    data and related information on the status of Partnership
    Agreements and Partnership Grants signed by the states and EPA.
    For the remaining objectives, we first contacted EPA (headquarters
    and regional) officials to identify appropriate state
    environmental programs for detailed study. In selecting states, we
    were primarily concerned with the degree of state participation in
    this voluntary program, the length of time they have been
    participating, and the desirability of examining states with
    different experiences and geographical locations. On the basis of
    these criteria, we visited six states that have experience with
    NEPPS for detailed study-Connecticut, Florida, Georgia, Maine,
    Minnesota, and Oregon. In each case, we interviewed officials in
    the states' lead environmental agency. For each state, we first
    discussed the program with officials that have overall
    responsibility for NEPPS. To get insights into the status of NEPPS
    at the program level, we interviewed program managers from each of
    three environmental programs: the Resource Conservation Page 19
    GAO/RCED-99-171 Performance Partnership System Chapter 1
    Introduction and Recovery Act, the Clean Water Act, and the Clean
    Air Act. We also interviewed program officials in the EPA regional
    office with jurisdiction for each state we visited. After these
    visits, we conducted telephone interviews with environmental
    officials from two states that have limited their participation in
    NEPPS-Michigan and Pennsylvania-to determine their views of NEPPS
    and the reasons why they chose not to participate more fully. At
    EPA headquarters, we contacted officials from the various offices
    with NEPPS responsibilities, including the Offices of Air and
    Radiation; Water; Solid Waste and Emergency Response; Enforcement
    and Compliance Assurance; Reinvention; and State and Local
    Relations, to discuss our objectives as well as the results of our
    specific work at the states and EPA regional offices. We also
    gathered information on our objectives through interviews with
    officials from other organizations with an interest in NEPPS,
    including the Association of State and Interstate Water Pollution
    Control Administrators, Association of State and Territorial Solid
    Waste Management Officials, Environmental Council of the States,
    National Academy of Public Administration, National Governors
    Association, and State and Territorial Air Pollution Program
    Administrators. Regarding the second objective, we interviewed
    officials from the Green Mountain Institute for Environmental
    Democracy, which participated in studies of issues related to the
    development and/or use of core performance measures. We conducted
    our work from June 1998 through April 1999 in accordance with
    generally accepted government auditing standards. We provided
    copies of this report to EPA and the Environmental Council of the
    States for their review and comment. EPA's comments and our
    responses are included in appendix I. The Council indicated that
    since its response had been prepared without the benefit of review
    by Council membership, its comments should be viewed not as
    reflecting the Council's positions, but rather as informal
    suggestions to enhance the accuracy and completeness of the
    report. We made revisions as appropriate to incorporate these
    comments. We also provided relevant sections of the draft to
    representatives of the eight states included in our review to
    verify statements attributed to them, and to verify other
    information they provided, and have made revisions as appropriate
    to incorporate their comments. Page 20
    GAO/RCED-99-171 Performance Partnership System Chapter 2 Growth of
    State Participation in NEPPS State participation in the National
    Environmental Performance Partnership System has grown
    significantly in the 4 years since the system was created,
    increasing from 6 pilot states in fiscal year 1996 to 45 states by
    the end of fiscal year 1998.1 However, the extent of participation
    among these 45 states varied considerably: 31 states had both
    performance partnership agreements and grants; 12 states had
    grants only; 2 states had agreements only; and 5 states did not
    participate at all. Moreover, while some states included a full
    range of environmental programs under their agreements, others
    included only one or two programs (such as pesticide or drinking
    water programs). Initial Implementation NEPPS was initially tested
    on a pilot basis in fiscal year 1996 with 6 Was Devoted to
    participating states. This first year was viewed as a time to
    experiment with the new system and various ways to implement it.
    According to a Experimentation                   1996 study of
    five of the six pilot efforts conducted by the Environmental Law
    Institute with funding from EPA,2 although the pilot states shared
    ideas during the process of developing their agreements, the
    states deliberately avoided discussing some of the specifics of
    their approaches so as to ensure diversity. The Environmental Law
    Institute's study focused on whether, and how, the pilot
    performance partnership agreements achieved and measured
    environmental results, how flexibility was exercised under the
    program, and how accountability was ensured. Based on the
    experiences of the pilot states, the Institute's study concluded
    that NEPPS showed great promise for improving the relationship
    between EPA and the states and for improving the administration of
    the environmental statutes. However, the study cited a number of
    issues that would need to be addressed as the program evolved. It
    stated, for example, that while states and EPA had made progress
    toward the goal of increasing the use of environmental indicators
    (measures of overall progress in achieving environmental
    objectives), much remained to be done to develop appropriate
    measures. The study also concluded improvements were needed to (1)
    clarify the relationship between Performance Partnership
    Agreements and Grants, (2) more 1For this report, NEPPS
    participation is defined as participation in Performance
    Partnership Agreements, Performance Partnership Grants, or both.
    2An Independent Review of the State-Federal Environmental
    Partnership Agreements for 1996, Environmental Law Institute,
    (1996). The Performance Partnership Agreement between EPA and the
    sixth state was signed after the Institute completed its review
    and analysis of the other five agreements and thus was not covered
    by this study. The Institute's study did not include a review of
    Performance Partnership Grants since the authority for these
    grants was not provided by the Congress until the middle of fiscal
    year 1996. Page 21                                       GAO/RCED-
    99-171 Performance Partnership System Chapter 2 Growth of State
    Participation in NEPPS effectively communicate EPA's national
    priorities to EPA regions and states in time to impact state and
    EPA regional office negotiations on Performance Partnership
    Agreements, and (3) increase public participation in the program.
    State Participation                   State participation in
    Performance Partnership Agreements and Grants Expanded Rapidly
    expanded rapidly after the first year. In fiscal year 1997, states
    and regional offices were expected by EPA headquarters to build on
    the prior Since Initial                         year's experiences
    and work on areas that needed additional clarification
    Implementation                        or where barriers needed to
    be removed. Participation grew that year to 44 states and to 45
    states in fiscal year 1998. Figure 2.1: State Participation in
    Performance Partnership Agreements
    Agreement only - 2 and Grants, Fiscal Year 1998 Not participating
    - 5 * * *                           *              Agreement and
    grant - 31 Grant only - 12 Source: Prepared by GAO from EPA's
    data. Of the 45 states participating in fiscal year 1998, 31 had
    both Performance Partnership Agreements and Grants, 12 states had
    grants only, 2 states had agreements only, and 5 states did not
    participate at all. (See fig. 2.1.) Since states can have multiple
    Performance Partnership Agreements and Grants, Page 22
    GAO/RCED-99-171 Performance Partnership System Chapter 2 Growth of
    State Participation in NEPPS depending on which state agencies
    handle the different environmental programs, the 45 states
    accounted for a total of 38 agreements and 52 grants. According to
    EPA, states vary in the extent of their participation, with half
    the states participating broadly by negotiating both Performance
    Partnership Agreements and Performance Partnership Grants that
    cover most EPA programs through their state environmental
    agencies, while other states limit their participation by
    negotiating, for example, a partnership grant through their
    agricultural agency that covers pesticide programs. As shown in
    figure 2.2, of those states that participated in NEPPS through
    their lead environmental agencies in fiscal year 1998, 25 had both
    Performance Partnership Agreements and Grants, 4 had grants only,
    and 6 had agreements only. Figure 2.2: State Environmental
    Agencies Participating in Performance
    Grant only - 4 Partnership Agreements and Grants, Fiscal Year 1998
    Agreement only - 6 * * *                Agreement and grant - 25
    Source: Prepared by GAO from EPA's data. States also vary
    considerably in terms of the detail and content of their
    partnership agreements. Senior officials in EPA's Office of State
    and Local Page 23                                     GAO/RCED-99-
    171 Performance Partnership System Chapter 2 Growth of State
    Participation in NEPPS Relations explained that the agency has not
    attempted to impose uniformity on the development of partnership
    agreements at this early stage of the NEPPS process and has,
    therefore, refrained from issuing guidance on how partnership
    agreements should be structured. Hence, the agreements vary widely
    in content and emphasis, reflecting individual state's conditions
    and priorities, and their negotiations with their respective EPA
    regional offices. Most States Have             As discussed in
    chapter 1, Performance Partnership Grants allow eligible
    Performance Partnership      states to request that funds from two
    or more categorical grants (such as Grants, but Few Take Full
    those authorized under the Clean Water Act or those used to
    implement Advantage of the             the Clean Air Act) be
    combined into one or more grants to give greater Flexibility
    Offered          flexibility in targeting limited resources to
    their most pressing environmental needs. Thus far, however, the
    states have consolidated less than one-third of the eligible
    categorical grant funds under partnership grants. Of the eligible
    grants, 29 percent, or $217 million, was consolidated in fiscal
    year 1998, while 71 percent, or $528 million, remained as
    categorical grants. This represents an increase of 28 percent over
    the $169 million that was consolidated the previous year. Page 24
    GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
    States Have Made Progress in Developing Results-Oriented
    Performance Measures Both EPA and individual states have a number
    of efforts underway to develop effective performance measures to
    better understand whether their programs are achieving their
    intended results. Their collective effort to develop such measures
    for NEPPS has centered on the "Core Performance Measures" that
    have been negotiated between EPA and the Environmental Council of
    the States during the past several years. These measures are
    intended to be used in tracking states' progress towards achieving
    the most important goals of the nation's environmental programs.
    In developing the performance measures, EPA and the states have
    retained a number of the traditional output measures they have
    used in the past but have attempted to focus increasingly on
    measuring desired environmental outcomes. However, overcoming a
    number of technical challenges, and reaching agreement on the most
    important environmental outcomes and on the methodologies to
    measure progress toward those outcomes, has been difficult.
    Nevertheless, considerable progress has been made in developing
    and improving the performance measures-as evidenced by agreement
    on a set of measures for fiscal year 2000 that are widely regarded
    as improved measures from previous years. Developing and
    EPA and state officials agree on the importance of measuring the
    outcomes Agreeing on Core        of environmental activities
    rather than just the activities themselves. However, developing
    such measures has faced a number of challenges. Performance
    Outputs, by their nature, are inherently easier to measure,
    report, and Measures Has Been       understand than outcomes and
    environmental results. Compared to output measures, developing
    defensible results-oriented measures has proven to Difficult
    be substantially more difficult. In addition to these technical
    challenges, EPA and the states have differed on what the measures
    should look like (particularly regarding the relative emphasis of
    output versus outcome measures) and on the degree of flexibility
    with which they should be implemented. Technical Challenges    EPA
    and state officials identified several key technical challenges
    that they have had to address in their efforts to focus
    performance measurement on desired results. These include (1) an
    absence of baseline data against which environmental improvements
    could be measured, (2) the inherent difficulty in quantifying
    certain results, (3) the difficulty of linking program activities
    to environmental results, and (4) the considerable resources
    needed for high-quality performance measurement. Page 25
    GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
    States Have Made Progress in Developing Results-Oriented
    Performance Measures Need for Baseline Data to    As noted in our
    May 1998 report on EPA's enforcement program,1 the Measure
    Progress             absence of adequate baseline data for
    comparison is a common problem among many organizations engaged in
    performance measurement, including federal and state agencies.
    Measuring environmental improvements requires a starting point
    against which to measure changes. Without such a baseline, any
    environmental measurement system can only provide a snapshot in
    time; it cannot tell whether conditions are getting better or
    worse. Federal and state agencies have therefore frequently had to
    build entirely new data systems and ways of collecting data
    because the old systems are of limited use in analyzing programs'
    performance. Our 1998 report noted that compliance data are
    especially scarce for small businesses that historically received
    few inspections. Consequently, state programs that are just now
    attempting to measure results have limited data with which to
    compare them. Florida officials, for example, told us that their
    recent environmental reports showing industry-wide compliance
    rates generally have a baseline of 1997 or 1998, because past
    information is unavailable or unreliable. An EPA official
    responsible for NEPPS implementation also noted that the scarcity
    of baseline information by which to measure program improvements
    attributable to NEPPS is a particular challenge and a major
    concern to the agency. Inherent Difficulty in       Generating
    relevant and accurate data is a challenge under the best of
    Quantifying Data             circumstances. Not only do
    appropriate measures need to be defined, methodologies need to be
    established to develop the necessary data. In enforcement
    programs, for example, it is difficult to determine the impact on
    the overall environment from individual inspections conducted or
    enforcement actions taken. In addition, as officials told us
    during our review of enforcement programs, quantifying industry-
    wide compliance rates and other outcomes has been complicated by
    the difficulty of deciding both how to define a compliance rate
    and how to calculate it. As another example, the results of
    activities designed to improve water quality can take years to
    appear, and the capability of many states to monitor a significant
    share of their waters is limited. These challenges have led some
    state officials to note that it may be exceedingly difficult to
    achieve comparability from state to state, both in what is being
    measured and the methodology used in gathering data. In
    particular, a state with more complete data may appear to have
    greater 1Environmental Protection: EPA's and States' Efforts to
    Focus State Enforcement Programs on Results (GAO/RCED-98-113, May
    27, 1998). Page 26                                    GAO/RCED-99-
    171 Performance Partnership System Chapter 3 EPA and States Have
    Made Progress in Developing Results-Oriented Performance Measures
    environmental problems than a state with poor data. Minnesota
    officials, for example, told us that their data base for "impaired
    waters" (waters that do not meet state water quality standards)
    includes waters that have undergone far more rigorous analysis
    than that performed by other states. Consequently, according to
    these officials, Minnesota's impaired waters may appear to be far
    more severe than those of another state that does not subject its
    waters to such rigorous analysis. Similar findings were reached in
    a 1998 study evaluating an effort where six New England state
    environmental management agencies and EPA's Boston office
    collaborated on a menu of environmental indicators intended to
    measure (1) the status and trends of the quality of the New
    England environment and (2) program accomplishments toward
    reaching state and regional environmental goals.2 The findings of
    the study were based on an evaluation of data availability and
    quality for 12 example indicators, which included 6 specific
    performance measures. A key finding of this effort was that the
    level of consistency required for regional indicators is difficult
    to achieve given (1) a lack of clarity in terms of what the
    indicators intend to measure and for what purpose and (2) a lack
    of consistency across states in both the type of data collected
    and methodology used. Challenges in Linking Program    Assuming
    environmental conditions could be reliably and consistently
    Activities to Environmental      measured, it may still be
    difficult to demonstrate the extent to which a Outcomes
    government program affected that condition. As we noted in a 1997
    report on the complexities associated with performance measures,
    "Separating the impact of [a] program from the impact of other
    factors external to the program was cited by government agency
    officials as the most difficult challenge in analyzing and
    reporting government performance."3 Even in the case of the
    Florida Department of Environmental Protection's significant
    commitment to measuring compliance rates and environmental
    indicators, regulators made a conscious decision not to link their
    enforcement programs with trends in environmental indicators or
    outcomes like compliance rates. The regulators explained that the
    causes of these trends are subject to other influences outside
    their control, such as the state of the economy, the weather, and
    other departmental actions 2Green Mountain Institute for
    Environmental Democracy, "Indicator Data Catalog, An Evaluation of
    Data Issues Related to the Development of Core Performance
    Measures and Regional Environmental Indicators," (Nov. 1998).
    3Managing for Results: Analytic Challenges in Measuring
    Performance (GAO/HEHS/GGD-97-138, May 30, 1997). Page 27
    GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
    States Have Made Progress in Developing Results-Oriented
    Performance Measures besides enforcement. The Department's
    consultant agreed, noting, for example, that "If and when the
    scallop population in Tampa Bay is restored to healthy levels,
    Florida's Department of Environmental Protection . . . would be
    hard pressed to prove beyond doubt that their interventions
    actually produced this result, no matter how compelling their
    scientific analyses and explanations."4 Determining causality has
    proven to be particularly difficult among pollution prevention
    programs. According to EPA headquarters officials, EPA and the
    states have not yet been able to determine how to establish a
    cause and effect relationship to measure the impacts on the
    environment from many activities that prevent pollution from
    occurring. Resource Limitations          Another barrier, which
    essentially flows from the others, relates to the significant
    resources and expertise required for identifying and testing
    potential results-oriented performance measures. Once measures are
    in place, gathering and analyzing the data can also be resource-
    intensive and can take years to show environmental improvements.
    In addition, several program officials of the states we visited
    told us that some federal and state data bases will require
    significant improvement in order to track the new information to
    support results measures. A member of the Environmental Council of
    the States' Information Management Workgroup agreed, noting that
    this is an issue EPA and the states still need to address. Two
    states that have developed systems to measure the results of
    selected enforcement efforts found that considerable resources are
    needed to do quality performance measurement. The Florida
    Department of Environmental Protection hired a consultant to
    assist them in developing their new performance measurement system
    and dedicated several of its own staff to this effort. A
    Massachusetts environmental official found that monitoring the
    results of even a single program can require considerable
    resources. The former Deputy Commissioner said that in a pilot
    test of its new Environmental Results Program, the agency had to
    invest a great deal of time and energy to work with the facilities
    and measure the ultimate results, even though the test involved
    only 18 participating companies. Officials from these and other
    states noted that it is difficult to commit resources to the
    development and implementation of new results-oriented performance
    measures while still meeting other program requirements. Results-
    Oriented Measures     As challenging as the exercise may be for
    all programs, we found that Easier to Develop for Some
    developing results-oriented performance measures has been easier
    in the Programs Than Others 4Malcolm Sparrow, "Regulatory
    Agencies, Searching for Performance Measures That Count," (June 9,
    1997). Page 28                                    GAO/RCED-99-171
    Performance Partnership System Chapter 3 EPA and States Have Made
    Progress in Developing Results-Oriented Performance Measures case
    of some programs than others. Air programs, for example, have long
    had a monitoring network in place to measure ambient air quality
    throughout the country. Accordingly, as officials of EPA's Office
    of Air and Radiation told us, the air program has had considerable
    background with results-oriented performance measures, and that
    this experience has limited both the burden of developing specific
    performance measures and the burden on the states of implementing
    these measures. Officials of the states we visited generally
    confirmed this assessment. A senior official in Georgia's
    environmental protection division, for example, told us that
    developing results-oriented measures is easiest for the air
    program, more difficult for the water program, and most difficult
    for the waste program. The Georgia official attributed the
    differences to the extensive historical experience of the air
    program with results-oriented measures, the length of time it
    takes to see measurable results in the water program, and the
    difficulty in identifying suitable measures for the waste program.
    Similar comments were made by a Florida air program official that
    noted that states and EPA have been monitoring air quality for
    some time, have good data, and can show results. Challenges in
    Obtaining        In addition to these technical challenges, EPA
    and states have had to Agreement Between EPA          resolve
    fundamental disagreement over (1) the degree to which states and
    the States on the          should be permitted to vary from the
    national core measures and (2) the Measures
    composition of the measures, particularly regarding the degree to
    which pre-existing output measures are to be retained as newer
    outcome measures are added. Extent to Which States Can     EPA's
    goal to use the performance measures to provide a national picture
    Vary From the Core Measures    of environmental progress
    necessitates a degree of consistency among the states in what is
    being measured. To achieve consistency, the May 1995 NEPPS
    Agreement provides that EPA and the states will ". . .develop a
    limited number of program and multi-media performance measures
    that each state will report so that critical national program data
    is collected." However, recognizing that a set of national
    measures may not necessarily address individual states' priorities
    (or represent what individual states consider to be the best
    measures for their state-specific situations), the agreement
    further provides that states may develop other goals and
    performance indicators that will present a more meaningful picture
    of their state's environmental quality. This apparent need was
    further recognized in the August 1997 joint statement by EPA and
    the Environmental Council of the States, which accompanied the
    release of the measures for fiscal year 1998. The statement
    indicated that where a Page 29
    GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
    States Have Made Progress in Developing Results-Oriented
    Performance Measures particular performance measure does not fit a
    state's situation, that measure may be modified, substituted, or
    eliminated if mutually agreed to by both the state and EPA.
    Deviations could be warranted, for example, where (1) there may
    not be adequate data to report on the measure, (2) alternative
    measures may work better, or (3) there may be higher priorities in
    a state. According to Council officials, in the first year of the
    performance measures, EPA regions were inconsistent in
    implementing the performance measures across the country: some EPA
    staff in regional offices allowed states flexibility in
    implementing performance measures (as intended by the 1997 joint
    statement) while staff in other regions tried to portray the
    national performance measures as mandatory and inflexible. In June
    1998, the president of the Council wrote to the Deputy
    Administrator of EPA, asking that the agency reaffirm its support
    for the flexibility provisions of the joint statement. Noting that
    one of the most challenging aspects of implementing the
    performance measures is balancing the need for uniform national
    measures with the need to accommodate the circumstances of
    individual states, the Deputy Administrator's September 1998
    response reaffirmed that under certain circumstances, EPA regions
    can adjust a measure that is inappropriate for a particular state.
    Updated EPA-Council joint guidance on the use of performance
    measures, issued in April 1999 as an addendum to the 1997 Joint
    Statement along with the release of the fiscal year 2000 measures,
    reiterates EPA's commitment to allow flexible implementation of
    the measures in specific situations and with approval of both the
    state and EPA. 5 States Have Implemented Both Core Performance
    Measures and Their Own Measures As permitted by the 1995 NEPPS
    agreement, four of the six states that we visited have developed
    some performance measures on their own, separately from the
    national core measures. These states use their own measures to
    track priority issues in their respective states and to report
    environmental progress to their state legislatures and the public.
    Florida environmental officials developed their separate measures
    in conjunction with the NEPPS program, and they continue to use
    them because they 5Specifically, the addendum states that a state
    and EPA may jointly agree to deviate from particular performance
    measures where (1) the measure does not apply to a state's or
    region's physical setting or environmental condition; (2) the
    state does not have authority for the program to which the measure
    applies; (3) data for the measure are not available or alternative
    data are more relevant in painting a picture of environmental
    progress; (4) the state and EPA agree that the measure or the work
    associated with it are not a high priority in the state. Page 30
    GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
    States Have Made Progress in Developing Results-Oriented
    Performance Measures believe they are better measures of results
    than the Core Performance Measures. Georgia and Minnesota
    officials developed measures that focus on specific state
    priorities, and Oregon officials developed measures that were
    specifically tailored to the state's strategic plan. While
    environmental program officials in Connecticut and Maine have not
    developed performance measures apart from the core measures, they
    told us that they believed state-specific rather than national
    measures would be more useful to them and more appropriate to
    measure the results of environmental programs in their states.
    Regardless of whether a state developed its own performance
    measures, each of the states we visited also agreed to report on
    the national core measures. Normally, the states did not adopt the
    core performance measures verbatim; they made minor changes where
    appropriate to meet state-specific situations. In each case,
    however, the states' changes to the national Core Performance
    Measures were reviewed and approved by the appropriate EPA
    regional office to ensure that they were compatible with the
    national measures. EPA officials told us that they were aware of
    only one state (New Jersey) that had deviated significantly from
    the national Core Performance Measures, and in that instance, the
    deviation was reviewed and approved by the appropriate EPA
    headquarters program office. Concerns About Applying Core
    Performance Measures to Nonparticipating States NEPPS is a
    voluntary program and not all states have chosen to participate.
    Because core performance measures are a component of NEPPS,
    environmental officials in many states initially presumed that
    they did not apply to nonparticipating states. EPA's intent to use
    performance measure data to present a national environmental
    picture, however, led the agency to request this type of data from
    all states-not just NEPPS participants. Accordingly, in an October
    1998 internal memorandum on EPA implementation of core performance
    measures, the Acting Deputy Administrator stated that: "The
    Regions are responsible for obtaining data on the Core Performance
    Measures from all States (whether or not they have a Performance
    Partnership Agreement with EPA) because these measures are
    intended to paint a picture of environmental and program progress
    across the nation." At the Environmental Council of the States'
    October 1998 annual conference, states expressed concern that
    EPA's policy of seeking to make Page 31
    GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
    States Have Made Progress in Developing Results-Oriented
    Performance Measures performance measures applicable to all states
    is inappropriate and in conflict with the voluntary concept of the
    NEPPS program. Subsequently, joint EPA-Council guidance was issued
    with the fiscal year 2000 performance measures which stated that
    "[Core Performance Measures] as such only apply to states
    participating in NEPPS," but added that "States not participating
    in NEPPS will continue to provide key information needed by EPA
    through State/EPA Agreements, grant work plans, or other operating
    agreements." Relative Emphasis on Outputs    Among federal and
    state officials, there is a broad agreement in principle Vs.
    Outcomes                    on the importance of measuring
    outcomes rather than just outputs. A major concern among state
    officials, however, has been a continued emphasis on output
    measures by EPA. Ironically, many state officials maintain that
    much of EPA's continued emphasis on outputs stems from the
    agency's implementation of the Results Act. The Results Act
    requires agencies to clearly define their missions, establish
    long-term strategic goals, measure their performance against the
    goals they have set, and report this information to the Congress.
    The statute emphasizes the need for agencies to focus on and
    achieve measurable program results, rather than focusing on the
    performance of prescribed tasks and processes. Thus, EPA's goals
    under NEPPS and the Results Act would appear to share the same
    focus on environmental results. However, as we noted in a 1998
    report on the first set of performance measures EPA prepared
    pursuant to the Results Act, the overwhelming share of measures
    were heavily weighted toward numerical targets and other outputs.6
    Broad concern was expressed among the states that we interviewed
    about the impact that EPA's implementation of the Results Act has
    had on core performance measures. To varying degrees, senior level
    and program management officials in five states we visited, and
    EPA program officials in two regions, expressed concern about the
    apparent conflict between the results-oriented performance
    measures being developed under NEPPS and the generally output-
    oriented performance measures EPA has thus far used to report on
    the Results Act. The officials were concerned that EPA's
    implementation of the Results Act is (1) maintaining an emphasis
    on output rather than outcome measures and (2) adding new measures
    on top of existing measures, leading to an overall increase in the
    amount of data states must gather and report. 6Observations on
    EPA's Annual Performance Plan for Fiscal Year 1999 (GAO/RCED-98-
    166R, Apr. 28, 1998). Page 32
    GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
    States Have Made Progress in Developing Results-Oriented
    Performance Measures EPA's enforcement program was illustrative of
    states' concerns about the difficulty in moving toward outcome-
    oriented performance measures. Performance measures from an
    enforcement standpoint have tended to focus heavily on outputs,
    such as the number of inspections conducted, the number of
    significant violations detected, and how violations are handled.
    Senior and program management level officials in half the states
    and EPA regional offices we visited specifically cited the
    relatively heavy focus of EPA's enforcement program on such
    outputs as a barrier to achieving greater progress in developing
    outcome-oriented performance measures. This view echoed those
    expressed by state officials in our May 1998 report on EPA
    enforcement efforts, which relayed concerns among most of the
    state officials interviewed that EPA's Office of Enforcement and
    Compliance Assurance overemphasizes output measures. We
    recommended at that time that EPA ensure that the enforcement-
    related provisions of EPA's Performance Plan, prepared pursuant to
    the Results Act, focus on outcomes in a manner consistent with
    that of the Core Performance Measures developed under NEPPS. In a
    November 1998 response to our enforcement report, EPA emphasized a
    number of initiatives underway, most notably its National
    Performance Measures Strategy, to build in more outcome measures
    in its own enforcement program and to assist states in doing so
    for their programs.7 The Office also acknowledged the need to
    reorient its performance plan increasingly towards outcomes and
    signaled its intent to integrate some outcome measures into the
    fiscal year 2000 core performance measures. The Office's fiscal
    year 2000 measures list seven measures, four of which are
    identified as providing outcome measures. The implementation
    approach for three of the four measures is to work with volunteer
    states to test the measures. In this connection, the Office has
    recently announced the availability of funds for states for
    projects that will improve the design and use of performance
    measures for enforcement and compliance/assistance activities. In
    evaluating project proposals, the Office plans to give priority to
    projects designed to develop outcome measures. Progress has also
    been made in other EPA programs in reorienting the agency's
    Results Act measures toward outcomes. Specifically, we found
    7Among the outcome measures the Office has already implemented
    under this strategy are measures of improvements resulting from
    EPA enforcement actions. Outcome measures currently being
    implemented include (1) the average number of days for significant
    violators to return to compliance or enter enforceable plans or
    agreements and (2) the percentage of significant violators with
    new or recurrent significant violations within 2 years of
    receiving previous enforcement action. Outcome measures targeted
    for implementation in October 1999 include assessments of the
    levels of compliance among selected regulated populations. Page 33
    GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
    States Have Made Progress in Developing Results-Oriented
    Performance Measures that EPA's fiscal year 2000 annual
    Performance Plan, which contains the measures to be used to track
    progress toward achieving its programs' goals, demonstrated some
    progress since the performance plan of the previous year.8 Further
    progress in coming years would help to reduce the disparity
    between the generally output-oriented focus of EPA measures
    prepared pursuant to the Results Act and the efforts by EPA
    regions and states to focus their negotiations under NEPPS
    increasingly on achieving results. Status of Core
    Notwithstanding concerns among state and some regional officials
    about Performance       the potential impact of EPA's
    implementation of the Results Act on their efforts to orient their
    NEPPS-related activities toward outcomes, EPA and the Measures
    Environmental Council of the States have managed to agree on a
    third set of Core Performance Measures for use in fiscal year 2000
    and beyond which, by most accounts, are a significant improvement
    over the 1998 and 1999 measures. As both EPA and Council officials
    have noted, one of the most apparent differences between the new
    measures and those of past years is that the fiscal year 2000
    measures are significantly fewer in number. Specifically, as shown
    in table 3.1, data provided by EPA show that the number of Core
    Performance Measures has been reduced from an initial set of 104
    measures for fiscal year 1998 to 37 measures for fiscal year
    2000.9 8Observations on the Environmental Protection Agency's
    Annual Performance Plan For Fiscal Year 2000 (draft).
    Specifically, we noted that among the improvements in the fiscal
    year 2000 plan are goals and measures of generally better quality,
    and we note some additional efforts to implement outcome measures.
    Overall, however, we found that the plan still focuses heavily on
    output measures. 9Such a sizable reduction reflects the efforts by
    EPA and the Council to focus Core Performance Measures on what
    they agreed are the most important measures. The magnitude of the
    reduction, however, should be interpreted with caution for several
    reasons. First, the figures reflect the temporary deletion of all
    25 measures for the Pollution Prevention and Toxic Substances
    Program. Work is currently underway to develop new measures for
    pollution prevention and toxic substances, which are expected to
    be ready for use in fiscal year 2001. Second, the dropping of a
    measure as a Core Performance Measure does not necessarily mean
    that data will not be gathered in response to that measure.
    Rather, the inclusion or exclusion of the measure as a core
    measure is an expression of its relative importance to the
    national environmental picture. Third, some core measures have
    multiple parts, such as "trends in air quality for each of the six
    criteria air pollutants" (actually six measures) or "trends in
    emissions of toxic air pollutants" (189 hazardous air pollutants
    the Clean Air Act identifies). The discrete data that are
    necessary to report under such measures may be aggregated or
    disaggregated depending on the amount of detail used to measure
    performance. Disaggregating the data increases the number of
    perceived performance measures. Page 34
    GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
    States Have Made Progress in Developing Results-Oriented
    Performance Measures Table 3.1: Number of Core Performance
    Measures, Fiscal Years    Program                           Fiscal
    year 1998          Fiscal year 1999    Fiscal year 2000 1998
    Through 2000                     Air and radiation
    23                 16                  10 Water
    31                 31                  13 Hazardous waste
    17                 17                   7 Pollution prevention and
    toxic substances                                      25
    25                   0 Enforcement and compliance
    8                   8                   7 Totals
    104                     97                  37 Source: EPA's
    Office of State and Local Relations. In addition to reducing the
    number of measures to provide greater focus on what are perceived
    as the most important measures, progress was also made in shifting
    the proportion of fiscal year 2000 measures increasingly toward
    outcomes and environmental indicators. Specifically, according to
    EPA, while about 40 percent of the measures focused on outcomes or
    environmental indicators in fiscal year 1998, about 60 percent of
    the measures focus on outcomes and environmental indicators in
    fiscal year 2000. Moreover, while EPA and Council officials are
    not expected to formally vote on a comprehensive set of new
    measures each year, the fiscal year 2000 measures are to be
    periodically updated as deemed appropriate by EPA and the Council.
    In this connection, EPA program officials, told us that they have
    a number of projects currently under way (in addition to those in
    the enforcement program discussed earlier) that are specifically
    designed to develop additional results-oriented performance
    measures. Finally, EPA and the states have also made progress
    addressing the states' concern that EPA had required additional
    reporting by the states to help the agency meet its data
    requirements under the Results Act. Under the April 1999 Addendum
    to the Joint Statement, co-signed by EPA and the Environmental
    Council of the States, Core Performance Measures and other current
    reporting requirements will be relied upon to satisfy EPA's
    Results Act-related data needs. Conclusions
    There is broad agreement among federal and state officials on the
    importance of measuring the outcomes of environmental activities.
    While considerable progress has been made in developing and
    implementing results-oriented Core Performance Measures, a number
    of challenges Page 35
    GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
    States Have Made Progress in Developing Results-Oriented
    Performance Measures involving technical and policy issues have
    complicated the process. Progress has nonetheless been made in
    developing fiscal year 2000 measures which, by most accounts, are
    a significant improvement over measures used in previous years.
    Continued progress in developing the measures-and the data systems
    needed to support the measures-will be critical to states' and
    EPA's efforts to demonstrate the efficacy of their programs under
    NEPPS. In the past, it has been difficult for states to achieve
    the flexibility they desire without the performance measures in
    place to demonstrate that their environmental goals are being
    achieved, and it will likely continue to be so in the future.
    Agency Comments    Citing our observations that (1) EPA has
    focused on outputs to meet its obligations under the Results Act
    while supporting a transition to outcome-based management under
    NEPPS and (2) these conflicting priorities have led to confusion
    that hinders performance partnerships, EPA said that, to the
    contrary, both the Results Act and NEPPS encourage the development
    of outcome measures and outcome-based management. We acknowledge
    the shared objective of NEPPS and the act in focusing on results.
    The key word, however, is implementation: as we have documented in
    other recent work, the measures EPA has used in its implementation
    of the Results Act have thus far been heavily output-oriented and,
    therefore, convey priorities that are often in conflict with the
    more outcome-oriented measures being employed under NEPPS. We
    acknowledge EPA's ongoing efforts to orient its Results Act-
    related measures increasingly toward outcomes and believe that
    further progress toward this end will help to alleviate this
    problem. In addition, we modified our discussion of this issue to
    reflect the progress made by EPA and the states in addressing the
    states' complaint that EPA had required additional reporting by
    the states to help the agency meet its data requirements under the
    Results Act. The chapter notes that pursuant to the April 1999
    Addendum to the Joint Statement, co-signed by EPA and the
    Environmental Council of the States, Core Performance Measures and
    other current reporting requirements will be relied upon to
    satisfy EPA's Results Act-related data needs. Page 36
    GAO/RCED-99-171 Performance Partnership System Chapter 4
    Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
    Been Modest As originally envisioned, the principle of
    differential oversight was a key element of NEPPS. Under this
    principle, states with stronger environmental programs would be
    accorded reduced oversight and greater autonomy over delegated
    programs, thereby allowing these states greater flexibility to
    manage their programs, and providing EPA the opportunity to shift
    greater attention of its own resources toward weaker programs. An
    important component of the concept of differential oversight was
    that programs eligible for reduced oversight would meet certain
    criteria and that the EPA and states would work together to choose
    a group of measures to use in assessing state performance. In the
    years immediately following the 1995 agreement, EPA and many
    states agreed that a formal system implementing differential
    oversight, whereby the merits of a state program would be
    evaluated based on certain standards or criteria to determine
    whether it qualifies for reduced oversight, would be both
    controversial and difficult to implement. Nonetheless, the
    original concept of reduced EPA oversight in exchange for
    acceptable state environmental performance remains an important
    goal for both EPA and participating states. Among the six states
    we visited, we found instances in which some oversight reduction
    was successfully negotiated between states and their corresponding
    EPA regions. Such instances, however, have thus far been limited
    in both scope and frequency. A number of interrelated factors were
    cited as limiting the reduction of EPA oversight, including (1)
    statutory and/or regulatory requirements that specify state
    reporting requirements and other methods of ensuring state
    accountability to EPA; (2) EPA's reluctance to reduce oversight
    without measurable assurances that environmental goals are still
    being achieved; (3) the inherent difficulty in "letting go" on the
    part of some regulators that have implemented the existing EPA-
    state oversight arrangement for several decades; and (4) the
    challenge faced by EPA of communicating to states through a
    complex, multilevel organization involving both headquarters and
    regional offices. Initial Expectations     The May 1995 joint
    agreement between EPA and the Environmental Council Concerning EPA
    of the States stated that "a differential approach to oversight
    should provide an incentive for state programs to perform well,
    rewarding strong Oversight of             state programs and
    freeing up federal resources to address problems Participating
    States'    where state programs need assistance." It added that
    "after agreement is reached, EPA will focus on program-wide,
    limited after-the-fact reviews Programs Page 37
    GAO/RCED-99-171 Performance Partnership System Chapter 4
    Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
    Been Modest rather than case-by-case intervention and will work
    with states to identify other ways to reduce oversight."
    Accompanying differential oversight was the concept of
    "performance leadership," whereby qualifying programs having a
    record of strong performance would be nationally recognized with
    "leadership" status. In such instances, the leadership programs
    would be afforded minimum allowable oversight based on the belief
    that they "deserve to be treated with deference whenever possible
    and do not need federal oversight on a routine basis." In
    subsequent years, however, both EPA and the states found it
    difficult to implement both a formal differential oversight
    process and to formally designate certain state programs as
    performance leadership programs. One key problem was the inability
    of EPA and the states to agree on criteria to use in making such
    determinations. EPA officials responsible for NEPPS noted that
    because the capacity of a state program can change depending upon
    circumstances, the proper level of oversight should be determined
    on a state-by-state basis by EPA regional managers-not on the
    basis of specific criteria that would be universally applied to
    all states. In addition, as noted by the Environmental Council of
    the State's Executive Director, many state environmental leaders
    expressed concern that formal designations of such programs as
    performance leaders could be interpreted by EPA, state
    legislatures, and the public as a "report card" of good and bad
    performers. Such designations would probably be challenged,
    particularly given the difficulty of developing and applying
    specific criteria to use in making these determinations.
    Nonetheless, the concept of differential oversight, albeit in a
    less structured and visible form, remained an important component
    of state and EPA regional NEPPS negotiations. Officials in the six
    states told us that their early expectations for NEPPS were that
    the program would help them to reduce their oversight workload in
    some well-run program areas and to allow them a stronger focus on
    state priorities and problem areas. Officials in three states
    noted in particular that they believed the NEPPS framework would
    better allow them to identify and address opportunities for
    multimedia projects, rather than continuing to expend time and
    resources only on the traditional, single media air, water, and
    waste programs. Page 38                                GAO/RCED-
    99-171 Performance Partnership System Chapter 4 Reductions in
    EPA's Oversight Attributable to NEPPS Have Thus Far Been Modest
    States and Regional          State officials cited a number of
    instances in which they negotiated some Offices Report
    reduction in regional oversight of their programs. These efforts
    tended to focus on reducing the frequency of reporting, and in
    some cases the Limited Oversight            frequency of
    conducting on-site reviews, in situations where both sides
    Reduction Thus Far           agreed such activities were
    duplicative or otherwise of limited value. However, most state
    program officials indicated that the extent of Directly
    Attributable        reporting required has either remained the
    same or actually increased in to NEPPS                     spite
    of NEPPS, and that few instances were identified where states
    obtained more significant independence in operating their programs
    (e.g., focusing their resources on state priorities). Most
    regional staff we interviewed generally agreed that, to date,
    oversight reduction attributable to NEPPS has been limited.
    Instances of Reduced         Officials in Maine, Florida, Georgia,
    and Minnesota cited specific instances Oversight Cited by States
    in which reporting requirements were scaled back, at least in part
    as a and Regions                  result of their participation in
    NEPPS. Maine environmental officials, for example, noted that more
    frequent dialogue and less formal reporting between the program
    staff and regional staff had replaced written reports, saving time
    and improving the level of cooperation between EPA and state
    staff. While Maine program officials attributed the reductions in
    large part to the assignment by EPA's Boston Regional Office of a
    liaison for each state's delegated programs, they credited NEPPS
    with formalizing or legitimizing the changes. Florida program
    officials identified sizable reporting reductions in its Resources
    Conservation and Recovery Act program as a result of a joint
    state/EPA effort included in the Performance Partnership
    Agreement. The Chief of Florida's Bureau of Water Facilities also
    noted that under the agreement, the state was able to streamline
    oversight of its pretreatment program through reduced reporting
    and by negotiating with the EPA Atlanta office a shifting of
    resources from the conduct of routine annual inspections and
    audits to other priority areas in the program.1 In some cases,
    regional and state officials indicated that oversight had been
    scaled back, but that such efforts could not be tied directly to a
    state's participation in NEPPS. Connecticut officials reported
    that quarterly reporting had been eliminated in recent years for
    some of their air, water and waste programs, but attributed the
    change solely to EPA regional efforts that preceded NEPPS.
    Similarly, program officials in EPA's Boston, Chicago, and Seattle
    offices each cited instances in which quarterly 1Under EPA's
    Pretreatment Program, wastewater treatment plants are charged with
    monitoring and regulating contaminant discharges by industrial
    users into their sewer systems. Page 39
    GAO/RCED-99-171 Performance Partnership System Chapter 4
    Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
    Been Modest reviews and file reviews were eliminated, but
    indicated that such efforts often preceded independently of the
    signing of a NEPPS agreement. Few Instances of
    Notwithstanding the streamlining of reporting requirements and
    similar Significant Oversight    tracking efforts, the large
    majority of the state officials we interviewed Reduction Under
    NEPPS    generally maintained that participation in NEPPS has not
    yet brought about significant reductions in reporting and other
    oversight activities by regional program and audit level staff,
    nor has it resulted in significant opportunities to focus on other
    priorities or shift resources to weaker program areas. Oregon
    officials, for example, explained that their initiatives to focus
    on the state's highest priorities are having difficulty competing
    with their obligations to track and report on the national core
    performance measures and to comply with other EPA reporting
    requirements. Program managers in Connecticut, Florida, Georgia,
    Maine, and Minnesota conveyed similar experiences, indicating that
    the addition of new core measures to preexisting reporting
    requirements had increased their reporting workload, or that they
    are likely to do so in the future. Program managers in three of
    these states indicated they will need to develop the data and
    systems to report on the new measures. Oregon officials also
    pointed to a significant increase in EPA oversight by the regional
    enforcement officials of its air, water, and waste programs. EPA
    Seattle officials told us that the enforcement reviews in Oregon
    were the outcome of nationwide enforcement reviews by both the
    Office of Enforcement and Compliance Assurance and of EPA's Office
    of the Inspector General, which raised concerns about whether and
    how states were bringing enforcement actions against violators.
    Georgia officials also said that oversight of their hazardous
    waste program has increased, noting that regional enforcement
    officials were making regular monthly visits to review program
    records. EPA regional program and enforcement officials generally
    acknowledged that oversight of state programs has not
    significantly decreased as a result of NEPPS, and that in some
    cases, has increased. Officials in the Atlanta and Chicago
    Regional Offices noted in particular that it may have been
    unrealistic to assume, as many states had at the outset of NEPPS,
    that states' participation in the program would necessarily lead
    quickly to reduced EPA oversight. Moreover, regional officials
    point to specific reasons why it has been difficult to scale back
    EPA oversight-and why oversight has actually increased in certain
    instances. Page 40                                GAO/RCED-99-171
    Performance Partnership System Chapter 4 Reductions in EPA's
    Oversight Attributable to NEPPS Have Thus Far Been Modest Factors
    Affecting          We asked both state and regional officials to
    identify what they believed to Potential to Reduce        be the
    most important considerations affecting the extent to which NEPPS
    has provided states with reduced oversight, greater program
    autonomy, Oversight Under            and the flexibility to
    emphasize their highest priorities. There was NEPPS
    considerable consistency on the factors identified by both state
    and EPA officials, although there was some variation on the degree
    to which various factors were emphasized. The key factors include
    (1) statutory and/or regulatory requirements that in some cases
    prescribe the kind of oversight required of states by EPA; (2)
    reluctance by EPA regulators to reduce oversight without
    measurable assurances that environmental protection will not be
    compromised; (3) the inherent difficulty in letting go on the part
    of some regulators that have implemented the existing EPA-state
    oversight arrangement for several decades; and (4) EPA's
    multilevel organizational structure, which complicates efforts to
    identify whether all key decision-makers among the agency's
    headquarters and regional offices are in agreement on key
    oversight-related questions. Statutory or Regulatory    In some
    cases, statutory and/or regulatory requirements may prescribe
    Requirements May Limit     certain types of EPA oversight,
    limiting the extent to which further Options to Reduce
    streamlining can be negotiated. EPA headquarters officials in the
    Office of Oversight                  Air and Radiation noted that
    some of the core performance measures for the air program are
    driven by statutes and thus are non-negotiable. The officials
    noted, for example, that dates by which areas in "non-attainment"
    with air quality standards must come into compliance are driven by
    the Clean Air Act and that EPA accordingly has no flexibility to
    alter them. Similarly, a regional official cited the Clean Water
    Act's requirement under section 305(b) that a Water Quality
    Inventory Report be issued every 2 years. One state requested an
    alternative schedule in which the state would submit its
    information for the report every 5 years for each watershed area.
    EPA denied the request as contradicting the 2-year frequency
    required by the act. In addition, EPA Atlanta and Boston regional
    staff pointed out that they have a responsibility to ensure that
    new regulations, which sometimes pose particular challenges for
    both federal and state regulators, are properly implemented. EPA
    headquarters officials cited as an example their new regulations
    concerning fine particulate matter, which required significant EPA
    action during the middle of the fiscal year. Regional staff said
    that such actions may inevitably require greater EPA oversight and
    more detailed reporting. Officials in EPA's Atlanta Regional
    Office cited another example where, in the middle of the year,
    headquarters Page 41                                GAO/RCED-99-
    171 Performance Partnership System Chapter 4 Reductions in EPA's
    Oversight Attributable to NEPPS Have Thus Far Been Modest
    implemented a new initiative that required the region to ask the
    states to do additional inspections of metal finishing plants that
    went beyond the commitment made by states in their Performance
    Partnership Agreements. State program managers acknowledged that
    statutory and regulatory requirements do in fact sometimes limit
    the potential to reduce EPA oversight. In addition, while
    welcoming the administrative relief and flexibility allowed under
    the Performance Partnership Grant Program, several noted that the
    implementation of these grants is still governed by certain
    statutory and regulatory requirements. For example, the grants are
    still subject to certain grant administrative requirements and
    cost accounting standards applicable to federal grants generally.
    Specifically, while the Partnership Grants do not require the
    detailed accounting required of categorical grants, states must
    still report to EPA on how funds have been spent under the broader
    categories. Furthermore, like other federal grants, the EPA grant
    agreements are supposed to include adequate oversight procedures
    to provide EPA assurance that federal funds are used efficiently
    and effectively. Perhaps more significantly, both state and
    regional officials added that the state programs are still held
    accountable for accomplishing program commitments outlined in
    their work plans and that base program requirements under the
    various statutes must still be met. Such competition for limited
    resources to meet the requirements of individual statutes has, in
    fact, been a long-standing issue that has complicated efforts to
    shift attention and resources to what are perceived as the highest
    environmental priorities. We noted in our 1988 general management
    review of EPA, for example, that the objective of setting risk-
    based priorities across environmental media has been complicated
    by the fact that each statute prescribes certain activities to
    deal with its own medium-specific problems.2 In 1991, we touched
    on the issue again noting, for example, that numerous legislative
    mandates have led to the creation of individual EPA program
    offices that tended to focus solely on reducing pollution within
    the particular environmental medium for which they have
    responsibility, rather than on reducing overall emissions.3 More
    recently, in testifying on efforts by EPA to improve its working
    relationship with the 2Environmental Protection Agency: Protecting
    Human Health and the Environment Through Improved Management
    (GAO/RCED-88-101, Aug. 16, 1988). 3Environmental Protection:
    Meeting Public Expectations With Limited Resources (GAO/RCED-91-
    97, June 18, 1991). Page 42
    GAO/RCED-99-171 Performance Partnership System Chapter 4
    Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
    Been Modest states and to provide them with additional
    flexibility,4 we concluded that as long as environmental laws are
    media-specific and prescriptive and EPA personnel are held
    accountable for meeting the requirements of the laws, it will be
    difficult for the agency to fundamentally change its relationships
    with the states to reduce day-to-day control over program
    activities. EPA Reluctance to Reduce    Program managers and staff
    in all four of the EPA regional offices we Oversight Without
    visited questioned the extent to which the agency can reduce
    oversight Measurable Assurances       without measurable
    assurances that program requirements, and That Environmental
    environmental objectives, will be achieved. The issue has become
    Protection Will Not Be      particularly pronounced in the
    enforcement program, where some states have taken issue with what
    they perceive to be heavy-handed oversight by Compromised
    EPA. Among state officials' complaints are that EPA enforcement
    officials inappropriately hold states accountable for the number
    of enforcement actions (outputs) taken rather than achieving
    better environmental compliance (outcomes). Some states have also
    cited the prospect of EPA taking direct enforcement action in
    states where the lead state environmental agency has primary
    enforcement authority, or of "overfiling" with an EPA action in
    instances where a state enforcement action was determined by EPA
    to be insufficient. State officials have also maintained that such
    a posture is inconsistent with the philosophy under NEPPS that EPA
    should focus its oversight on results and should provide states
    with greater flexibility as to how to achieve those results. EPA
    Seattle regional officials, however, have cited the Office of
    Enforcement and Compliance Assurance's recent reviews and those of
    the Office of Inspector General, which have concluded that (1)
    many states have underreported violations by dischargers of
    pollutant limitations and other environmental requirements and (2)
    the numbers of enforcement actions taken by state enforcement
    officials has declined. These reports, the officials contend,
    raised questions about the ability of states to achieve compliance
    by the regulated community without vigilant federal oversight.
    Moreover, according to the officials, states presently do not have
    the data to support their contentions that environmental
    compliance is still being achieved in cases where their
    enforcement activity has been curtailed. 4Environmental
    Protection: Status of EPA's Initiatives to Create a New
    Partnership With States (GAO/T-RCED-96-87). Page 43
    GAO/RCED-99-171 Performance Partnership System Chapter 4
    Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
    Been Modest State officials told us, both during this review as
    well as during our 1998 review of state enforcement programs,5
    that the absence of measurable results complicates efforts to use
    more flexible approaches-not just because it is harder to get EPA
    approval, but also because it is harder to obtain the confidence
    of the media and the general public. Florida officials, for
    example, told us that the number of penalties assessed, and dollar
    value of penalties collected, under its federally delegated
    programs decreased from 1994 to 1996, and that questions were
    raised as to whether these decreases resulted, at least in part,
    from a greater emphasis on the use of assistance to achieve
    compliance. In fact, newspapers in the state subsequently
    published articles questioning whether the state was letting
    violators continue to pollute without fear of punishment. Florida
    officials told us that their major investment in measuring the
    results of their enforcement and compliance assistance efforts was
    undertaken, in part, to determine whether these concerns were
    well-founded. The Office of Enforcement and Compliance Assurance
    points out that it is addressing the problem through its National
    Performance Measures Strategy and by collaborating on the
    development of enhanced outcome-oriented performance measures with
    a number of states. In addition to helping states develop outcome
    measures, enforcement officials also pointed to recently-issued
    guidance that encourages EPA regional offices to be more flexible
    in considering states' preferences when negotiating regulatory
    priorities.6 Specifically, the guidance calls on regions to
    "develop their priorities in partnership with their states . . ."
    and notes, "States are not required to adopt EPA's national
    priorities . . . This guidance provides flexibility for both
    regions and states to identify and implement their own
    priorities." The guidance further states that EPA is "addressing
    states' concerns about joint planning and priority-setting, work
    sharing, and oversight responsibilities by identifying this as a
    management focus area to be addressed by each region in the fiscal
    year 2000/2001 [memorandum of agreement] process." 5Environmental
    Protection: EPA's and States' Efforts to Focus State Enforcement
    Programs on Results (GAO/RCED-98-113, May 27, 1998). 6EPA Office
    of Enforcement and Compliance Assurance, Final FY 2000/2001 OECA
    Memorandum of Agreement (MOA) Guidance (Apr. 1999). Page 44
    GAO/RCED-99-171 Performance Partnership System Chapter 4
    Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
    Been Modest Resistance to Change at     Our 1997 report on EPA's
    efforts to "reinvent" environmental regulation Lower Levels Within
    Both    observed the widely held view, both within and outside
    EPA, that achieving EPA and Among State         a full commitment
    to reinvention by EPA staff will be difficult and will take
    Agencies                    time.7 The report further identified
    widespread agreement among EPA officials, state officials, and
    others that the agency has a long way to go before reinvention
    becomes an integral part of its staff's everyday activities, and
    cites a senior EPA reinvention official as noting that "many staff
    are comfortable with traditional ways of doing business and
    consider their program-specific job responsibilities as their
    first priority and reinvention projects as secondary." Many of the
    state officials we interviewed contended that comfort level among
    some EPA staff with the preexisting oversight arrangement-which
    has generally been in place for many years-helps to explain the
    reluctance by many of them to provide states with greater
    flexibility and reduced oversight. Program officials in five of
    the six states provided examples where they believed that regional
    program staff (tasked with the day-to-day implementation of
    specific programs) asked for information that was not included in
    the Partnership Agreement or that they had previously agreed with
    the region to drop. Minnesota officials said that EPA regional
    waste officials were asking for predictive or target numbers (such
    as the number of inspections the state intends to pursue during
    the coming year)-information, they said, that was not required nor
    included in their Partnership Agreement. Similarly, Georgia
    program officials said that EPA enforcement officials requested
    additional information after their Partnership Agreement had been
    negotiated and was ready to be signed. Georgia's Assistant
    Director and the Atlanta Deputy Regional Administrator,
    recognizing that the difficulty was due in part to different
    targets and schedules for enforcement and the media programs, set
    up an enforcement planning work group consisting of state and
    regional representatives from enforcement and the media programs
    to study and resolve the problem so that they could avoid last
    minute changes in the future. Other state officials told us that
    EPA has recently requested information related to the Results Act
    which, they believed, was outside the scope of their agreements.
    Several state officials commented that an openness toward seeking
    ways to reduce such information requests appears to be greater
    among senior EPA regional managers than among lower-level staff.
    7Environmental Protection: Challenges Facing EPA's Efforts to
    Reinvent Environmental Regulation (GAO/RCED-97-155, July 2, 1997).
    Page 45                                    GAO/RCED-99-171
    Performance Partnership System Chapter 4 Reductions in EPA's
    Oversight Attributable to NEPPS Have Thus Far Been Modest It is
    possible that what state officials may view as an "resistance to
    change" could be regarded by EPA staff as a well-founded concern
    that program requirements be implemented properly and in
    accordance with laws and regulations. However, officials in three
    of the four EPA regions we visited nonetheless acknowledged that
    support for NEPPS within EPA varies. One senior regional official
    said that managers and staff are often more comfortable with the
    preexisting way of doing business and are unsure as to how they
    can accomplish their work in the context of the partnership
    approach under NEPPS. He voiced the opinion that there may be a
    need for training in NEPPS implementation among regional staff.
    Another senior regional official said that some staff will only
    take NEPPS seriously when their salaries are tied to their
    performance in implementing the program. By the same token, our
    interviews with senior state officials suggest that cultural
    change is also needed at the state level if NEPPS is to achieve
    its full potential. Specifically, several state officials said
    that state program managers may not always be well-versed in
    recognizing opportunities that would allow them to exercise their
    responsibilities with greater flexibility. Some of them indicated
    that there is resistance to NEPPS at the state program manager and
    staff level because of the perceived threat to their programs. In
    one state, in order to get the program directors' support for
    participating in NEPPS, senior management made a commitment not to
    make any large-scale shift of funds among or between programs.
    Some regional staff and managers also commented that states have
    not taken advantage of opportunities to seek more flexibility
    under NEPPS, noting in particular that none of the states in their
    regions attempted to move significant amounts of funds among
    programs or across media lines. Challenges in               EPA's
    organizational structure poses additional challenges in
    negotiating Communicating               agreements that have the
    full buy-in of all key EPA decision-makers. Requirements Through a
    Headquarters interaction with the states is generally conducted
    indirectly Multi-Level EPA             through the regional
    offices. National Program Managers set national Organizational
    Structure    strategic direction, and core program requirements
    and priorities, for each of their environmental programs. The
    managers establish overall national goals for their respective
    programs based on a variety of factors, including the underlying
    statutory mandates, congressional directives,
    administration/administrator priorities, and their own view of
    programs and policies that their programs should focus upon. The
    managers also must develop an accountability system to ensure
    program delivery by EPA's regions. The regional offices consult
    with managers in determining national priorities and communicate
    these priorities to the states. As such, Page 46
    GAO/RCED-99-171 Performance Partnership System Chapter 4
    Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
    Been Modest the regional offices serve as the key EPA focal point
    in negotiating with states on program priorities and oversight
    arrangements to be reflected in NEPPS agreements. Importantly, the
    states generally have little direct communication with the
    managers. Thus, for example, if states wish to deviate from a
    national core performance measure or priority, it is the regions
    that consult with the managers. Buy-in by Key Decision Makers
    As a consequence of this structure, according to the majority of
    state and Mixed Messages Confuse       program managers we
    interviewed, it is not always clear that a Partnership States
    Agreement between the state and the region has the full buy-in of
    EPA's key headquarters managers. A senior official with Florida's
    Department of Environmental Protection cited the example of the
    state's "Joint Compliance and Enforcement Plan," negotiated under
    the state's 1998-1999 Performance Partnership Agreement. Under the
    plan, state and regional officials enter into a process that seeks
    agreement, on the basis of industry compliance data, on what the
    state's most important compliance problems are and which methods
    (e.g., enforcement action, technical assistance) are most
    appropriate to address them. The official said that while the
    state has already invested significant time and effort into the
    plan, and has had expressions of strong support from EPA's Atlanta
    Regional Office, it does not know the extent to which EPA's
    headquarters Office of Enforcement and Compliance Assurance
    supports the effort or whether that office will ultimately give
    its approval. Similar observations were made by other states'
    officials, who indicated that provisions were sometimes added at
    the request of EPA headquarters to Partnership Agreements after
    they were negotiated. The timing of headquarters guidance and
    special requests for input into Partnership Agreements was cited
    by some state and regional officials as a key factor: final
    headquarters guidance, or specific requests in some cases, often
    come too late to be included in regional and state negotiations,
    causing the need for some agreements to be renegotiated.8 State
    officials also indicated that some headquarters requirements are
    negotiated separately from the overall Partnership Agreement
    negotiations. Officials with the Minnesota Pollution Control
    Agency told us that after successfully negotiating its agreement
    with regional program officials, the Office of Enforcement and
    Compliance Assurance requested separate measures and a separate
    section apart from the media programs 8In response to a 1997
    survey by the Environmental Council of the States, participating
    states commented that headquarters guidance should be finalized by
    February of each year so that states and regions can meet in March
    to set joint priorities and begin the Performance Partnership
    Agreement process for the following fiscal year. Page 47
    GAO/RCED-99-171 Performance Partnership System Chapter 4
    Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
    Been Modest in the Partnership Agreement. In the opinion of the
    state officials, this process illustrated the difficulty in
    getting all headquarters interests incorporated into the agreement
    in a timely fashion. Officials in several other states cited
    similar circumstances where enforcement provisions had to be
    negotiated outside the scope of the Performance Partnership
    Agreement, making it difficult to develop the kind of integrated
    environmental program NEPPS is intended to encourage. EPA
    Officials Acknowledge    Officials in the four regional offices we
    visited told us that sometimes Need for Clearer and More    there
    are inconsistencies between headquarters and regional offices,
    Consistent Communication     which complicates the message the
    agency sends to the states. Boston regional officials cited one
    instance in which Maine and Connecticut had proposed to
    consolidate funds for their wetlands programs (1 of the 15
    eligible programs) under a performance partnership grant and were
    initially told by the regional office that the arrangement would
    be acceptable. However, EPA's headquarters Water Office
    subsequently objected to allowing all funds to be shifted from a
    categorical grant to a Performance Partnership Grant on the basis
    that a portion of the funds were supposed to be used in a
    competitive bid process for nonprofit organizations (and other
    eligible parties) to propose special projects. According to state
    and EPA Boston regional program managers, EPA's Boston Regional
    Office resolved the resulting confusion by brokering an agreement
    to allow for some funding from each of the New England states'
    wetlands grant programs to be set aside for special regional
    wetlands pilots. Many EPA regional officials said that
    headquarters officials sometimes view NEPPS negotiations as a
    regional-state matter, and that headquarters offices do not view
    themselves as "signatories" to the process. The officials noted
    that it is only when there is a significant deviation on the part
    of the state from a national priority that headquarters may become
    involved with decisions related to NEPPS agreements. Most of the
    headquarters managers that we interviewed acknowledged that EPA
    headquarters input into the NEPPS negotiation and agreements
    process is primarily left for the regions, to convey to the
    states, with headquarters primarily engaged in setting the
    national priorities and issuing national program guidance. These
    headquarters managers acknowledged that headquarters input into
    the NEPPS process can be improved, noting in particular that
    headquarters guidance, initiatives, and special requests sometimes
    arrive at the regions too late to be useful. In April 1999,
    headquarters managers issued 2-year program guidance to help Page
    48                                GAO/RCED-99-171 Performance
    Partnership System Chapter 4 Reductions in EPA's Oversight
    Attributable to NEPPS Have Thus Far Been Modest address some of
    the problems related to untimely requests. EPA intends that this
    guidance will allow the regions and states to include national
    program priorities earlier in the negotiating process for
    Performance Partnership Agreements. At the same time, however, the
    managers said certain circumstances that could affect a signed
    agreement, such as those dealing with new regulations, are
    sometimes out of their control. Agency Comments    EPA provided
    updated information about the concern that headquarters program
    guidance often arrived too late to be of use in Performance
    Partnership Agreement negotiations between states and their EPA
    regional offices. Specifically, the agency noted that in April
    1999, its headquarters National Program Managers issued 2-year
    program guidance to the regional offices simultaneously and on
    schedule, so that the information would be available prior to
    Performance Partnership Agreement negotiations. EPA said that the
    new procedure has been well received by the regional offices, and
    that the introduction of 2-year guidance will allow regional
    offices and states to extend their planning horizon without fear
    that the priorities of the National Program Managers will change
    dramatically on an annual basis. We have amended this chapter to
    reflect this progress. At the same time, the chapter still conveys
    agency officials' views that the guidance will not necessarily
    prevent other circumstances, which are out of EPA's control, from
    necessitating the reopening of an agreement. EPA also cautioned
    that the report should more clearly distinguish between the terms
    burden reduction and differential oversight. Burden reduction,
    according to EPA, applies to activities, particularly information
    exchanges, that both EPA and a state agree are unnecessary,
    duplicative, or inefficient. In such cases, EPA believes that all
    state programs should benefit from burden reduction. The term
    differential oversight, according to EPA, means that oversight may
    vary depending on how effectively a state program meets
    performance expectations. The EPA comment draws a clear
    distinction between issues associated with reporting burdens and
    other issues that are more appropriately viewed as related to
    EPA's oversight of state environmental programs. We acknowledge
    that there are circumstances, separate and apart from EPA
    oversight, in which EPA and a state collaboratively pursue
    strategies to reduce reporting requirements that they both agree
    are unnecessary, duplicative, or inefficient. However, the
    distinction between this activity and oversight is not always so
    clear. Specifically, in cases where states and EPA have disagreed
    on the need for data not required by statute and viewed by states
    as extraneous, and EPA Page 49
    GAO/RCED-99-171 Performance Partnership System Chapter 4
    Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
    Been Modest has continued to require reporting of such data,
    states have often characterized the issue as, in their view, a
    questionable exercise of EPA oversight. Page 50
    GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
    of NEPPS Participation Cited, but Full Potential Has Yet to Be
    Realized State participants' expectations for reduced EPA
    oversight and greater program flexibility-major anticipated
    benefits at the outset of NEPPS in 1995-have thus far met with
    some disappointment. Yet while these participants expressed
    disappointment at the rate of progress in achieving greater
    autonomy and greater emphasis on state priorities, senior
    officials and program managers from each of the six states that we
    reviewed agreed that NEPPS has provided their programs with
    worthwhile benefits. Among the benefits most frequently cited were
    that NEPPS (1) provided a means of getting buy-in for innovative
    and/or unique projects, (2) allowed states the option to shift
    resources and funds under the Performance Partnership Grants
    Program, (3) served as a tool to divide a burdensome workload more
    efficiently between federal and state regulators, and (4) improved
    communication and increased understanding among EPA and state
    program participants about each other's program priorities and
    other key matters. Officials in each of the four regions visited
    substantially agreed with many of the benefits of NEPPS
    participation cited by state officials. Yet while participants
    from each state indicated that their participation in the
    voluntary program would probably continue, they also shared a
    consistent opinion that the benefits of the program should be
    greater, that the program has yet to achieve its potential, and
    that improvements are needed. To some extent, such an outcome
    should not be surprising, given that the program (1) has been in
    place for just a few years and (2) began as an experiment in which
    participants were encouraged to try different tools and
    techniques. Yet these early years of the program have also
    provided a wealth of experiences as to what has worked well, what
    has not worked, and how the program can be improved. The 1995
    agreement anticipated the appropriateness of such reflection in
    calling for a joint evaluation system for EPA and the states to
    review the results of their efforts to ensure continuous
    improvement. On the basis of our work, we believe that it is now
    appropriate to begin such a joint evaluation process. Program
    State officials in each of the six states we visited identified a
    number of Improvements           benefits to their air, water, and
    waste programs, but frequently spoke of some benefits, such as the
    ability to move funds toward the state's highest Attributed to
    NEPPS    priorities, as potential future benefits rather than as
    benefits already realized. EPA regional staff acknowledged many of
    the benefits identified by state participants, but were often
    cautious in stating that additional flexibility could be exercised
    only so long as states continue to meet the statutory and
    regulatory requirements associated with their base Page 51
    GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
    of NEPPS Participation Cited, but Full Potential Has Yet to Be
    Realized programs. Benefits identified related to the flexibility
    to work on innovative and special projects; to use resources and
    gain administrative efficiencies through the consolidated
    environmental grant; to more efficiently divide the workload among
    EPA and state regulators; and as a means of improving public
    outreach and involvement in environmental policies and programs.
    The additional benefit most frequently cited by state officials is
    perhaps the most intangible one-that it helped to encourage a more
    systematic and effective communication between EPA and state
    officials on key issues and priorities, leading to increased
    mutual understanding and improved relations. Although many of
    these officials acknowledged that this progress has not yet
    resulted in the more equal partnership with EPA to the extent
    hoped for, the collaboration and negotiation fostered by the
    process was viewed as a definite step in the right direction.
    NEPPS Provides a Means      The majority of EPA regional and state
    officials we contacted cited the of Getting Buy-in for
    ability to work on, and get buy-in for, innovative and/or unique
    projects Innovative And/or Unique    (such as those dealing with
    cross-cutting issues or multimedia projects) as Projects
    a tangible benefit under NEPPS. Among the examples cited was a
    Quality Assessment Management Plan included in Florida's fiscal
    year 1999 Performance Partnership Agreement, signed between the
    state's Department of Environmental Protection and EPA's Atlanta
    Regional Office. Once fully developed and implemented, the plan is
    expected to provide the state with the ability to identify and
    improve the quality of data provided by private laboratories. The
    Florida project director spearheading the effort on behalf of the
    state said that elevating the project as a priority in the
    Partnership Agreement legitimized the concept and gained the
    support of key EPA and state decision makers. The prototype or
    model of the plan has been completed and submitted to state and
    regional officials with the expectation that the project staff
    will next move on to issues related to implementation. According
    to the project director, the Partnership Agreement-as a document
    signed by the senior officials at both the state and federal
    level-was crucial in conveying top management buy-in. The project
    director observed that the Agreement, in effect, provided the
    "impetus to innovate" whereby state and regional leadership
    formally endorsed a new way of doing business. Environmental
    officials in Minnesota recently reorganized the state's pollution
    control agency to eliminate its media-specific structure. The new
    organization has three geographic divisions to handle most
    environmental Page 52                                  GAO/RCED-
    99-171 Performance Partnership System Chapter 5 Benefits of NEPPS
    Participation Cited, but Full Potential Has Yet to Be Realized
    issues and two divisions to handle environmental planning and
    outcomes. The reorganization was undertaken because they believed
    that an integrated approach to environmental management was needed
    and because many problems transcend media boundaries. Agency
    officials noted that the Performance Partnership Agreement between
    the state and EPA's Chicago Regional Office was key to
    establishing a new working relationship with EPA and to
    Minnesota's efforts to find a better way to plan and carry out
    their work. They added that the Partnership Agreement provides the
    state the flexibility to go beyond reporting on media-based
    program outputs towards linking, tracking, and measuring agency
    activities with actual environmental results. Among other examples
    cited, a program manager in EPA's Atlanta Regional Office pointed
    to North Carolina's effort to use its Performance Partnership
    Agreement to pursue a multimedia inspection project for metal
    finishing plants. The inspections are conducted jointly from an
    air, water, and waste perspective so that each media program does
    not have to do its own separate inspection. The Partnership
    Agreement provided program managers in the state environmental
    agency with a recognized vehicle to propose and implement the
    inspection initiative to share resources across media lines by
    getting a formal buy-in from state and EPA officials through a
    signed agreement. Flexibility to Shift    As noted earlier in this
    report, Performance Partnership Grants allow Resources and Funds
    states the opportunity to combine individual categorical grant
    funds into a Under NEPPS Grant       consolidated grant. Once
    included in the consolidated grant, the funds Agreements
    essentially lose their category-specific identity and can be used
    with considerably greater flexibility. Environmental agencies
    within four of the six states included in our review (Maine,
    Connecticut, Georgia, and Minnesota) have Partnership Grants with
    their corresponding EPA regional offices. Importantly, officials
    in these states told us that they have not been able to take
    greater advantage of the ability to shift funds, primarily because
    the programs covered by the Partnership Grant each have their own
    base program requirements that must be funded.1 However, several
    of the officials told us that the flexibility allowed under a
    Partnership Grant to move funds where they are most needed remains
    an important potential benefit of the program. For example, a
    Georgia official said that they hoped to shift 1At a workshop
    sponsored in July 1997 by the Environmental Council of the States,
    two states reported plans to shift between 5 and 15 percent of the
    funds under their Performance Partnership Grants to address
    priorities such as pollution prevention. Also, one state in a 1997
    Council survey of the Performance Partnership Agreement process
    reported setting aside 5 percent of its funds from water grants to
    address wetlands lakes and a new Performance Partnership Grant
    coordinator position. Page 53
    GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
    of NEPPS Participation Cited, but Full Potential Has Yet to Be
    Realized funds sometime in the future to address nonpoint sources
    of water pollution and air quality in metropolitan Atlanta. This
    official added that if an emergency were to arise, the Partnership
    Grant would allow the state to move funds and staff quickly from
    various programs to address the problem. A grant official with
    EPA's Boston office noted that prior to the Partnership Grant
    program, states in the region often complained about their
    inability to shift funds from programs that had excess funds to
    other programs that were short of funds. He noted that such
    complaints have declined with the inception of the program.
    Program officials in all four of the case study states having
    Partnership Grants also cited administrative efficiencies from the
    ability to consolidate their categorical grants. The officials
    noted that the grants have allowed states to condense individual
    work plans into a single consolidated work plan, and states have
    gained additional flexibility in the way they account for staff
    time. State environmental agency officials noted that they were
    able to reduce the number of grant applications, budget documents,
    and work plans required. Some added that they gained
    administrative relief from not having to track staff time and
    charges on a detailed, grant-by-grant basis. A Maine official, for
    example, noted that under the traditional categorical grant
    process, staff positions funded by multiple categorical grants
    required controls to be in place to ensure that employees charge
    their time to specific grants and budget categories. The
    Partnership Grants provide the flexibility to accomplish necessary
    work without worrying about which tasks are funded by which
    categorical grants. EPA regions' responses to this increased
    flexibility have been mixed. Regional program managers in the four
    regions visited expressed concern about the flexibility of the
    NEPPS agreement and grants process and said that there is a need
    to retain or develop new state reporting requirements if EPA is to
    retain proper program oversight. These program managers commented
    that eliminating reporting requirements results in EPA losing its
    ability to hold states accountable and argued for states to
    provide predictive annual targets as to what they plan to
    accomplish and to develop short-term or interim measures for
    reporting states' progress towards measuring environmental
    results. Several managers said that it is important for the states
    to prove that work is actually being done and cited the
    consolidation of grant funds under a Partnership Grant as an
    example where EPA loses a level of control. Other regional program
    managers, however, were more optimistic and comfortable with the
    fact that states provide year-end reports on what they have done.
    Page 54                                  GAO/RCED-99-171
    Performance Partnership System Chapter 5 Benefits of NEPPS
    Participation Cited, but Full Potential Has Yet to Be Realized A
    Tool to Divide a     In 1997, we reported on EPA's and states'
    efforts to improve their Burdensome Workload    management of
    Superfund site cleanups,2 and cited innovative efforts in
    Efficiently Between    Minnesota and Washington where state and
    regional officials experienced Federal and State      substantial
    efficiencies through work-sharing agreements. In Washington,
    Regulators             state and EPA officials reported that under
    a formal written agreement signed by officials in EPA's Seattle
    office and the state's Department of Ecology, responsibility was
    formally divided for cleaning up the state's National Priority
    List sites between the two agencies. Both EPA and state officials
    reported that the formal, clearly articulated division of
    responsibility between the two parties helped to reduce both the
    acrimony and the duplication of effort that characterized their
    past relationship. The state official reported a strong consensus
    among the staff that the changes contributed to a significant
    reduction in the number of staff resources needed to oversee
    cleanups at NPL sites. Minnesota officials and Superfund officials
    with EPA's Chicago office reported similar success with such a
    work-sharing agreement. State and EPA regional officials cited
    similar benefits of Partnership Agreements, as formal documents
    that clearly articulate the obligations of both parties to the
    agreement. State officials noted that in some instances in the
    past, communication seemed to be one of EPA conveying its
    expectations of the state, rather than the two-way communication
    embodied in many Partnership Agreements. Even where the concept of
    dividing responsibilities and identifying work-sharing
    opportunities has been used, state officials indicated that a
    formal Partnership Agreement brings a commitment and focus to the
    need to share scarce resources and to formalize stated
    commitments. Program managers in several of the regions and states
    we visited cited a number of examples that illustrated the
    benefits associated with the formal division of labor memorialized
    in a Partnership Agreement. Connecticut's NEPPS coordinator, for
    example, said that the Connecticut Department of Environmental
    Protection negotiated with EPA's Boston Regional Office to pick up
    some of the state's training work load, because EPA could provide
    joint training for all the New England States at a lower cost than
    would be the case if each state provided training individually.
    The Connecticut Partnership Agreement specifically states that EPA
    agrees to assist with training in several areas, such as measuring
    and documenting the success of the state's compliance assistance
    and enforcement activities. The state's fiscal year 1999 agreement
    also 2Superfund: Stronger EPA-State Relationship Can Improve
    Cleanups and Reduce Costs (GAO/RCED-97-77, Apr. 24, 1997). Page 55
    GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
    of NEPPS Participation Cited, but Full Potential Has Yet to Be
    Realized documents coordination with the Boston office, indicating
    that the region agrees to work with the state on helping to reduce
    the state's reporting burden. According to the agreement, the
    region was to assume some of the state's inspection workload or
    streamline inspection requirements in order to free up state staff
    resources for compliance assistance activities. Georgia's
    Partnership Agreement includes provisions for EPA's Atlanta office
    to assist the state in training, enforcement, and inspection
    activities generally on an "as requested" basis. For example, the
    region provided some expertise to the state and committed to
    dedicating EPA resources to the training of compliance officers
    within the state. Oregon's Partnership Agreement was similarly
    used to address unmet needs in the state's water program.
    Officials with EPA's Seattle Regional Office and the state's
    Department of Environmental Quality agreed that the state's
    program to identify and remediate heavily polluted waters was
    understaffed and underfunded. Under the agreement, the regional
    office agreed to provide the state with two staff to assist in the
    program. Opportunity to Improve    A key intended benefit and one
    of the seven principal components of NEPPS Public Outreach and
    in its May 1995 joint agreement is the opportunity to share
    information Involvement               with the public on state
    environmental conditions, objectives, and performance. Officials
    with the Environmental Council of the States commented that public
    participation is a strong point of the NEPPS program-something
    that rarely occurred under the formal traditional system where
    public comment was generally sought on specific facilities or
    sites only. At the time of the 1995 agreement, some states had
    begun to share such information through their annual state of the
    environment reports. The NEPPS process, however, offered greater
    opportunities for constructive public involvement. EPA and state
    officials told us that increased public participation and
    involvement remains a principle benefit of the EPA-state NEPPS
    process, but its full potential is largely unmet. State officials
    have found that public interest and input into the NEPPS process
    has varied but that, overall, it has thus far tended to be
    limited. Minnesota officials, for example, said that they sent out
    a press release and copies of their Partnership Agreement to about
    400 entities comprised of industry, environmental, community, and
    tribal groups and received only a handful of comments. Georgia
    sought comments on its Partnership Agreement at a public meeting
    and received limited comments, and Connecticut held an evening
    meeting with an advisory board consisting of representatives for
    the different media and Page 56
    GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
    of NEPPS Participation Cited, but Full Potential Has Yet to Be
    Realized similarly obtained little feedback. In general, regional
    and state officials said that it will take time to increase the
    public's understanding and interest in focusing on the states' and
    EPA's long-term environmental goals and performance, rather than
    only on specific activities or conditions of more immediate
    concern. Improved Communication      Nearly all EPA regional and
    state officials that we interviewed said that a Among Participants
    About    key benefit of NEPPS has been improved communications
    among program Program Priorities and      participants and the
    fostering of a better federal-state working Other Key Matters
    relationship. Members of the EPA Chicago Regional Office's NEPPS
    coordinating committee (which represents all media and enforcement
    programs), said that NEPPS has provided the region with a better
    understanding of states' strategic plans, which has assisted the
    states when negotiating a change with the region. In addition,
    NEPPS has encouraged regional and state staff of all media
    programs to discuss their programs jointly, a practice that has
    helped program officials at both the state and regional level gain
    a better understanding of each other's needs. According to state
    and regional officials, this higher level of understanding has
    been a major factor that has helped them to improve the way they
    set priorities across programs. Headquarters enforcement officials
    also point to regional efforts to try to use NEPPS as a vehicle to
    more actively engage the states in joint enforcement planning and
    priority-setting. EPA's Boston Regional Office, for example,
    systematically arrayed a number of multimedia enforcement and
    compliance assistance programs for discussion and possible
    incorporation in states' fiscal year 2000 Performance Partnership
    Agreements. In each case, the priority the agency attaches to the
    program is indicated as well as the type of collaboration EPA
    anticipates having with the states.3 Similarly, the officials
    cited as another example a Chicago Regional Office's analysis of
    its Performance Partnership Agreement with Minnesota which
    describes, on a media-specific basis, the state's and EPA's
    commitments to participate in mutually agreed-upon enforcement and
    compliance assurance activities to realize jointly determined
    environmental objectives. Senior officials and program managers in
    all six states we visited also agreed that the NEPPS process has
    improved EPA-state communication and overall relations. Many also
    noted that NEPPS highlights and enhances 3EPA's Boston Regional
    Office, "Assistance and Pollution Prevention Programs & Priorities
    For Fiscal Year 2000 State/EPA Performance Partnership Agreement
    (PPA) & Compliance Strategy Discussions" Page 57
    GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
    of NEPPS Participation Cited, but Full Potential Has Yet to Be
    Realized communication among their own state media programs, as
    well as among EPA regional media programs. Noting that improved
    communications can solve 95 percent of their state-regional
    problems, Minnesota officials have instituted routine monthly
    conference calls with EPA's Chicago Regional Office to address
    waste issues and are considering implementing the same process for
    their other media programs. EPA Chicago Regional Office officials
    told us that they are also relying increasingly on oral
    communications with their states in an effort to encourage a more
    collegial and efficient approach to resolving problems. Future
    Prospects for    Officials in each of the case study states that
    we interviewed agree that the Success Depend on       concept
    behind NEPPS, and its potential for achieving a more effective
    partnership between EPA and the states, is worth pursuing. Yet
    while Further Progress        acknowledging some benefit from
    their participation, they also consistently expressed the view
    that the benefits should be greater; that the program has yet to
    achieve its potential; and that improvements are needed. Of
    particular note, providing states with the incentives envisioned
    initially under NEPPS, including the differential oversight as
    discussed in chapter 4, was seen by almost all of the state
    officials we interviewed as critical to the future success of the
    program. This view is reinforced by the resource commitment that
    some states feel has been required to take part in the program.
    Oregon officials, for example, said that they invested a
    significant amount of their resources in conducting a state
    environmental self assessment and other activities to participate
    in the NEPPS-Performance Partnership Agreement process. To date,
    however, these officials noted that they have not gained the
    advantages of reduced oversight leading to increased self
    management of their delegated programs and greater autonomy to
    focus on state priorities. Similarly, in explaining a major reason
    for their decision not to participate in the program, the Deputy
    Director of Michigan's Department of Environmental Quality noted
    that the heavy investment cited by participating states and the
    modest benefits achieved by those states has led to the
    Department's decision to wait and see how NEPPS evolves. The
    Deputy Secretary of Pennsylvania's Department of Environmental
    Protection had similar reasons for that Department's
    nonparticipation, noting that the department had several state
    initiatives underway that were important and, therefore, they
    would be reluctant to shift resources to NEPPS. The Deputy
    Secretary said that Pennsylvania is reserving judgment as to its
    future participation in NEPPS, noting that if greater Page 58
    GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
    of NEPPS Participation Cited, but Full Potential Has Yet to Be
    Realized progress and benefits under NEPPS accrue over time, it
    may become advantageous for the state to participate. For their
    part, EPA officials acknowledge the states' desire for greater
    program flexibility and autonomy, but believe they are not in a
    position to grant it unconditionally. Specifically, the officials
    maintain that additional program flexibility will have to be
    accompanied by demonstrated, measurable assurances that statutory
    and regulatory requirements and program objectives will still be
    met. As we noted in chapter 4, both EPA and state officials have
    pointed to the difficulty of developing specific, nationwide
    criteria to be used in determining the appropriate level of
    regional oversight of state programs under NEPPS. However, given
    the importance to the program's future of making progress on this
    issue, it may be helpful for EPA and state officials to
    collaborate in developing some type of non-binding guidance that
    could be used in guiding the negotiations of individual regions
    and states on this sensitive issue. In addition to this overriding
    concern about oversight, NEPPS participants believe that the
    benefits that have accrued from their participation in NEPPS have
    not reached their full potential. For example, many participants
    have noted improvements in communication under NEPPS, but said
    that further improvements are needed to ensure that all key EPA
    offices provide timely input into Partnership Agreement and
    Partnership Grant negotiations to help state agencies understand
    whether their agreements have full buy-in of all EPA offices.
    Similarly, while Partnership Grants allow for greater flexibility
    in shifting funds among media programs, states have thus far taken
    advantage of this opportunity to only a limited degree. To some
    extent, the base program requirements under individual programs
    combined with financial constraints have limited states'
    flexibility in shifting funds as freely as they would like.
    However, other factors may explain the problem as well, including
    specific grant regulations, resistance by EPA headquarters and/or
    regional staff, or similar resistance among state agencies
    themselves. Joint EPA-State Evaluation    These concerns pose
    challenges for the future of the program. However, Process Needed
    to             we believe such challenges are to be expected in
    the context of a new Improve NEPPS                 program that
    strives to chart a new direction in the EPA-state relationship.
    Importantly, the need to address such challenges was anticipated
    by the 1995 Agreement that launched the program, which called for
    a joint Page 59                                  GAO/RCED-99-171
    Performance Partnership System Chapter 5 Benefits of NEPPS
    Participation Cited, but Full Potential Has Yet to Be Realized
    evaluation system for EPA and the states to review the results of
    their efforts to ensure continuous improvement. To some extent,
    such a joint evaluation process was undertaken to produce the core
    performance measures. The intergovernmental committees that
    developed these measures, composed of representatives of EPA and
    state agencies, produced an initial set of measures for fiscal
    year 1998 that was modified and improved in subsequent years. As
    noted in chapter 3, the measures approved for fiscal year 2000 are
    widely viewed as substantially improved by both EPA and state
    officials. EPA's and states' recent efforts to improve their
    working relationship in cleaning up priority Superfund sites may
    offer another useful precedent for such an effort. Reflecting a
    growing consensus among many in the administration, state
    government, and the Congress that states should take on more
    responsibilities for leading priority site cleanups, EPA and
    representatives from different states formed a number of
    intergovernmental workgroups to recommend ways to overcome the key
    barriers toward this goal.4 For example, a "State Readiness
    Workgroup," composed of representatives of EPA headquarters and
    regional offices and state agencies, was charged with clarifying
    the requirements and circumstances under which states could be
    granted additional responsibilities to clean up these priority
    sites. Similarly, an intergovernmental "Assistance Workgroup" was
    also established to identify the technical financial,
    administrative, and legal assistance needs of the states in their
    efforts to take a lead role in successfully cleaning up Superfund
    sites. According to the Director of the State, Tribal, and Site
    Identification Center (within the Office of Solid Waste and
    Emergency Response), the workgroups were particularly useful in
    fostering collaboration among representatives of EPA's
    headquarters and regional offices involved in the cleanups in a
    manner that helped to identify where the key problems were and
    what practices worked well to address them. The Director said that
    the results of the workgroups have since been incorporated into
    pilot projects in seven states (and their corresponding regional
    offices) designed to increase states' responsibilities in leading
    cleanups of these sites. Conclusions    On the basis of
    information that can be learned from experiences to date of a
    number of states and their corresponding EPA regional offices, we
    4State and EPA efforts to augment states' roles in leading
    Superfund cleanups are discussed in our 1997 report, Superfund:
    Stronger EPA-State Relationship Can Improve Cleanups and Reduce
    Costs (GAO/RCED-97-77, Apr. 1997). Page 60
    GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
    of NEPPS Participation Cited, but Full Potential Has Yet to Be
    Realized believe the systematic joint evaluation process called
    for by the 1995 Joint Commitment to Reform Oversight and Create a
    National Environmental Performance Partnership System should be
    initiated. The goals of this effort should be to (1) identify best
    practices among participating states for dealing with the most
    challenging problems facing the program and (2) eventually obtain
    agreement on actions that will improve and expand the program.
    Such a process has already been used to develop and improve the
    Core Performance Measures used in the NEPPS program, and has
    served as a successful model elsewhere in EPA where new ideas have
    been developed and tested, and agreement among diverse parties on
    their implementation has been reached. We believe a similar
    effort, which targets key issues affecting NEPPS progress and
    which involves representation from EPA headquarters offices, EPA
    regional offices, and participating state agencies, could
    similarly help to expand both the participation in, and
    effectiveness of, this important program. The precise format to be
    used for this process (e.g., whether individual working groups
    should be established or whether a single committee composed of
    senior state and EPA officials should be used) should be
    determined by EPA and state environmental leaders. Recommendations
    We recommend that the Administrator of EPA work with senior-level
    state officials to initiate a joint evaluation process that (1)
    seeks agreement on the key issues impeding progress in developing
    a more effective National Environmental Performance Partnership
    System and (2) develops mutually agreeable remedies for these
    issues. Among the issues such a process could focus on are *
    developing a set of flexible guidelines, to be used as a tool by
    state and EPA regional NEPPS negotiators, that could help to
    clarify the appropriate performance expectations and other
    conditions that states must meet to achieve reduced oversight in
    carrying out their environmental programs and the type of reduced
    oversight (e.g., reduced frequency of reporting, greater autonomy
    in setting program priorities) that could be achieved; *
    identifying what additional work is needed to address the
    challenges in implementing the Core Performance Measures recently
    negotiated by EPA and the Environmental Council of the States for
    fiscal year 2000, including how these measures can best be
    reconciled with the measures adopted by EPA under the Results Act;
* alleviating the resistance among some staff (both within EPA
    offices and among participating state agencies) toward
    implementing the National Page 61
    GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
    of NEPPS Participation Cited, but Full Potential Has Yet to Be
    Realized Environmental Performance Partnership System, through
    training and other strategies; * determining what appropriate
    steps should be taken by EPA and the states to allow for greater
    use by states of the flexibility envisioned under the Performance
    Partnership Grant system to shift resources and funding among
    their media programs; * determining how effective public
    participation in the NEPPS process can best be ensured; * and
    developing ways to improve communication among EPA's headquarters
    and regional offices and participating states to ensure that
    states are given a clear and timely indication on whether key
    elements of their agreements pursuant to the system have the full
    buy-in of major EPA offices. Agency Comments      EPA agreed with
    the report's recommendation that EPA and state efforts to improve
    NEPPS should include training and other efforts to achieve the
    "cultural change" necessary for greater success. The agency also
    pointed out that it recently agreed with representatives of the
    Environmental Council of the States on a basic outline of a joint
    evaluation process. We acknowledge this milestone and note that
    further progress on the details of such a process, including the
    specific issues to be addressed and a timetable for addressing
    them, will be important steps toward improving NEPPS. EPA also
    commented on our recommendation that EPA and state environmental
    leaders should develop guidelines that would help to clarify, for
    EPA and state negotiators, the appropriate performance
    expectations that states must meet to achieve reduced oversight in
    carrying out their environmental programs and the type of reduced
    oversight (e.g., reduced frequency of reporting, greater autonomy
    in setting program priorities) that could be achieved. EPA noted
    that while it agreed with this recommendation in principle, the
    agency and the states believe that each state's Performance
    Partnership Agreement should specify the degree of oversight
    necessary to accommodate the unique environmental problems and
    varied program capabilities of that state. We agree that oversight
    arrangements should be negotiated between each state and its
    corresponding regional office in a manner that accounts for that
    state's unique circumstances, and that these arrangements should
    be specified in the Performance Partnership Agreement. We continue
    to believe, however, that nonbinding national guidance-to be
    agreed upon in advance by EPA and state environmental leaders-
    would be useful in Page 62
    GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
    of NEPPS Participation Cited, but Full Potential Has Yet to Be
    Realized introducing objective parameters to be considered by
    regional and state negotiators as they seek agreement over this
    sensitive issue. Page 63
    GAO/RCED-99-171 Performance Partnership System Appendix I Comments
    From the Environmental Protection Agency and Our Evaluation See
    comment 1. See comment 2. Page 64      GAO/RCED-99-171 Performance
    Partnership System Appendix I Comments From the Environmental
    Protection Agency and Our Evaluation See comment 3. See comment 4.
    See comment 5. Page 65                                 GAO/RCED-
    99-171 Performance Partnership System Appendix I Comments From the
    Environmental Protection Agency and Our Evaluation The following
    are GAO's comments on the Environmental Protection Agency's (EPA)
    letter dated May 20, 1999. 1. We have clarified, in the executive
    summary and chapter 5, that EPA officials and representatives of
    the Environmental Council of the States have recently agreed to
    certain characteristics of a joint evaluation process, and that
    further progress (including decisions on the specific issues to
    address and a timetable for addressing them) would be important
    steps in improving NEPPS. 2. We have amended the report to reflect
    the agency's expectation that its April 1999 2-year guidance
    should allow the regions and states to consider national program
    priorities earlier in their partnership agreement negotiations,
    and thus limit the need to renegotiate priorities that had been
    previously established. At the same time, the report still conveys
    agency officials' views that the guidance will not necessarily
    prevent other circumstances, which are out of EPA's control, from
    necessitating the reopening of an agreement. 3. Citing the
    observation in chapter 3 that (1) EPA has focused on outputs to
    meet its obligations under the Results Act while supporting a
    transition to outcome-based management under NEPPS and (2) these
    conflicting priorities have led to confusion that hinders
    performance partnerships, EPA stated that, to the contrary, both
    the Results Act and NEPPS encourage the development of outcome
    measures and outcome-based management. We acknowledge the shared
    objective of NEPPS and the act in focusing on results. The key
    word, however, is implementation: as we have documented in other
    recent work, the measures EPA has used in its implementation of
    the Results Act have thus far been heavily output-oriented and
    therefore convey priorities that are often in conflict with the
    more outcome-oriented measures being employed under NEPPS. We
    acknowledge the agency's ongoing efforts to orient its Results
    Act-related measures increasingly toward outcomes, and believe
    that further progress toward this end will help to alleviate this
    problem. In addition, we modified our discussion of this issue in
    Chapter 3 to note that the April 1999 Addendum to the Joint
    Statement, co-signed by EPA and the Environmental Council of the
    States, states that core performance measures and other current
    reporting requirements will be relied upon to satisfy EPA's
    Results Act-related data needs. 4. We agree that oversight
    arrangements should be negotiated between each state and its
    corresponding regional office in a manner that accounts Page 66
    GAO/RCED-99-171 Performance Partnership System Appendix I Comments
    From the Environmental Protection Agency and Our Evaluation for
    that state's unique circumstances, and that these arrangements
    should be specified in the state's Performance Partnership
    Agreement. We continue to believe, however, that nonbinding
    national guidance-to be agreed upon by EPA and state environmental
    leaders-would be useful in introducing objective parameters to be
    considered by regional and state negotiators as they seek
    agreement over this sensitive issue. 5. EPA's comment draws a
    clear distinction between issues associated with reporting burdens
    and other issues related to EPA's oversight of state environmental
    programs. We acknowledge circumstances in which EPA and a state
    collaboratively pursue strategies to reduce reporting requirements
    that both agree are unnecessary, duplicative, or inefficient; and
    that such circumstances could be viewed as outside the two
    parties' oversight arrangement. However, the distinction between
    this activity and oversight is not always so clear. Specifically,
    where states and EPA have disagreed on the need for data not
    required by statute and viewed by states as extraneous, and EPA
    has continued to require reporting of such data, states have often
    characterized the issue as, in their view, a questionable exercise
    of EPA oversight. Page 67
    GAO/RCED-99-171 Performance Partnership System Appendix II GAO
    Contacts and Staff Acknowledgments GAO Contacts       Steven
    Elstein, (202) 512-6515 Acknowledgments    In addition to those
    named above, Maureen Driscoll, Gerald Laudermilk, Susan McCartin
    and Lisa Pittelkau made key contributions to this report. (160447)
    Page 68                           GAO/RCED-99-171 Performance
    Partnership System Ordering Information The first copy of each GAO
    report and testimony is free. Additional copies are $2 each.
    Orders should be sent to the following address, accompanied by a
    check or money order made out to the Superintendent of Documents,
    when necessary. VISA and MasterCard credit cards are accepted,
    also. Orders for 100 or more copies to be mailed to a single
    address are discounted 25 percent. Orders by mail: U.S. General
    Accounting Office P.O. Box 37050 Washington, DC  20013 or visit:
    Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S.
    General Accounting Office Washington, DC Orders may also be placed
    by calling (202) 512-6000 or by using fax number (202) 512-6061,
    or TDD (202) 512-2537. Each day, GAO issues a list of newly
    available reports and testimony.  To receive facsimile copies of
    the daily list or any list from the past 30 days, please call
    (202) 512-6000 using a touchtone phone.  A recorded menu will
    provide information on how to obtain these lists. For information
    on how to access GAO reports on the INTERNET, send an e-mail
    message with "info" in the body to: [email protected] or visit
    GAO's World Wide Web Home Page at: http://www.gao.gov PRINTED ON
    RECYCLED PAPER United States General Accounting Office
    Bulk Rate Washington, D.C. 20548-0001     Postage & Fees Paid GAO
    Permit No. G100 Official Business Penalty for Private Use $300
    Address Correction Requested

*** End of document. ***