Environmental Protection: Collaborative EPA-State Effort Needed to
Improve New Performance Partnership System (Chapter Report, 06/21/1999,
GAO/RCED-99-171).
The Environmental Protection Agency (EPA) has had long-standing problems
in building effective partnerships with the states, which have the lead
responsibility for implementing many environmental programs. Among the
key issues affecting the EPA-state relationship have been concerns that
EPA is inconsistent in its oversight across regions, sometimes
micromanages state programs, does not provide enough technical support
for state programs, and often does not adequately consult with states
before making key decisions affecting them. In 1995, EPA established the
National Environmental Performance Partnership System, a key element of
which was EPA's commitment to give states with strong environmental
performance greater flexibility and autonomy in running their
environmental programs. This report (1) identifies the status of grants
and agreements made under the system between EPA and the states, (2)
examines progress by EPA and the states in developing results-oriented
performance measures to be incorporated into system agreements and
grants to the states, (3) examines how EPA oversight may or may not be
changing in states that are participating in the system, and (4)
discusses the extent to which the use of these performance partnership
agreements and grants has yielded expected benefits.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: RCED-99-171
TITLE: Environmental Protection: Collaborative EPA-State Effort
Needed to Improve New Performance Partnership System
DATE: 06/21/1999
SUBJECT: Grants to states
Environmental monitoring
Environmental policies
Federal/state relations
Performance measures
State-administered programs
IDENTIFIER: EPA National Environmental Performance Partnership System
Maine
Florida
Georgia
Minnesota
Connecticut
Oregon
EPA Performance Partnership Grant
EPA Core Performance Measures
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. This text was extracted from a PDF file. **
** Delineations within the text indicating chapter titles, **
** headings, and bullets have not been preserved, and in some **
** cases heading text has been incorrectly merged into **
** body text in the adjacent column. Graphic images have **
** not been reproduced, but figure captions are included. **
** Tables are included, but column deliniations have not been **
** preserved. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
United States General Accounting Office GAO Report
to the Subcommittee on VA, HUD, and Independent Agencies,
Committee on Appropriations, House of Representatives June 1999
ENVIRONMENTAL PROTECTION Collaborative EPA-State Effort Needed to
Improve New Performance Partnership System GAO/RCED-99-171 GAO
United States General Accounting Office Washington, D.C. 20548
Resources, Community, and Economic Development Division B-282588
June 21, 1999 The Honorable James T. Walsh Chairman, Subcommittee
on VA, HUD, and Independent Agencies Committee on Appropriations
House of Representatives Dear Mr. Chairman: As requested, we are
reporting on the Environmental Protection Agency's (EPA) and the
States' progress in implementing the National Environmental
Performance Partnership System. As arranged with your office,
unless you publicly announce its contents earlier, we plan no
further distribution of this report until 7 days from the date of
this letter. At that time, we will send copies to the appropriate
congressional committees; the Honorable Carol Browner,
Administrator, EPA, and the Honorable Jacob Lew, Director, Office
of Management and Budget. We will also make copies available to
others upon request. Please call me at (202) 512-6111 if you or
your staff have any questions. Major contributors to this report
are listed in appendix II. Sincerely yours, Peter F. Guerrero
Director, Environmental Protection Issues Executive Summary
Purpose The Environmental Protection Agency (EPA) has had
long-standing difficulties in establishing effective partnerships
with the states, which generally have the lead responsibility in
implementing many environmental programs. Among the key issues
affecting EPA-state relationships have been concerns that EPA (1)
is inconsistent in its oversight across regions, (2) sometimes
micromanages state programs, (3) does not provide sufficient
technical support for state programs' increasingly complex
requirements, and (4) often does not adequately consult the states
before making key decisions affecting them. To address these
problems and improve the effectiveness of environmental program
implementation, EPA's Administrator and leaders of state
environmental programs established the National Environmental
Performance Partnership System (NEPPS) in May 1995. In signing the
agreement that established NEPPS, EPA and state leaders said that
the system is designed to strengthen protection of public health
and the environment by directing scarce public resources toward
improving environmental results, allowing states greater
flexibility to achieve those results, and enhancing accountability
to the public and taxpayers. A key element of NEPPS was EPA's
commitment to give states with strong environmental performance
greater flexibility and autonomy in running their environmental
programs. Given the expectation among participants that NEPPS
could deal with many of the issues that have long impeded EPA-
state relationships, the Chairman, Subcommittee on VA, HUD, and
Independent Agencies, House Committee on Appropriations, asked GAO
to examine the progress made by EPA and the states since the 1995
agreement. Specifically, as agreed with the Chairman's office,
this report (1) identifies the status of grants and agreements
made under NEPPS between EPA and participating states, (2)
examines the progress that EPA and the states have made in
developing results-oriented performance measures to be
incorporated into NEPPS agreements and grants to the states, (3)
examines how EPA oversight may or may not be changing in states
that are participating in NEPPS, and (4) discusses the extent to
which the use of these Performance Partnership Agreements and
Grants has achieved the benefits envisioned for the states and the
public. Background Under NEPPS, states may voluntarily enter
into "Performance Partnership Agreements" with their EPA regional
offices. While there is considerable flexibility in how the
agreements may be designed, they typically provide a means for EPA
and the states to negotiate such matters as (1) which problems
will receive priority attention within the state programs, Page 2
GAO/RCED-99-171 Performance Partnership System Executive Summary
(2) what EPA's and the states' respective roles will be, and (3)
how the states' progress in achieving clearly defined program
objectives will be assessed. An important component of the
Partnership Agreements is the use of a common set of national
environmental indicators (called "Core Performance Measures") to
measure the effectiveness and success of states' environmental
programs. In their efforts to develop these performance measures,
EPA and state officials have sought to move beyond counting the
number of actions (such as the number of inspections conducted or
environmental enforcement actions taken) and increasingly toward
evaluating the impact of programs on the environment. While NEPPS
provides the overarching framework for developing Partnership
Agreements, the Performance Partnership Grants Program, authorized
by the Congress in April 1996, is used by many states as a major
tool to implement them. This program allows states to request that
funds from 2 or more of the 15 eligible categorical grants be
combined to give governmental entities greater flexibility in
targeting limited resources to their most pressing environmental
needs. These grants are also intended to be used to better
coordinate existing activities across environmental media and to
develop multimedia programs. While the Partnership Agreements are
designed to complement the Partnership Grants, states are free to
negotiate agreements and/or grants or to decline participation in
NEPPS altogether. Results in Brief State participation in the
National Environmental Performance Partnership System grew from 6
pilot states in its initial year in fiscal year 1996 to 45 states
by the end of fiscal year 1998. Of that number, 31 states had both
Performance Partnership Agreements and Performance Partnership
Grants with EPA in 1998; 12 states had grants only; 2 states had
agreements only; and 5 states did not participate at all.
Nationwide, for that year, $217 million of $745 million in state
environmental program grants was consolidated into Performance
Partnership Grants-an increase of 28 percent from the previous
year. EPA and the states agree on the importance of measuring the
outcomes of environmental activities rather than just the
activities themselves. However, the development of these measures
has been impeded by a number of technical challenges, including
(1) an absence of baseline data against which environmental
improvements could be measured, (2) the inherent difficulty in
quantifying certain results, (3) the difficulty of linking program
activities to environmental results, and (4) the considerable Page
3 GAO/RCED-99-171 Performance
Partnership System Executive Summary resources needed for high-
quality performance measurement. In addition, EPA and the states
have had to resolve fundamental disagreements over a number of
issues, including (1) the degree to which states should be
permitted to vary from the national core measures and (2) the
composition of the measures-particularly regarding the degree to
which preexisting output measures are to be retained as newer
outcome measures are added. Despite these barriers, EPA and state
leaders have managed to agree on a set of core measures for fiscal
year 2000 that are widely regarded by EPA and state officials as
significantly improved from those negotiated in previous years.
The initial expectation that participation in NEPPS would be
accompanied by reduced federal oversight of states has thus far
been realized to a limited degree. A number of instances were
identified among the six participating states GAO visited where
oversight reduction did accompany participation in the system.1
However, in other cases cited by both state and EPA regional
officials, (1) decreased oversight could either not be linked
directly to NEPPS participation or (2) oversight had either
remained the same or increased. Among the factors cited by these
officials as complicating reduced EPA oversight were (1) statutory
and/or regulatory requirements that in some cases prescribe the
kind of oversight required of states by EPA; (2) reluctance by EPA
regulators to reduce oversight without the measures in place to
ensure that environmental quality would not be compromised; (3)
the inherent difficulty in "letting go" on the part of some
regulators that have implemented the existing EPA-state oversight
arrangement for several decades; and (4) EPA's multi-level
organizational structure, which complicates efforts to identify
whether all key agency decisionmakers among the agency's
headquarters and regional offices are in agreement on key
oversight-related questions. EPA and state participants
nonetheless cited a number of benefits associated with NEPPS,
noting in particular that participation (1) provided a means of
getting buy-in for innovative and/or unique projects, (2) allowed
states the option to shift resources and funds under the
Performance Partnership Grants Program, (3) served as a tool to
divide an often-burdensome workload more efficiently between
federal and state regulators, and (4) improved communication and
increased understanding among EPA and state program participants
about program priorities and other key matters. Yet while
participants from each state indicated that their participation in
the voluntary program would probably continue, they also
consistently expressed the view that the benefits of the program
1The six states were Connecticut, Florida, Georgia, Maine,
Minnesota, and Oregon. Page 4
GAO/RCED-99-171 Performance Partnership System Executive Summary
should be greater; that the program has yet to achieve its
potential; and that improvements are needed. The 1995 agreement
anticipated the appropriateness of such reflection in calling for
"a joint evaluation system for EPA and the states to review the
results of their efforts to ensure continuous improvement." GAO
recommends in this report that such a joint evaluation process be
initiated and suggests a number of issues to be considered for
attention during such a process. Principal Findings Growth of
State NEPPS was initially tested on a pilot basis in
fiscal year 1996 with six Participation in NEPPS participating
states. This first year was viewed by EPA and the states as a time
to experiment with the new system and various ways to implement
it. The number of participating states has increased since that
time to 45 states in fiscal year 1998, although the extent of
their participation has varied widely. For example, half the
states have negotiated both Partnership Agreements and Partnership
Grants through their lead environmental agencies that cover most
EPA programs; other states have thus far limited their
participation to a Partnership Grant, such as one administered by
their agriculture agency that, for example, addresses only
pesticide programs. States have also varied considerably in the
detail and content of their agreements. Senior officials in EPA's
Office of State and Local Relations explained that the agency has
not attempted to impose uniformity on the development of
Partnership Agreements at this early stage of the NEPPS process
and has, therefore, refrained from issuing guidance on how the
agreements should be structured. Hence, the agreements vary widely
in content and emphasis, reflecting individual states' conditions
and priorities and reflecting the results of negotiations with
their respective EPA regional offices. While Performance
Partnership Grants allow eligible states to request that funds
from two or more categorical grants (such as those authorized
under the Clean Water Act or those used to implement the Clean Air
Act) be combined to allow for greater flexibility in targeting
limited resources to states' most pressing environmental needs,
the percentage of eligible grant funds consolidated under these
Grants is less than one-third. For fiscal year 1998, $217 million
(29 percent) of eligible grants was consolidated among the
participating states, while $528 million (71 percent) remained as
categorical grants. This level of consolidation Page 5
GAO/RCED-99-171 Performance Partnership System Executive Summary
represents an increase of 28 percent over the $169 million that
was consolidated the previous year. Progress in Developing
Both EPA and individual states have a number of efforts under way
to Results-Oriented Measures develop effective performance
measures to better understand whether their programs are achieving
their intended results. Their collective effort to develop such
measures for NEPPS has centered on the "Core Performance Measures"
that have been negotiated between EPA and the Environmental
Council of the States during the past several years.2 The effort
has faced a number of technical challenges inherent in developing
defensible results-oriented measures. The results of activities
designed to improve water quality, for example, can take years to
appear, and the capability of many states to monitor a significant
share of their waters is limited. Moreover, even if environmental
conditions could be reliably and consistently measured, it may be
particularly difficult to demonstrate the extent to which a
government program affected that condition. Officials from Florida
(a state that has made a significant commitment to measuring
compliance rates and environmental indicators), for example,
explained that factors outside their control, such as economic
activity and weather conditions, make it particularly difficult to
link program activities with changes in environmental conditions.
In addition to these technical challenges in developing results-
oriented measures, the effort has also been challenged by
disagreements between EPA and the states on a number of issues,
including (1) the degree to which states should be permitted to
vary from the national core measures and (2) the composition of
the measures, particularly regarding the degree to which
preexisting output measures are to be retained as newer outcome
measures are added. Overall, however, the states and EPA have made
progress in meeting these challenges. For example, officials in
four of the six states whose programs GAO examined have developed
and implemented their own measures to address their own
priorities. At the same time, program officials in each of the six
states have also agreed to report information required for the
national core measures agreed upon between the Environmental
Council of the States and EPA. In addition, while they maintain
that further refinement will still be needed, EPA and state
officials have agreed on a set of fiscal year 2000 measures for
use in negotiating EPA-state partnership agreements that, by most
accounts, are a substantial improvement over those negotiated from
previous years in that 2The Environmental Council of the States is
a national nonpartisan, nonprofit association of state and
territorial environmental commissioners. Page 6
GAO/RCED-99-171 Performance Partnership System Executive Summary
they are fewer in number (i.e., better targeted to address key
goals) and generally more outcome-oriented. Reductions in
Oversight Instances of greater state flexibility and reduced
EPA oversight tended to Attributable to NEPPS focus on
reducing the frequency of reporting and, in some cases, the Have
Thus Far Been frequency of on-site reviews. Maine
environmental officials, for example, Modest
noted that more frequent, and less formal, dialogue between the
program staff and regional staff had replaced written reports,
saving time and improving the level of cooperation between EPA and
state staff. While Maine program officials attributed the
reductions in part to the assignment by EPA's Boston Regional
Office of a liaison for each state's delegated programs, they
credited NEPPS with formalizing or legitimizing the changes.
Florida program officials identified sizable reporting reductions
in its waste program as a result of a joint state-EPA effort
included in the Partnership Agreement. Other instances were cited
by officials in Georgia and Minnesota. Yet aside from such
individual instances of streamlining reporting requirements and
similar tracking efforts, the large majority of the state
officials GAO contacted generally maintained that participation in
NEPPS has not yet brought about significant reductions in
reporting and other oversight activities by EPA staff, nor has it
resulted in significant opportunities for them to focus on other
priorities or to shift resources to weaker program areas. EPA
officials generally acknowledged this point, but provided specific
reasons why oversight of state programs has not significantly
decreased as a result of NEPPS-and in some cases has actually
increased. Some headquarters and regional officials, for example,
noted that environmental statutes or regulations sometimes
prescribe the level of oversight required of EPA, leaving little
room for EPA to scale it back. The officials also pointed to (1)
audits that identified problems in some state enforcement programs
(such as the underreporting by states of significant violations
and precipitous decreases in the number of state enforcement
actions taken) that they believed called for greater oversight and
(2) the difficulty in scaling back oversight without measurable
assurances indicating that state programs experimenting with
alternative compliance strategies are achieving their desired
results. At the same time, EPA officials cited a number of
barriers preventing greater state flexibility that could be more
readily addressed. For example, senior EPA officials in three of
the four regional offices that GAO visited acknowledged that
support for NEPPS within EPA varies. One senior Page 7
GAO/RCED-99-171 Performance Partnership System Executive Summary
regional official explained that many regional managers and staff
are often more comfortable with preexisting ways of doing business
and are unsure as to how they can accomplish their work in the
context of the partnership approach under NEPPS. The official also
said that there may be a need for training EPA regional staff in
NEPPS implementation. Another senior regional official said that
some agency staff will only take NEPPS seriously when their reward
system is more closely tied to their performance in implementing
the program. Headquarters officials also acknowledged another
problem cited by many of the state officials GAO contacted-that
headquarters' guidance, initiatives, and special requests
sometimes arrive at the regions too late to be used effectively in
regional-state Partnership Agreement negotiations and that they
have taken steps to address the problem. Benefits of NEPPS
Despite their disappointment at the rate of progress in achieving
greater Participation Cited, autonomy and greater emphasis on
state priorities, senior officials and program managers from each
of the six states in GAO's review agreed that but Full Potential
Has NEPPS has provided their programs with worthwhile benefits,
and that its Yet to Be Realized potential for achieving a
more effective partnership between EPA and the states is still
worth pursuing. Among the examples cited were instances in which
Partnership Agreements were used to more efficiently divide a
heavy workload between regional and state staff, and in which
states were able to take at least limited advantage of the
flexibility in their Performance Partnership Grant agreements to
shift resources among their media programs. Overall, however, the
most frequently cited benefit among both state and EPA regional
participants was that the two-way negotiation process inherent in
the program has fostered more frequent and effective communication
between regional and state participants and improved their overall
working relationship. At the same time, state officials almost
unanimously expressed the view that the benefits from their
investment of time and resources into NEPPS should be greater;
that the program has yet to achieve its potential; and that
improvements are needed. Of particular note, almost all of the
state officials GAO interviewed cited progress in achieving
reduced oversight and greater autonomy as critical to the future
success of the program. Also cited was the need to continue
improving performance measures; addressing the barriers impeding
greater acceptance of NEPPS among staff within both EPA and state
agencies; determining how to make greater use of the flexibility
under Performance Partnership Grants to shift resources and
funding to address higher priorities; and improving the manner in
Page 8 GAO/RCED-99-171 Performance
Partnership System Executive Summary which headquarters offices
provide their input into regional-state NEPPS negotiations. These
concerns pose challenges for the future of NEPPS-challenges that
were anticipated by the 1995 agreement that launched the program,
which called for a joint evaluation system for EPA and the states
to review the results of their efforts to ensure continuous
improvement. On the basis of the considerable information that can
be learned from the experiences to date of participating states
and regional offices, GAO believes that it is now appropriate to
undertake such a joint evaluation process, with the goals of (1)
identifying best practices among participating states for dealing
with the most challenging problems facing the program and (2)
eventually obtaining agreement on actions that will improve and
expand the program. EPA officials and representatives of the
Environmental Council of the States have, in fact, recently agreed
upon the basic outline of such a joint evaluation process. Further
progress (including decisions on the specific issues to address
and a timetable for addressing them) would be important steps in
expanding both the participation in, and effectiveness of, this
important program. Recommendations GAO recommends that the
Administrator, EPA, work with senior-level state officials to
initiate a joint evaluation process that (1) seeks agreement on
the key issues impeding progress in developing a more effective
National Environmental Performance Partnership System and (2)
develops mutually agreeable remedies for these issues. Among the
issues such a process could focus on are these: * Developing a set
of flexible guidelines, to be used as a tool by state and EPA
regional NEPPS negotiators, that could help to clarify the
appropriate performance expectations and other conditions that
states must meet to achieve reduced oversight in carrying out
their environmental programs and the type of reduced oversight
(e.g., reduced frequency of reporting, greater autonomy in setting
program priorities) that could be achieved. * Identifying what
additional work is needed to improve the Core Performance Measures
recently negotiated by EPA and state representatives for fiscal
year 2000. * Alleviating the resistance among some staff (both
within EPA offices and among participating state agencies) toward
implementing NEPPS, through training efforts and other strategies.
* Determining what appropriate steps should be taken by EPA and
the states to allow for greater use by states of the flexibility
envisioned under the Page 9 GAO/RCED-
99-171 Performance Partnership System Executive Summary
Performance Partnership Grant system to shift resources and
funding among their media programs. * Determining how effective
public participation in the NEPPS process can best be ensured. *
Developing ways to improve communication among EPA's headquarters
and regional offices and participating states to ensure that
states are given clear and timely information on whether key
elements of their NEPPS-related agreements have the full buy-in of
key EPA offices. Agency Comments GAO provided a draft of this
report for review and comment to EPA and the Environmental Council
of the States. EPA said that "the Report describes, in a fair and
balanced manner, the progress EPA and the States have made through
performance partnerships." EPA also agreed with the report's
recommendation that agency and state efforts to improve NEPPS
should include training and other efforts to achieve the cultural
change necessary for greater success. EPA also commented on GAO's
recommendation that EPA and state environmental leaders should
agree on guidelines that would help to clarify, for EPA and state
negotiators, the appropriate performance expectations that states
must meet to achieve reduced oversight in carrying out their
environmental programs and the type of reduced oversight that
could be achieved. EPA noted that while it agreed with this
recommendation in principle, EPA and the states believe that each
state's Performance Partnership Agreement should specify the
degree of oversight necessary to accommodate the unique
environmental problems and varied program capabilities of that
state. GAO agrees that oversight arrangements should be negotiated
between each state and its corresponding regional office in a
manner that accounts for that state's unique circumstances, and
that these arrangements should be specified in the Performance
Partnership Agreement. GAO continues to believe, however, that
nonbinding national guidance-to be agreed upon in advance by EPA
and state environmental leaders-would be useful in introducing
objective parameters to be considered by regional and state
negotiators as they seek agreement over this sensitive issue. In
addition to these comments, EPA provided updated information and
comments on several other issues (discussed at the end of chs. 3,
4, and 5). EPA's comments, together with GAO's detailed responses,
are included in appendix I. Page 10
GAO/RCED-99-171 Performance Partnership System Executive Summary
Representatives of the Council provided a number of suggested
clarifications. They cautioned that since their comments had not
been reviewed by the Council's membership, they should be viewed
as informal suggestions to enhance the accuracy and completeness
of the report. GAO made revisions as appropriate to incorporate
these comments. Page 11 GAO/RCED-99-171
Performance Partnership System Contents Executive Summary
2 Chapter 1
14 Introduction NEPPS Was Designed to Improve
the Effectiveness of the 15 EPA-State Working
Relationship Objectives, Scope, and Methodology
19 Chapter 2
21 Growth of State Initial Implementation Was
Devoted to Experimentation 21 State
Participation Expanded Rapidly Since Initial
22 Participation in Implementation NEPPS Chapter
3
25 EPA and States Have Developing and Agreeing on Core
Performance Measures Has 25 Been Difficult Made
Progress in Status of Core Performance Measures
34 Developing Conclusions
35 Results-Oriented Agency Comments
36 Performance Measures Chapter 4
37 Reductions in EPA's Initial Expectations Concerning
EPA Oversight of Participating 37 States' Programs
Oversight Attributable States and Regional Offices Report Limited
Oversight Reduction 39 to NEPPS Have Thus
Thus Far Directly Attributable to NEPPS Far Been Modest
Factors Affecting Potential to Reduce Oversight Under NEPPS
41 Agency Comments
49 Chapter 5
51 Benefits of NEPPS Program Improvements Attributed
to NEPPS 51 Future Prospects
for Success Depend on Further Progress 58
Participation Cited, Conclusions
60 but Full Potential Has Recommendations
61 Yet to Be Realized Agency Comments
62 Page 12 GAO/RCED-99-171 Performance
Partnership System Contents Appendixes Appendix I: Comments
From the Environmental Protection 64 Agency
and Our Evaluation Appendix II: GAO Contacts and Staff
Acknowledgments 68 Tables Table
1.1: Categories of Environmental Performance
18 Measurement Table 3.1: Number of Core Performance Measures,
Fiscal Years 35 1998 Through 2000 Figures
Figure 2.1: State Participation in Performance Partnership
22 Agreements and Grants, Fiscal Year 1998 Figure 2.2: State
Environmental Agencies Participating in 23
Performance Partnership Agreements and Grants, Fiscal Year 1998
Abbreviations EPA Environmental Protection Agency GAO
General Accounting Office NEPPS National Environmental
Performance Partnership System Page 13
GAO/RCED-99-171 Performance Partnership System Chapter 1
Introduction The Environmental Protection Agency (EPA) has had
long-standing difficulties in establishing effective partnerships
with the states. Among the key issues affecting EPA-state
relationships have been concerns that EPA (1) is inconsistent in
its oversight across regions, (2) sometimes micromanages state
programs, (3) does not provide sufficient technical support for
state programs' increasingly complex requirements, and (4) often
does not adequately consult the states before making key decisions
affecting them. In an effort to address these problems and improve
the effectiveness of environmental program implementation, EPA and
state environmental agencies established the National
Environmental Performance Partnership System (NEPPS). Under this
system, strong state programs were to be given more leeway to set
environmental priorities, design new strategies for addressing
these priorities, and manage their own programs-allowing EPA to
concentrate more effort, oversight, and technical assistance on
weaker programs. A major component of the system is the
development of Performance Partnership Agreements. These
agreements are to provide a means for EPA and the states to
negotiate such matters as (1) which problems will receive priority
attention within state programs, (2) what EPA's and the states'
respective roles will be, and (3) how the states' progress in
achieving clearly defined program objectives will be assessed.
States may also establish Performance Partnership Grants, which
allow them to consolidate grants as a way of providing more
flexibility in managing their environmental grant funds, and to
cut paperwork and simplify financial management. For example, a
state that would otherwise have separate water, air, and pesticide
grants can now combine the funds from some or all of these grants
into one or more performance partnership grants. Given the
expectation among participants that NEPPS could deal with many of
the issues that have long impeded the EPA-state relationship, the
Chairman, Subcommittee on VA, HUD, and Independent Agencies, House
Committee on Appropriations, asked us to examine the progress made
by EPA and the states since the 1995 agreement. Specifically, as
agreed with the Chairman's office, this report (1) identifies the
status of grants and agreements made under NEPPS between EPA and
participating states, (2) examines the progress that EPA and the
states have made in developing results-oriented performance
measures to be incorporated into NEPPS agreements and grants to
the states, (3) examines how EPA oversight may or may not be
changing in states that are participating in NEPPS, and (4)
discusses the extent to which the use of these performance
partnership Page 14 GAO/RCED-99-171
Performance Partnership System Chapter 1 Introduction agreements
and grants has achieved the benefits envisioned for the states and
the public. NEPPS Was Designed Most of the nation's
environmental statutes envision a strong role for the to Improve
the states in implementing and managing environmental
programs. Toward this end, in 1993, a joint State/EPA task force
recommended that EPA and Effectiveness of the the states
adopt a more systematic approach to manage environmental programs
in a way that allows each level of government to contribute EPA-
State Working according to its respective strengths. In
May 1993, the EPA Administrator Relationship
established a State/EPA Steering Committee to oversee the
implementation of the task force's recommendations. Subcommittees
were established to pursue work on oversight reform, with the goal
of increasing state participation in EPA decision-making,
developing national environmental goals and measures, allowing
flexible funding across programs, and improving communications
between EPA and states. As a result of these efforts, on May 17,
1995, the EPA Administrator and the leaders of state environmental
programs formally agreed to implement a new environmental
partnership entitled the National Environmental Performance
Partnership System. This agreement, entitled the Joint Commitment
to Reform Oversight and Create a National Environmental
Performance Partnership System, stated that the long-range goal of
NEPPS was "to provide strong public health and environmental
protection by developing a system where EPA and the states work
together for continuous gains in environmental quality and
productivity." In establishing NEPPS, EPA and the leaders of state
environmental programs indicated the system is designed to
strengthen protection of public health and the environment by
directing scarce resources toward improving environmental results,
allowing states greater flexibility to achieve those results, and
enhancing accountability to the public and taxpayers. The seven
principle components of NEPPS are * increased use of environmental
goals and indicators in order to measure the effectiveness and
success of environmental programs; * a new approach for conducting
assessments of environmental programs, which will include a
greater reliance on annual environmental and programmatic self-
assessments conducted by each state and sharing with the public
information about environmental conditions, goals, priorities, and
achievements; * the development of environmental performance
agreements that outline environmental priorities and goals agreed
to jointly by EPA and the states; Page 15
GAO/RCED-99-171 Performance Partnership System Chapter 1
Introduction * a reduction in oversight for those states with
strong environmental programs, which will enable EPA to focus
resources on states that need more assistance; * the designation
of strong state environmental programs as "leadership programs"
that are afforded minimal oversight; * increased opportunity for
constructive public involvement in the management of environmental
programs through a program that encourages regulated entities and
the general public to review and comment on environmental issues;
and * the development of a joint system evaluation for EPA and the
states to review the results of their efforts to ensure continuous
improvement. As we reported in May 1998, NEPPS is intended to
strengthen the effectiveness of the nation's environmental
programs by redefining the federal and state roles to ensure that
public resources are used efficiently to address the most
important environmental problems.1 According to EPA, NEPPS is
based on a shared recognition that continued environmental
progress can be achieved most effectively by working together as
partners. Accordingly, the effort is designed to promote joint
planning and joint priority-setting, which takes into account each
state's environmental conditions and objectives. A key element of
this program is EPA's commitment to give states with strong
environmental performance greater flexibility and autonomy in
running their environmental programs. To help document this
capability, a primary objective of the program is the measuring
and reporting of EPA's and states' progress toward achieving their
environmental and programmatic goals. Negotiation of
Under NEPPS, states and their corresponding EPA regional offices
are Performance Partnership expected to reach an
understanding of the state's environmental Agreements and
conditions and to agree on appropriate environmental goals and
priorities Performance Partnership and on program performance
indicators to measure progress. The results Grants
of these negotiations are documented in Performance Partnership
Agreements and/or Performance Partnership Grants. Partnership
Agreements are comprehensive agreements that are expected to be
used as the principal mechanism for implementing NEPPS. According
to EPA, the agreements are derived from joint discussions by EPA
and the state on their 1Environmental Protection: EPA's and
States' Efforts to Focus State Enforcement Programs on Results
(GAO/RCED-98-113, May 27, 1998). Page 16
GAO/RCED-99-171 Performance Partnership System Chapter 1
Introduction interests, concerns, choices, and commitments for
sound environmental performance. While NEPPS provides the
overarching framework for developing partnership agreements, the
Performance Partnership Grants Program serves as a major tool to
implement them. Performance Partnership Grants are intended to
allow states greater flexibility in deciding how federal grant
funds can best be spent to achieve their environmental goals.
Under these grants, which were authorized by the Congress in April
1996, eligible states and tribes may request that funds from two
or more categorical grants (such as those authorized under the
Clean Water Act or those used to implement the Clean Air Act) be
combined into one or more grants to give governmental agencies
greater flexibility in targeting limited resources to their most
pressing environmental needs. These grants are also intended to be
used to better coordinate existing activities across environmental
media and to develop multimedia programs. Importantly, state
participation in NEPPS is voluntary. In particular, while
Partnership Agreements are designed to complement Partnership
Grants, states are free to negotiate both agreements and grants or
to decline participation in NEPPS altogether. Development of
A key component of the 1995 NEPPS agreement was the commitment by
EPA Performance Measures Is a and the Environmental Council of
the States to identify a common set of Key Component of NEPPS
national environmental indicators to measure the effectiveness and
success of states' environmental programs.2 In an effort to
fulfill this commitment, on August 20, 1997, EPA and the Council
agreed on a set of "Core Performance Measures" for EPA and states
to use in measuring progress toward the achievement of
environmental and program goals. This first set was used to
measure progress in fiscal year 1998 and, with some minor
revisions, was used again in fiscal year 1999. In their efforts to
develop these performance measures, EPA and state officials have
sought to move beyond counting the number of actions and
increasingly toward evaluating the impact of programs on the
environment. Traditionally, performance measures have focused on
tracking "outputs," such as the number of inspections conducted
and enforcement actions taken. Such actions are easiest to count,
and they provide a useful measure of the level of agency activity.
On the other hand, 2The Environmental Council of the States is a
national nonpartisan, nonprofit association of state and
territorial environmental commissioners. Page 17
GAO/RCED-99-171 Performance Partnership System Chapter 1
Introduction measuring the actual results a program is intended to
achieve, such as the degree to which progress is made in achieving
air or water quality standards, is more difficult but provides
information on whether the goals of the regulatory program are
being achieved.3 In order to strike a better balance between
output measures and measures of program results, EPA and the
Council developed a tiered approach, shown in table 1.1, to better
account for program results. As the table indicates, an output
measure considers numbers of actions taken, demonstrating the
level of a particular activity or how resources are used. An
outcome, on the other hand, can measure the results associated
with a particular policy, such as the percent of facilities in
environmental compliance. Finally, environmental indicators
demonstrate whether overall, long-term agency objectives are being
achieved, such as the trend in the number of bodies of water
meeting clean water standards. Table 1.1: Categories of
Environmental Performance Measurement Measure
Characteristic Examples Purpose Output
Numbers of actions Number of penalty Demonstrates level dollars
collected; of activity; number of demonstrates
how violations resources are used discovered Outcome
Environmental or Tons of pollution Demonstrates
programmatic reduced or results of specific
results associated percent of facilities initiatives or with a
particular in environmental policies program or policy
compliance Environmental indicator Indicators
Trend in number of Demonstrates associated with bodies of
water whether overall, overall meeting
clean long-term agency environmental or water
standards objectives are program objectives
being achieved Note: In its efforts to develop overall performance
measures for the Government Performance and Results Act of 1993,
EPA uses slightly different terms: "outputs," "intermediate
outcomes," and "long-term outcomes." In its guide to implementing
the act, the Office of Management and Budget distinguishes between
"output goals" and "outcome goals" and calls on federal agencies
to measure progress toward both. Other experts in the field of
government performance measurement labeled the three tiers
"outputs," "policy or behavioral outcomes," and "program
outcomes." See for example, Sparrow, Malcolm, "Regulatory
Agencies, Searching for Performance Measures That Count," and
Greiner, John M., "Positioning Performance Measurement for the
Twenty-first Century, "Organization Performance and Measurement in
the Public Sector, Quorum Books, (1996). Source: Environmental
Protection: EPA's and States' Efforts to Focus State Enforcement
Programs on Results (GAO/RCED-98-113, May 27,1998). 3Thus, for
example, one outcome-oriented core measure in the air program
tracks overall emission reductions for key pollutants over time.
Page 18 GAO/RCED-99-171
Performance Partnership System Chapter 1 Introduction NEPPS'
emphasis on performance measurement also provides a critical link
to the Congress' intent in passing the Government Performance and
Results Act of 1993. The Results Act requires agencies to clearly
define their missions, establish long-term strategic goals (and
annual goals linked to them), measure their performance against
the goals they have set, and report this information to the
Congress. Importantly, rather than focusing on the performance of
prescribed tasks and processes, the statute emphasizes the need
for agencies to focus on and achieve measurable program results.
Objectives, Scope, Our objectives in this review were to (1)
identify the status of grants and and Methodology agreements
made pursuant to NEPPS between EPA and participating states, (2)
examine the progress that EPA and the states have made in
developing results-oriented performance measures to be
incorporated into NEPPS agreements and grants to the states, (3)
examine how EPA oversight may or may not be changing in states
that are participating in NEPPS, and (4) discuss the extent to
which the use of performance partnership agreements and grants has
achieved the benefits envisioned for the states and the public.
For the first objective, we reviewed EPA documents describing the
overall status of performance partnership grants and agreements
made between EPA and states. We also interviewed officials from
EPA's Office of State and Local Relations to obtain the latest
data and related information on the status of Partnership
Agreements and Partnership Grants signed by the states and EPA.
For the remaining objectives, we first contacted EPA (headquarters
and regional) officials to identify appropriate state
environmental programs for detailed study. In selecting states, we
were primarily concerned with the degree of state participation in
this voluntary program, the length of time they have been
participating, and the desirability of examining states with
different experiences and geographical locations. On the basis of
these criteria, we visited six states that have experience with
NEPPS for detailed study-Connecticut, Florida, Georgia, Maine,
Minnesota, and Oregon. In each case, we interviewed officials in
the states' lead environmental agency. For each state, we first
discussed the program with officials that have overall
responsibility for NEPPS. To get insights into the status of NEPPS
at the program level, we interviewed program managers from each of
three environmental programs: the Resource Conservation Page 19
GAO/RCED-99-171 Performance Partnership System Chapter 1
Introduction and Recovery Act, the Clean Water Act, and the Clean
Air Act. We also interviewed program officials in the EPA regional
office with jurisdiction for each state we visited. After these
visits, we conducted telephone interviews with environmental
officials from two states that have limited their participation in
NEPPS-Michigan and Pennsylvania-to determine their views of NEPPS
and the reasons why they chose not to participate more fully. At
EPA headquarters, we contacted officials from the various offices
with NEPPS responsibilities, including the Offices of Air and
Radiation; Water; Solid Waste and Emergency Response; Enforcement
and Compliance Assurance; Reinvention; and State and Local
Relations, to discuss our objectives as well as the results of our
specific work at the states and EPA regional offices. We also
gathered information on our objectives through interviews with
officials from other organizations with an interest in NEPPS,
including the Association of State and Interstate Water Pollution
Control Administrators, Association of State and Territorial Solid
Waste Management Officials, Environmental Council of the States,
National Academy of Public Administration, National Governors
Association, and State and Territorial Air Pollution Program
Administrators. Regarding the second objective, we interviewed
officials from the Green Mountain Institute for Environmental
Democracy, which participated in studies of issues related to the
development and/or use of core performance measures. We conducted
our work from June 1998 through April 1999 in accordance with
generally accepted government auditing standards. We provided
copies of this report to EPA and the Environmental Council of the
States for their review and comment. EPA's comments and our
responses are included in appendix I. The Council indicated that
since its response had been prepared without the benefit of review
by Council membership, its comments should be viewed not as
reflecting the Council's positions, but rather as informal
suggestions to enhance the accuracy and completeness of the
report. We made revisions as appropriate to incorporate these
comments. We also provided relevant sections of the draft to
representatives of the eight states included in our review to
verify statements attributed to them, and to verify other
information they provided, and have made revisions as appropriate
to incorporate their comments. Page 20
GAO/RCED-99-171 Performance Partnership System Chapter 2 Growth of
State Participation in NEPPS State participation in the National
Environmental Performance Partnership System has grown
significantly in the 4 years since the system was created,
increasing from 6 pilot states in fiscal year 1996 to 45 states by
the end of fiscal year 1998.1 However, the extent of participation
among these 45 states varied considerably: 31 states had both
performance partnership agreements and grants; 12 states had
grants only; 2 states had agreements only; and 5 states did not
participate at all. Moreover, while some states included a full
range of environmental programs under their agreements, others
included only one or two programs (such as pesticide or drinking
water programs). Initial Implementation NEPPS was initially tested
on a pilot basis in fiscal year 1996 with 6 Was Devoted to
participating states. This first year was viewed as a time to
experiment with the new system and various ways to implement it.
According to a Experimentation 1996 study of
five of the six pilot efforts conducted by the Environmental Law
Institute with funding from EPA,2 although the pilot states shared
ideas during the process of developing their agreements, the
states deliberately avoided discussing some of the specifics of
their approaches so as to ensure diversity. The Environmental Law
Institute's study focused on whether, and how, the pilot
performance partnership agreements achieved and measured
environmental results, how flexibility was exercised under the
program, and how accountability was ensured. Based on the
experiences of the pilot states, the Institute's study concluded
that NEPPS showed great promise for improving the relationship
between EPA and the states and for improving the administration of
the environmental statutes. However, the study cited a number of
issues that would need to be addressed as the program evolved. It
stated, for example, that while states and EPA had made progress
toward the goal of increasing the use of environmental indicators
(measures of overall progress in achieving environmental
objectives), much remained to be done to develop appropriate
measures. The study also concluded improvements were needed to (1)
clarify the relationship between Performance Partnership
Agreements and Grants, (2) more 1For this report, NEPPS
participation is defined as participation in Performance
Partnership Agreements, Performance Partnership Grants, or both.
2An Independent Review of the State-Federal Environmental
Partnership Agreements for 1996, Environmental Law Institute,
(1996). The Performance Partnership Agreement between EPA and the
sixth state was signed after the Institute completed its review
and analysis of the other five agreements and thus was not covered
by this study. The Institute's study did not include a review of
Performance Partnership Grants since the authority for these
grants was not provided by the Congress until the middle of fiscal
year 1996. Page 21 GAO/RCED-
99-171 Performance Partnership System Chapter 2 Growth of State
Participation in NEPPS effectively communicate EPA's national
priorities to EPA regions and states in time to impact state and
EPA regional office negotiations on Performance Partnership
Agreements, and (3) increase public participation in the program.
State Participation State participation in
Performance Partnership Agreements and Grants Expanded Rapidly
expanded rapidly after the first year. In fiscal year 1997, states
and regional offices were expected by EPA headquarters to build on
the prior Since Initial year's experiences
and work on areas that needed additional clarification
Implementation or where barriers needed to
be removed. Participation grew that year to 44 states and to 45
states in fiscal year 1998. Figure 2.1: State Participation in
Performance Partnership Agreements
Agreement only - 2 and Grants, Fiscal Year 1998 Not participating
- 5 * * * * Agreement and
grant - 31 Grant only - 12 Source: Prepared by GAO from EPA's
data. Of the 45 states participating in fiscal year 1998, 31 had
both Performance Partnership Agreements and Grants, 12 states had
grants only, 2 states had agreements only, and 5 states did not
participate at all. (See fig. 2.1.) Since states can have multiple
Performance Partnership Agreements and Grants, Page 22
GAO/RCED-99-171 Performance Partnership System Chapter 2 Growth of
State Participation in NEPPS depending on which state agencies
handle the different environmental programs, the 45 states
accounted for a total of 38 agreements and 52 grants. According to
EPA, states vary in the extent of their participation, with half
the states participating broadly by negotiating both Performance
Partnership Agreements and Performance Partnership Grants that
cover most EPA programs through their state environmental
agencies, while other states limit their participation by
negotiating, for example, a partnership grant through their
agricultural agency that covers pesticide programs. As shown in
figure 2.2, of those states that participated in NEPPS through
their lead environmental agencies in fiscal year 1998, 25 had both
Performance Partnership Agreements and Grants, 4 had grants only,
and 6 had agreements only. Figure 2.2: State Environmental
Agencies Participating in Performance
Grant only - 4 Partnership Agreements and Grants, Fiscal Year 1998
Agreement only - 6 * * * Agreement and grant - 25
Source: Prepared by GAO from EPA's data. States also vary
considerably in terms of the detail and content of their
partnership agreements. Senior officials in EPA's Office of State
and Local Page 23 GAO/RCED-99-
171 Performance Partnership System Chapter 2 Growth of State
Participation in NEPPS Relations explained that the agency has not
attempted to impose uniformity on the development of partnership
agreements at this early stage of the NEPPS process and has,
therefore, refrained from issuing guidance on how partnership
agreements should be structured. Hence, the agreements vary widely
in content and emphasis, reflecting individual state's conditions
and priorities, and their negotiations with their respective EPA
regional offices. Most States Have As discussed in
chapter 1, Performance Partnership Grants allow eligible
Performance Partnership states to request that funds from two
or more categorical grants (such as Grants, but Few Take Full
those authorized under the Clean Water Act or those used to
implement Advantage of the the Clean Air Act) be
combined into one or more grants to give greater Flexibility
Offered flexibility in targeting limited resources to
their most pressing environmental needs. Thus far, however, the
states have consolidated less than one-third of the eligible
categorical grant funds under partnership grants. Of the eligible
grants, 29 percent, or $217 million, was consolidated in fiscal
year 1998, while 71 percent, or $528 million, remained as
categorical grants. This represents an increase of 28 percent over
the $169 million that was consolidated the previous year. Page 24
GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
States Have Made Progress in Developing Results-Oriented
Performance Measures Both EPA and individual states have a number
of efforts underway to develop effective performance measures to
better understand whether their programs are achieving their
intended results. Their collective effort to develop such measures
for NEPPS has centered on the "Core Performance Measures" that
have been negotiated between EPA and the Environmental Council of
the States during the past several years. These measures are
intended to be used in tracking states' progress towards achieving
the most important goals of the nation's environmental programs.
In developing the performance measures, EPA and the states have
retained a number of the traditional output measures they have
used in the past but have attempted to focus increasingly on
measuring desired environmental outcomes. However, overcoming a
number of technical challenges, and reaching agreement on the most
important environmental outcomes and on the methodologies to
measure progress toward those outcomes, has been difficult.
Nevertheless, considerable progress has been made in developing
and improving the performance measures-as evidenced by agreement
on a set of measures for fiscal year 2000 that are widely regarded
as improved measures from previous years. Developing and
EPA and state officials agree on the importance of measuring the
outcomes Agreeing on Core of environmental activities
rather than just the activities themselves. However, developing
such measures has faced a number of challenges. Performance
Outputs, by their nature, are inherently easier to measure,
report, and Measures Has Been understand than outcomes and
environmental results. Compared to output measures, developing
defensible results-oriented measures has proven to Difficult
be substantially more difficult. In addition to these technical
challenges, EPA and the states have differed on what the measures
should look like (particularly regarding the relative emphasis of
output versus outcome measures) and on the degree of flexibility
with which they should be implemented. Technical Challenges EPA
and state officials identified several key technical challenges
that they have had to address in their efforts to focus
performance measurement on desired results. These include (1) an
absence of baseline data against which environmental improvements
could be measured, (2) the inherent difficulty in quantifying
certain results, (3) the difficulty of linking program activities
to environmental results, and (4) the considerable resources
needed for high-quality performance measurement. Page 25
GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
States Have Made Progress in Developing Results-Oriented
Performance Measures Need for Baseline Data to As noted in our
May 1998 report on EPA's enforcement program,1 the Measure
Progress absence of adequate baseline data for
comparison is a common problem among many organizations engaged in
performance measurement, including federal and state agencies.
Measuring environmental improvements requires a starting point
against which to measure changes. Without such a baseline, any
environmental measurement system can only provide a snapshot in
time; it cannot tell whether conditions are getting better or
worse. Federal and state agencies have therefore frequently had to
build entirely new data systems and ways of collecting data
because the old systems are of limited use in analyzing programs'
performance. Our 1998 report noted that compliance data are
especially scarce for small businesses that historically received
few inspections. Consequently, state programs that are just now
attempting to measure results have limited data with which to
compare them. Florida officials, for example, told us that their
recent environmental reports showing industry-wide compliance
rates generally have a baseline of 1997 or 1998, because past
information is unavailable or unreliable. An EPA official
responsible for NEPPS implementation also noted that the scarcity
of baseline information by which to measure program improvements
attributable to NEPPS is a particular challenge and a major
concern to the agency. Inherent Difficulty in Generating
relevant and accurate data is a challenge under the best of
Quantifying Data circumstances. Not only do
appropriate measures need to be defined, methodologies need to be
established to develop the necessary data. In enforcement
programs, for example, it is difficult to determine the impact on
the overall environment from individual inspections conducted or
enforcement actions taken. In addition, as officials told us
during our review of enforcement programs, quantifying industry-
wide compliance rates and other outcomes has been complicated by
the difficulty of deciding both how to define a compliance rate
and how to calculate it. As another example, the results of
activities designed to improve water quality can take years to
appear, and the capability of many states to monitor a significant
share of their waters is limited. These challenges have led some
state officials to note that it may be exceedingly difficult to
achieve comparability from state to state, both in what is being
measured and the methodology used in gathering data. In
particular, a state with more complete data may appear to have
greater 1Environmental Protection: EPA's and States' Efforts to
Focus State Enforcement Programs on Results (GAO/RCED-98-113, May
27, 1998). Page 26 GAO/RCED-99-
171 Performance Partnership System Chapter 3 EPA and States Have
Made Progress in Developing Results-Oriented Performance Measures
environmental problems than a state with poor data. Minnesota
officials, for example, told us that their data base for "impaired
waters" (waters that do not meet state water quality standards)
includes waters that have undergone far more rigorous analysis
than that performed by other states. Consequently, according to
these officials, Minnesota's impaired waters may appear to be far
more severe than those of another state that does not subject its
waters to such rigorous analysis. Similar findings were reached in
a 1998 study evaluating an effort where six New England state
environmental management agencies and EPA's Boston office
collaborated on a menu of environmental indicators intended to
measure (1) the status and trends of the quality of the New
England environment and (2) program accomplishments toward
reaching state and regional environmental goals.2 The findings of
the study were based on an evaluation of data availability and
quality for 12 example indicators, which included 6 specific
performance measures. A key finding of this effort was that the
level of consistency required for regional indicators is difficult
to achieve given (1) a lack of clarity in terms of what the
indicators intend to measure and for what purpose and (2) a lack
of consistency across states in both the type of data collected
and methodology used. Challenges in Linking Program Assuming
environmental conditions could be reliably and consistently
Activities to Environmental measured, it may still be
difficult to demonstrate the extent to which a Outcomes
government program affected that condition. As we noted in a 1997
report on the complexities associated with performance measures,
"Separating the impact of [a] program from the impact of other
factors external to the program was cited by government agency
officials as the most difficult challenge in analyzing and
reporting government performance."3 Even in the case of the
Florida Department of Environmental Protection's significant
commitment to measuring compliance rates and environmental
indicators, regulators made a conscious decision not to link their
enforcement programs with trends in environmental indicators or
outcomes like compliance rates. The regulators explained that the
causes of these trends are subject to other influences outside
their control, such as the state of the economy, the weather, and
other departmental actions 2Green Mountain Institute for
Environmental Democracy, "Indicator Data Catalog, An Evaluation of
Data Issues Related to the Development of Core Performance
Measures and Regional Environmental Indicators," (Nov. 1998).
3Managing for Results: Analytic Challenges in Measuring
Performance (GAO/HEHS/GGD-97-138, May 30, 1997). Page 27
GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
States Have Made Progress in Developing Results-Oriented
Performance Measures besides enforcement. The Department's
consultant agreed, noting, for example, that "If and when the
scallop population in Tampa Bay is restored to healthy levels,
Florida's Department of Environmental Protection . . . would be
hard pressed to prove beyond doubt that their interventions
actually produced this result, no matter how compelling their
scientific analyses and explanations."4 Determining causality has
proven to be particularly difficult among pollution prevention
programs. According to EPA headquarters officials, EPA and the
states have not yet been able to determine how to establish a
cause and effect relationship to measure the impacts on the
environment from many activities that prevent pollution from
occurring. Resource Limitations Another barrier, which
essentially flows from the others, relates to the significant
resources and expertise required for identifying and testing
potential results-oriented performance measures. Once measures are
in place, gathering and analyzing the data can also be resource-
intensive and can take years to show environmental improvements.
In addition, several program officials of the states we visited
told us that some federal and state data bases will require
significant improvement in order to track the new information to
support results measures. A member of the Environmental Council of
the States' Information Management Workgroup agreed, noting that
this is an issue EPA and the states still need to address. Two
states that have developed systems to measure the results of
selected enforcement efforts found that considerable resources are
needed to do quality performance measurement. The Florida
Department of Environmental Protection hired a consultant to
assist them in developing their new performance measurement system
and dedicated several of its own staff to this effort. A
Massachusetts environmental official found that monitoring the
results of even a single program can require considerable
resources. The former Deputy Commissioner said that in a pilot
test of its new Environmental Results Program, the agency had to
invest a great deal of time and energy to work with the facilities
and measure the ultimate results, even though the test involved
only 18 participating companies. Officials from these and other
states noted that it is difficult to commit resources to the
development and implementation of new results-oriented performance
measures while still meeting other program requirements. Results-
Oriented Measures As challenging as the exercise may be for
all programs, we found that Easier to Develop for Some
developing results-oriented performance measures has been easier
in the Programs Than Others 4Malcolm Sparrow, "Regulatory
Agencies, Searching for Performance Measures That Count," (June 9,
1997). Page 28 GAO/RCED-99-171
Performance Partnership System Chapter 3 EPA and States Have Made
Progress in Developing Results-Oriented Performance Measures case
of some programs than others. Air programs, for example, have long
had a monitoring network in place to measure ambient air quality
throughout the country. Accordingly, as officials of EPA's Office
of Air and Radiation told us, the air program has had considerable
background with results-oriented performance measures, and that
this experience has limited both the burden of developing specific
performance measures and the burden on the states of implementing
these measures. Officials of the states we visited generally
confirmed this assessment. A senior official in Georgia's
environmental protection division, for example, told us that
developing results-oriented measures is easiest for the air
program, more difficult for the water program, and most difficult
for the waste program. The Georgia official attributed the
differences to the extensive historical experience of the air
program with results-oriented measures, the length of time it
takes to see measurable results in the water program, and the
difficulty in identifying suitable measures for the waste program.
Similar comments were made by a Florida air program official that
noted that states and EPA have been monitoring air quality for
some time, have good data, and can show results. Challenges in
Obtaining In addition to these technical challenges, EPA
and states have had to Agreement Between EPA resolve
fundamental disagreement over (1) the degree to which states and
the States on the should be permitted to vary from the
national core measures and (2) the Measures
composition of the measures, particularly regarding the degree to
which pre-existing output measures are to be retained as newer
outcome measures are added. Extent to Which States Can EPA's
goal to use the performance measures to provide a national picture
Vary From the Core Measures of environmental progress
necessitates a degree of consistency among the states in what is
being measured. To achieve consistency, the May 1995 NEPPS
Agreement provides that EPA and the states will ". . .develop a
limited number of program and multi-media performance measures
that each state will report so that critical national program data
is collected." However, recognizing that a set of national
measures may not necessarily address individual states' priorities
(or represent what individual states consider to be the best
measures for their state-specific situations), the agreement
further provides that states may develop other goals and
performance indicators that will present a more meaningful picture
of their state's environmental quality. This apparent need was
further recognized in the August 1997 joint statement by EPA and
the Environmental Council of the States, which accompanied the
release of the measures for fiscal year 1998. The statement
indicated that where a Page 29
GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
States Have Made Progress in Developing Results-Oriented
Performance Measures particular performance measure does not fit a
state's situation, that measure may be modified, substituted, or
eliminated if mutually agreed to by both the state and EPA.
Deviations could be warranted, for example, where (1) there may
not be adequate data to report on the measure, (2) alternative
measures may work better, or (3) there may be higher priorities in
a state. According to Council officials, in the first year of the
performance measures, EPA regions were inconsistent in
implementing the performance measures across the country: some EPA
staff in regional offices allowed states flexibility in
implementing performance measures (as intended by the 1997 joint
statement) while staff in other regions tried to portray the
national performance measures as mandatory and inflexible. In June
1998, the president of the Council wrote to the Deputy
Administrator of EPA, asking that the agency reaffirm its support
for the flexibility provisions of the joint statement. Noting that
one of the most challenging aspects of implementing the
performance measures is balancing the need for uniform national
measures with the need to accommodate the circumstances of
individual states, the Deputy Administrator's September 1998
response reaffirmed that under certain circumstances, EPA regions
can adjust a measure that is inappropriate for a particular state.
Updated EPA-Council joint guidance on the use of performance
measures, issued in April 1999 as an addendum to the 1997 Joint
Statement along with the release of the fiscal year 2000 measures,
reiterates EPA's commitment to allow flexible implementation of
the measures in specific situations and with approval of both the
state and EPA. 5 States Have Implemented Both Core Performance
Measures and Their Own Measures As permitted by the 1995 NEPPS
agreement, four of the six states that we visited have developed
some performance measures on their own, separately from the
national core measures. These states use their own measures to
track priority issues in their respective states and to report
environmental progress to their state legislatures and the public.
Florida environmental officials developed their separate measures
in conjunction with the NEPPS program, and they continue to use
them because they 5Specifically, the addendum states that a state
and EPA may jointly agree to deviate from particular performance
measures where (1) the measure does not apply to a state's or
region's physical setting or environmental condition; (2) the
state does not have authority for the program to which the measure
applies; (3) data for the measure are not available or alternative
data are more relevant in painting a picture of environmental
progress; (4) the state and EPA agree that the measure or the work
associated with it are not a high priority in the state. Page 30
GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
States Have Made Progress in Developing Results-Oriented
Performance Measures believe they are better measures of results
than the Core Performance Measures. Georgia and Minnesota
officials developed measures that focus on specific state
priorities, and Oregon officials developed measures that were
specifically tailored to the state's strategic plan. While
environmental program officials in Connecticut and Maine have not
developed performance measures apart from the core measures, they
told us that they believed state-specific rather than national
measures would be more useful to them and more appropriate to
measure the results of environmental programs in their states.
Regardless of whether a state developed its own performance
measures, each of the states we visited also agreed to report on
the national core measures. Normally, the states did not adopt the
core performance measures verbatim; they made minor changes where
appropriate to meet state-specific situations. In each case,
however, the states' changes to the national Core Performance
Measures were reviewed and approved by the appropriate EPA
regional office to ensure that they were compatible with the
national measures. EPA officials told us that they were aware of
only one state (New Jersey) that had deviated significantly from
the national Core Performance Measures, and in that instance, the
deviation was reviewed and approved by the appropriate EPA
headquarters program office. Concerns About Applying Core
Performance Measures to Nonparticipating States NEPPS is a
voluntary program and not all states have chosen to participate.
Because core performance measures are a component of NEPPS,
environmental officials in many states initially presumed that
they did not apply to nonparticipating states. EPA's intent to use
performance measure data to present a national environmental
picture, however, led the agency to request this type of data from
all states-not just NEPPS participants. Accordingly, in an October
1998 internal memorandum on EPA implementation of core performance
measures, the Acting Deputy Administrator stated that: "The
Regions are responsible for obtaining data on the Core Performance
Measures from all States (whether or not they have a Performance
Partnership Agreement with EPA) because these measures are
intended to paint a picture of environmental and program progress
across the nation." At the Environmental Council of the States'
October 1998 annual conference, states expressed concern that
EPA's policy of seeking to make Page 31
GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
States Have Made Progress in Developing Results-Oriented
Performance Measures performance measures applicable to all states
is inappropriate and in conflict with the voluntary concept of the
NEPPS program. Subsequently, joint EPA-Council guidance was issued
with the fiscal year 2000 performance measures which stated that
"[Core Performance Measures] as such only apply to states
participating in NEPPS," but added that "States not participating
in NEPPS will continue to provide key information needed by EPA
through State/EPA Agreements, grant work plans, or other operating
agreements." Relative Emphasis on Outputs Among federal and
state officials, there is a broad agreement in principle Vs.
Outcomes on the importance of measuring
outcomes rather than just outputs. A major concern among state
officials, however, has been a continued emphasis on output
measures by EPA. Ironically, many state officials maintain that
much of EPA's continued emphasis on outputs stems from the
agency's implementation of the Results Act. The Results Act
requires agencies to clearly define their missions, establish
long-term strategic goals, measure their performance against the
goals they have set, and report this information to the Congress.
The statute emphasizes the need for agencies to focus on and
achieve measurable program results, rather than focusing on the
performance of prescribed tasks and processes. Thus, EPA's goals
under NEPPS and the Results Act would appear to share the same
focus on environmental results. However, as we noted in a 1998
report on the first set of performance measures EPA prepared
pursuant to the Results Act, the overwhelming share of measures
were heavily weighted toward numerical targets and other outputs.6
Broad concern was expressed among the states that we interviewed
about the impact that EPA's implementation of the Results Act has
had on core performance measures. To varying degrees, senior level
and program management officials in five states we visited, and
EPA program officials in two regions, expressed concern about the
apparent conflict between the results-oriented performance
measures being developed under NEPPS and the generally output-
oriented performance measures EPA has thus far used to report on
the Results Act. The officials were concerned that EPA's
implementation of the Results Act is (1) maintaining an emphasis
on output rather than outcome measures and (2) adding new measures
on top of existing measures, leading to an overall increase in the
amount of data states must gather and report. 6Observations on
EPA's Annual Performance Plan for Fiscal Year 1999 (GAO/RCED-98-
166R, Apr. 28, 1998). Page 32
GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
States Have Made Progress in Developing Results-Oriented
Performance Measures EPA's enforcement program was illustrative of
states' concerns about the difficulty in moving toward outcome-
oriented performance measures. Performance measures from an
enforcement standpoint have tended to focus heavily on outputs,
such as the number of inspections conducted, the number of
significant violations detected, and how violations are handled.
Senior and program management level officials in half the states
and EPA regional offices we visited specifically cited the
relatively heavy focus of EPA's enforcement program on such
outputs as a barrier to achieving greater progress in developing
outcome-oriented performance measures. This view echoed those
expressed by state officials in our May 1998 report on EPA
enforcement efforts, which relayed concerns among most of the
state officials interviewed that EPA's Office of Enforcement and
Compliance Assurance overemphasizes output measures. We
recommended at that time that EPA ensure that the enforcement-
related provisions of EPA's Performance Plan, prepared pursuant to
the Results Act, focus on outcomes in a manner consistent with
that of the Core Performance Measures developed under NEPPS. In a
November 1998 response to our enforcement report, EPA emphasized a
number of initiatives underway, most notably its National
Performance Measures Strategy, to build in more outcome measures
in its own enforcement program and to assist states in doing so
for their programs.7 The Office also acknowledged the need to
reorient its performance plan increasingly towards outcomes and
signaled its intent to integrate some outcome measures into the
fiscal year 2000 core performance measures. The Office's fiscal
year 2000 measures list seven measures, four of which are
identified as providing outcome measures. The implementation
approach for three of the four measures is to work with volunteer
states to test the measures. In this connection, the Office has
recently announced the availability of funds for states for
projects that will improve the design and use of performance
measures for enforcement and compliance/assistance activities. In
evaluating project proposals, the Office plans to give priority to
projects designed to develop outcome measures. Progress has also
been made in other EPA programs in reorienting the agency's
Results Act measures toward outcomes. Specifically, we found
7Among the outcome measures the Office has already implemented
under this strategy are measures of improvements resulting from
EPA enforcement actions. Outcome measures currently being
implemented include (1) the average number of days for significant
violators to return to compliance or enter enforceable plans or
agreements and (2) the percentage of significant violators with
new or recurrent significant violations within 2 years of
receiving previous enforcement action. Outcome measures targeted
for implementation in October 1999 include assessments of the
levels of compliance among selected regulated populations. Page 33
GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
States Have Made Progress in Developing Results-Oriented
Performance Measures that EPA's fiscal year 2000 annual
Performance Plan, which contains the measures to be used to track
progress toward achieving its programs' goals, demonstrated some
progress since the performance plan of the previous year.8 Further
progress in coming years would help to reduce the disparity
between the generally output-oriented focus of EPA measures
prepared pursuant to the Results Act and the efforts by EPA
regions and states to focus their negotiations under NEPPS
increasingly on achieving results. Status of Core
Notwithstanding concerns among state and some regional officials
about Performance the potential impact of EPA's
implementation of the Results Act on their efforts to orient their
NEPPS-related activities toward outcomes, EPA and the Measures
Environmental Council of the States have managed to agree on a
third set of Core Performance Measures for use in fiscal year 2000
and beyond which, by most accounts, are a significant improvement
over the 1998 and 1999 measures. As both EPA and Council officials
have noted, one of the most apparent differences between the new
measures and those of past years is that the fiscal year 2000
measures are significantly fewer in number. Specifically, as shown
in table 3.1, data provided by EPA show that the number of Core
Performance Measures has been reduced from an initial set of 104
measures for fiscal year 1998 to 37 measures for fiscal year
2000.9 8Observations on the Environmental Protection Agency's
Annual Performance Plan For Fiscal Year 2000 (draft).
Specifically, we noted that among the improvements in the fiscal
year 2000 plan are goals and measures of generally better quality,
and we note some additional efforts to implement outcome measures.
Overall, however, we found that the plan still focuses heavily on
output measures. 9Such a sizable reduction reflects the efforts by
EPA and the Council to focus Core Performance Measures on what
they agreed are the most important measures. The magnitude of the
reduction, however, should be interpreted with caution for several
reasons. First, the figures reflect the temporary deletion of all
25 measures for the Pollution Prevention and Toxic Substances
Program. Work is currently underway to develop new measures for
pollution prevention and toxic substances, which are expected to
be ready for use in fiscal year 2001. Second, the dropping of a
measure as a Core Performance Measure does not necessarily mean
that data will not be gathered in response to that measure.
Rather, the inclusion or exclusion of the measure as a core
measure is an expression of its relative importance to the
national environmental picture. Third, some core measures have
multiple parts, such as "trends in air quality for each of the six
criteria air pollutants" (actually six measures) or "trends in
emissions of toxic air pollutants" (189 hazardous air pollutants
the Clean Air Act identifies). The discrete data that are
necessary to report under such measures may be aggregated or
disaggregated depending on the amount of detail used to measure
performance. Disaggregating the data increases the number of
perceived performance measures. Page 34
GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
States Have Made Progress in Developing Results-Oriented
Performance Measures Table 3.1: Number of Core Performance
Measures, Fiscal Years Program Fiscal
year 1998 Fiscal year 1999 Fiscal year 2000 1998
Through 2000 Air and radiation
23 16 10 Water
31 31 13 Hazardous waste
17 17 7 Pollution prevention and
toxic substances 25
25 0 Enforcement and compliance
8 8 7 Totals
104 97 37 Source: EPA's
Office of State and Local Relations. In addition to reducing the
number of measures to provide greater focus on what are perceived
as the most important measures, progress was also made in shifting
the proportion of fiscal year 2000 measures increasingly toward
outcomes and environmental indicators. Specifically, according to
EPA, while about 40 percent of the measures focused on outcomes or
environmental indicators in fiscal year 1998, about 60 percent of
the measures focus on outcomes and environmental indicators in
fiscal year 2000. Moreover, while EPA and Council officials are
not expected to formally vote on a comprehensive set of new
measures each year, the fiscal year 2000 measures are to be
periodically updated as deemed appropriate by EPA and the Council.
In this connection, EPA program officials, told us that they have
a number of projects currently under way (in addition to those in
the enforcement program discussed earlier) that are specifically
designed to develop additional results-oriented performance
measures. Finally, EPA and the states have also made progress
addressing the states' concern that EPA had required additional
reporting by the states to help the agency meet its data
requirements under the Results Act. Under the April 1999 Addendum
to the Joint Statement, co-signed by EPA and the Environmental
Council of the States, Core Performance Measures and other current
reporting requirements will be relied upon to satisfy EPA's
Results Act-related data needs. Conclusions
There is broad agreement among federal and state officials on the
importance of measuring the outcomes of environmental activities.
While considerable progress has been made in developing and
implementing results-oriented Core Performance Measures, a number
of challenges Page 35
GAO/RCED-99-171 Performance Partnership System Chapter 3 EPA and
States Have Made Progress in Developing Results-Oriented
Performance Measures involving technical and policy issues have
complicated the process. Progress has nonetheless been made in
developing fiscal year 2000 measures which, by most accounts, are
a significant improvement over measures used in previous years.
Continued progress in developing the measures-and the data systems
needed to support the measures-will be critical to states' and
EPA's efforts to demonstrate the efficacy of their programs under
NEPPS. In the past, it has been difficult for states to achieve
the flexibility they desire without the performance measures in
place to demonstrate that their environmental goals are being
achieved, and it will likely continue to be so in the future.
Agency Comments Citing our observations that (1) EPA has
focused on outputs to meet its obligations under the Results Act
while supporting a transition to outcome-based management under
NEPPS and (2) these conflicting priorities have led to confusion
that hinders performance partnerships, EPA said that, to the
contrary, both the Results Act and NEPPS encourage the development
of outcome measures and outcome-based management. We acknowledge
the shared objective of NEPPS and the act in focusing on results.
The key word, however, is implementation: as we have documented in
other recent work, the measures EPA has used in its implementation
of the Results Act have thus far been heavily output-oriented and,
therefore, convey priorities that are often in conflict with the
more outcome-oriented measures being employed under NEPPS. We
acknowledge EPA's ongoing efforts to orient its Results Act-
related measures increasingly toward outcomes and believe that
further progress toward this end will help to alleviate this
problem. In addition, we modified our discussion of this issue to
reflect the progress made by EPA and the states in addressing the
states' complaint that EPA had required additional reporting by
the states to help the agency meet its data requirements under the
Results Act. The chapter notes that pursuant to the April 1999
Addendum to the Joint Statement, co-signed by EPA and the
Environmental Council of the States, Core Performance Measures and
other current reporting requirements will be relied upon to
satisfy EPA's Results Act-related data needs. Page 36
GAO/RCED-99-171 Performance Partnership System Chapter 4
Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
Been Modest As originally envisioned, the principle of
differential oversight was a key element of NEPPS. Under this
principle, states with stronger environmental programs would be
accorded reduced oversight and greater autonomy over delegated
programs, thereby allowing these states greater flexibility to
manage their programs, and providing EPA the opportunity to shift
greater attention of its own resources toward weaker programs. An
important component of the concept of differential oversight was
that programs eligible for reduced oversight would meet certain
criteria and that the EPA and states would work together to choose
a group of measures to use in assessing state performance. In the
years immediately following the 1995 agreement, EPA and many
states agreed that a formal system implementing differential
oversight, whereby the merits of a state program would be
evaluated based on certain standards or criteria to determine
whether it qualifies for reduced oversight, would be both
controversial and difficult to implement. Nonetheless, the
original concept of reduced EPA oversight in exchange for
acceptable state environmental performance remains an important
goal for both EPA and participating states. Among the six states
we visited, we found instances in which some oversight reduction
was successfully negotiated between states and their corresponding
EPA regions. Such instances, however, have thus far been limited
in both scope and frequency. A number of interrelated factors were
cited as limiting the reduction of EPA oversight, including (1)
statutory and/or regulatory requirements that specify state
reporting requirements and other methods of ensuring state
accountability to EPA; (2) EPA's reluctance to reduce oversight
without measurable assurances that environmental goals are still
being achieved; (3) the inherent difficulty in "letting go" on the
part of some regulators that have implemented the existing EPA-
state oversight arrangement for several decades; and (4) the
challenge faced by EPA of communicating to states through a
complex, multilevel organization involving both headquarters and
regional offices. Initial Expectations The May 1995 joint
agreement between EPA and the Environmental Council Concerning EPA
of the States stated that "a differential approach to oversight
should provide an incentive for state programs to perform well,
rewarding strong Oversight of state programs and
freeing up federal resources to address problems Participating
States' where state programs need assistance." It added that
"after agreement is reached, EPA will focus on program-wide,
limited after-the-fact reviews Programs Page 37
GAO/RCED-99-171 Performance Partnership System Chapter 4
Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
Been Modest rather than case-by-case intervention and will work
with states to identify other ways to reduce oversight."
Accompanying differential oversight was the concept of
"performance leadership," whereby qualifying programs having a
record of strong performance would be nationally recognized with
"leadership" status. In such instances, the leadership programs
would be afforded minimum allowable oversight based on the belief
that they "deserve to be treated with deference whenever possible
and do not need federal oversight on a routine basis." In
subsequent years, however, both EPA and the states found it
difficult to implement both a formal differential oversight
process and to formally designate certain state programs as
performance leadership programs. One key problem was the inability
of EPA and the states to agree on criteria to use in making such
determinations. EPA officials responsible for NEPPS noted that
because the capacity of a state program can change depending upon
circumstances, the proper level of oversight should be determined
on a state-by-state basis by EPA regional managers-not on the
basis of specific criteria that would be universally applied to
all states. In addition, as noted by the Environmental Council of
the State's Executive Director, many state environmental leaders
expressed concern that formal designations of such programs as
performance leaders could be interpreted by EPA, state
legislatures, and the public as a "report card" of good and bad
performers. Such designations would probably be challenged,
particularly given the difficulty of developing and applying
specific criteria to use in making these determinations.
Nonetheless, the concept of differential oversight, albeit in a
less structured and visible form, remained an important component
of state and EPA regional NEPPS negotiations. Officials in the six
states told us that their early expectations for NEPPS were that
the program would help them to reduce their oversight workload in
some well-run program areas and to allow them a stronger focus on
state priorities and problem areas. Officials in three states
noted in particular that they believed the NEPPS framework would
better allow them to identify and address opportunities for
multimedia projects, rather than continuing to expend time and
resources only on the traditional, single media air, water, and
waste programs. Page 38 GAO/RCED-
99-171 Performance Partnership System Chapter 4 Reductions in
EPA's Oversight Attributable to NEPPS Have Thus Far Been Modest
States and Regional State officials cited a number of
instances in which they negotiated some Offices Report
reduction in regional oversight of their programs. These efforts
tended to focus on reducing the frequency of reporting, and in
some cases the Limited Oversight frequency of
conducting on-site reviews, in situations where both sides
Reduction Thus Far agreed such activities were
duplicative or otherwise of limited value. However, most state
program officials indicated that the extent of Directly
Attributable reporting required has either remained the
same or actually increased in to NEPPS spite
of NEPPS, and that few instances were identified where states
obtained more significant independence in operating their programs
(e.g., focusing their resources on state priorities). Most
regional staff we interviewed generally agreed that, to date,
oversight reduction attributable to NEPPS has been limited.
Instances of Reduced Officials in Maine, Florida, Georgia,
and Minnesota cited specific instances Oversight Cited by States
in which reporting requirements were scaled back, at least in part
as a and Regions result of their participation in
NEPPS. Maine environmental officials, for example, noted that more
frequent dialogue and less formal reporting between the program
staff and regional staff had replaced written reports, saving time
and improving the level of cooperation between EPA and state
staff. While Maine program officials attributed the reductions in
large part to the assignment by EPA's Boston Regional Office of a
liaison for each state's delegated programs, they credited NEPPS
with formalizing or legitimizing the changes. Florida program
officials identified sizable reporting reductions in its Resources
Conservation and Recovery Act program as a result of a joint
state/EPA effort included in the Performance Partnership
Agreement. The Chief of Florida's Bureau of Water Facilities also
noted that under the agreement, the state was able to streamline
oversight of its pretreatment program through reduced reporting
and by negotiating with the EPA Atlanta office a shifting of
resources from the conduct of routine annual inspections and
audits to other priority areas in the program.1 In some cases,
regional and state officials indicated that oversight had been
scaled back, but that such efforts could not be tied directly to a
state's participation in NEPPS. Connecticut officials reported
that quarterly reporting had been eliminated in recent years for
some of their air, water and waste programs, but attributed the
change solely to EPA regional efforts that preceded NEPPS.
Similarly, program officials in EPA's Boston, Chicago, and Seattle
offices each cited instances in which quarterly 1Under EPA's
Pretreatment Program, wastewater treatment plants are charged with
monitoring and regulating contaminant discharges by industrial
users into their sewer systems. Page 39
GAO/RCED-99-171 Performance Partnership System Chapter 4
Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
Been Modest reviews and file reviews were eliminated, but
indicated that such efforts often preceded independently of the
signing of a NEPPS agreement. Few Instances of
Notwithstanding the streamlining of reporting requirements and
similar Significant Oversight tracking efforts, the large
majority of the state officials we interviewed Reduction Under
NEPPS generally maintained that participation in NEPPS has not
yet brought about significant reductions in reporting and other
oversight activities by regional program and audit level staff,
nor has it resulted in significant opportunities to focus on other
priorities or shift resources to weaker program areas. Oregon
officials, for example, explained that their initiatives to focus
on the state's highest priorities are having difficulty competing
with their obligations to track and report on the national core
performance measures and to comply with other EPA reporting
requirements. Program managers in Connecticut, Florida, Georgia,
Maine, and Minnesota conveyed similar experiences, indicating that
the addition of new core measures to preexisting reporting
requirements had increased their reporting workload, or that they
are likely to do so in the future. Program managers in three of
these states indicated they will need to develop the data and
systems to report on the new measures. Oregon officials also
pointed to a significant increase in EPA oversight by the regional
enforcement officials of its air, water, and waste programs. EPA
Seattle officials told us that the enforcement reviews in Oregon
were the outcome of nationwide enforcement reviews by both the
Office of Enforcement and Compliance Assurance and of EPA's Office
of the Inspector General, which raised concerns about whether and
how states were bringing enforcement actions against violators.
Georgia officials also said that oversight of their hazardous
waste program has increased, noting that regional enforcement
officials were making regular monthly visits to review program
records. EPA regional program and enforcement officials generally
acknowledged that oversight of state programs has not
significantly decreased as a result of NEPPS, and that in some
cases, has increased. Officials in the Atlanta and Chicago
Regional Offices noted in particular that it may have been
unrealistic to assume, as many states had at the outset of NEPPS,
that states' participation in the program would necessarily lead
quickly to reduced EPA oversight. Moreover, regional officials
point to specific reasons why it has been difficult to scale back
EPA oversight-and why oversight has actually increased in certain
instances. Page 40 GAO/RCED-99-171
Performance Partnership System Chapter 4 Reductions in EPA's
Oversight Attributable to NEPPS Have Thus Far Been Modest Factors
Affecting We asked both state and regional officials to
identify what they believed to Potential to Reduce be the
most important considerations affecting the extent to which NEPPS
has provided states with reduced oversight, greater program
autonomy, Oversight Under and the flexibility to
emphasize their highest priorities. There was NEPPS
considerable consistency on the factors identified by both state
and EPA officials, although there was some variation on the degree
to which various factors were emphasized. The key factors include
(1) statutory and/or regulatory requirements that in some cases
prescribe the kind of oversight required of states by EPA; (2)
reluctance by EPA regulators to reduce oversight without
measurable assurances that environmental protection will not be
compromised; (3) the inherent difficulty in letting go on the part
of some regulators that have implemented the existing EPA-state
oversight arrangement for several decades; and (4) EPA's
multilevel organizational structure, which complicates efforts to
identify whether all key decision-makers among the agency's
headquarters and regional offices are in agreement on key
oversight-related questions. Statutory or Regulatory In some
cases, statutory and/or regulatory requirements may prescribe
Requirements May Limit certain types of EPA oversight,
limiting the extent to which further Options to Reduce
streamlining can be negotiated. EPA headquarters officials in the
Office of Oversight Air and Radiation noted that
some of the core performance measures for the air program are
driven by statutes and thus are non-negotiable. The officials
noted, for example, that dates by which areas in "non-attainment"
with air quality standards must come into compliance are driven by
the Clean Air Act and that EPA accordingly has no flexibility to
alter them. Similarly, a regional official cited the Clean Water
Act's requirement under section 305(b) that a Water Quality
Inventory Report be issued every 2 years. One state requested an
alternative schedule in which the state would submit its
information for the report every 5 years for each watershed area.
EPA denied the request as contradicting the 2-year frequency
required by the act. In addition, EPA Atlanta and Boston regional
staff pointed out that they have a responsibility to ensure that
new regulations, which sometimes pose particular challenges for
both federal and state regulators, are properly implemented. EPA
headquarters officials cited as an example their new regulations
concerning fine particulate matter, which required significant EPA
action during the middle of the fiscal year. Regional staff said
that such actions may inevitably require greater EPA oversight and
more detailed reporting. Officials in EPA's Atlanta Regional
Office cited another example where, in the middle of the year,
headquarters Page 41 GAO/RCED-99-
171 Performance Partnership System Chapter 4 Reductions in EPA's
Oversight Attributable to NEPPS Have Thus Far Been Modest
implemented a new initiative that required the region to ask the
states to do additional inspections of metal finishing plants that
went beyond the commitment made by states in their Performance
Partnership Agreements. State program managers acknowledged that
statutory and regulatory requirements do in fact sometimes limit
the potential to reduce EPA oversight. In addition, while
welcoming the administrative relief and flexibility allowed under
the Performance Partnership Grant Program, several noted that the
implementation of these grants is still governed by certain
statutory and regulatory requirements. For example, the grants are
still subject to certain grant administrative requirements and
cost accounting standards applicable to federal grants generally.
Specifically, while the Partnership Grants do not require the
detailed accounting required of categorical grants, states must
still report to EPA on how funds have been spent under the broader
categories. Furthermore, like other federal grants, the EPA grant
agreements are supposed to include adequate oversight procedures
to provide EPA assurance that federal funds are used efficiently
and effectively. Perhaps more significantly, both state and
regional officials added that the state programs are still held
accountable for accomplishing program commitments outlined in
their work plans and that base program requirements under the
various statutes must still be met. Such competition for limited
resources to meet the requirements of individual statutes has, in
fact, been a long-standing issue that has complicated efforts to
shift attention and resources to what are perceived as the highest
environmental priorities. We noted in our 1988 general management
review of EPA, for example, that the objective of setting risk-
based priorities across environmental media has been complicated
by the fact that each statute prescribes certain activities to
deal with its own medium-specific problems.2 In 1991, we touched
on the issue again noting, for example, that numerous legislative
mandates have led to the creation of individual EPA program
offices that tended to focus solely on reducing pollution within
the particular environmental medium for which they have
responsibility, rather than on reducing overall emissions.3 More
recently, in testifying on efforts by EPA to improve its working
relationship with the 2Environmental Protection Agency: Protecting
Human Health and the Environment Through Improved Management
(GAO/RCED-88-101, Aug. 16, 1988). 3Environmental Protection:
Meeting Public Expectations With Limited Resources (GAO/RCED-91-
97, June 18, 1991). Page 42
GAO/RCED-99-171 Performance Partnership System Chapter 4
Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
Been Modest states and to provide them with additional
flexibility,4 we concluded that as long as environmental laws are
media-specific and prescriptive and EPA personnel are held
accountable for meeting the requirements of the laws, it will be
difficult for the agency to fundamentally change its relationships
with the states to reduce day-to-day control over program
activities. EPA Reluctance to Reduce Program managers and staff
in all four of the EPA regional offices we Oversight Without
visited questioned the extent to which the agency can reduce
oversight Measurable Assurances without measurable
assurances that program requirements, and That Environmental
environmental objectives, will be achieved. The issue has become
Protection Will Not Be particularly pronounced in the
enforcement program, where some states have taken issue with what
they perceive to be heavy-handed oversight by Compromised
EPA. Among state officials' complaints are that EPA enforcement
officials inappropriately hold states accountable for the number
of enforcement actions (outputs) taken rather than achieving
better environmental compliance (outcomes). Some states have also
cited the prospect of EPA taking direct enforcement action in
states where the lead state environmental agency has primary
enforcement authority, or of "overfiling" with an EPA action in
instances where a state enforcement action was determined by EPA
to be insufficient. State officials have also maintained that such
a posture is inconsistent with the philosophy under NEPPS that EPA
should focus its oversight on results and should provide states
with greater flexibility as to how to achieve those results. EPA
Seattle regional officials, however, have cited the Office of
Enforcement and Compliance Assurance's recent reviews and those of
the Office of Inspector General, which have concluded that (1)
many states have underreported violations by dischargers of
pollutant limitations and other environmental requirements and (2)
the numbers of enforcement actions taken by state enforcement
officials has declined. These reports, the officials contend,
raised questions about the ability of states to achieve compliance
by the regulated community without vigilant federal oversight.
Moreover, according to the officials, states presently do not have
the data to support their contentions that environmental
compliance is still being achieved in cases where their
enforcement activity has been curtailed. 4Environmental
Protection: Status of EPA's Initiatives to Create a New
Partnership With States (GAO/T-RCED-96-87). Page 43
GAO/RCED-99-171 Performance Partnership System Chapter 4
Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
Been Modest State officials told us, both during this review as
well as during our 1998 review of state enforcement programs,5
that the absence of measurable results complicates efforts to use
more flexible approaches-not just because it is harder to get EPA
approval, but also because it is harder to obtain the confidence
of the media and the general public. Florida officials, for
example, told us that the number of penalties assessed, and dollar
value of penalties collected, under its federally delegated
programs decreased from 1994 to 1996, and that questions were
raised as to whether these decreases resulted, at least in part,
from a greater emphasis on the use of assistance to achieve
compliance. In fact, newspapers in the state subsequently
published articles questioning whether the state was letting
violators continue to pollute without fear of punishment. Florida
officials told us that their major investment in measuring the
results of their enforcement and compliance assistance efforts was
undertaken, in part, to determine whether these concerns were
well-founded. The Office of Enforcement and Compliance Assurance
points out that it is addressing the problem through its National
Performance Measures Strategy and by collaborating on the
development of enhanced outcome-oriented performance measures with
a number of states. In addition to helping states develop outcome
measures, enforcement officials also pointed to recently-issued
guidance that encourages EPA regional offices to be more flexible
in considering states' preferences when negotiating regulatory
priorities.6 Specifically, the guidance calls on regions to
"develop their priorities in partnership with their states . . ."
and notes, "States are not required to adopt EPA's national
priorities . . . This guidance provides flexibility for both
regions and states to identify and implement their own
priorities." The guidance further states that EPA is "addressing
states' concerns about joint planning and priority-setting, work
sharing, and oversight responsibilities by identifying this as a
management focus area to be addressed by each region in the fiscal
year 2000/2001 [memorandum of agreement] process." 5Environmental
Protection: EPA's and States' Efforts to Focus State Enforcement
Programs on Results (GAO/RCED-98-113, May 27, 1998). 6EPA Office
of Enforcement and Compliance Assurance, Final FY 2000/2001 OECA
Memorandum of Agreement (MOA) Guidance (Apr. 1999). Page 44
GAO/RCED-99-171 Performance Partnership System Chapter 4
Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
Been Modest Resistance to Change at Our 1997 report on EPA's
efforts to "reinvent" environmental regulation Lower Levels Within
Both observed the widely held view, both within and outside
EPA, that achieving EPA and Among State a full commitment
to reinvention by EPA staff will be difficult and will take
Agencies time.7 The report further identified
widespread agreement among EPA officials, state officials, and
others that the agency has a long way to go before reinvention
becomes an integral part of its staff's everyday activities, and
cites a senior EPA reinvention official as noting that "many staff
are comfortable with traditional ways of doing business and
consider their program-specific job responsibilities as their
first priority and reinvention projects as secondary." Many of the
state officials we interviewed contended that comfort level among
some EPA staff with the preexisting oversight arrangement-which
has generally been in place for many years-helps to explain the
reluctance by many of them to provide states with greater
flexibility and reduced oversight. Program officials in five of
the six states provided examples where they believed that regional
program staff (tasked with the day-to-day implementation of
specific programs) asked for information that was not included in
the Partnership Agreement or that they had previously agreed with
the region to drop. Minnesota officials said that EPA regional
waste officials were asking for predictive or target numbers (such
as the number of inspections the state intends to pursue during
the coming year)-information, they said, that was not required nor
included in their Partnership Agreement. Similarly, Georgia
program officials said that EPA enforcement officials requested
additional information after their Partnership Agreement had been
negotiated and was ready to be signed. Georgia's Assistant
Director and the Atlanta Deputy Regional Administrator,
recognizing that the difficulty was due in part to different
targets and schedules for enforcement and the media programs, set
up an enforcement planning work group consisting of state and
regional representatives from enforcement and the media programs
to study and resolve the problem so that they could avoid last
minute changes in the future. Other state officials told us that
EPA has recently requested information related to the Results Act
which, they believed, was outside the scope of their agreements.
Several state officials commented that an openness toward seeking
ways to reduce such information requests appears to be greater
among senior EPA regional managers than among lower-level staff.
7Environmental Protection: Challenges Facing EPA's Efforts to
Reinvent Environmental Regulation (GAO/RCED-97-155, July 2, 1997).
Page 45 GAO/RCED-99-171
Performance Partnership System Chapter 4 Reductions in EPA's
Oversight Attributable to NEPPS Have Thus Far Been Modest It is
possible that what state officials may view as an "resistance to
change" could be regarded by EPA staff as a well-founded concern
that program requirements be implemented properly and in
accordance with laws and regulations. However, officials in three
of the four EPA regions we visited nonetheless acknowledged that
support for NEPPS within EPA varies. One senior regional official
said that managers and staff are often more comfortable with the
preexisting way of doing business and are unsure as to how they
can accomplish their work in the context of the partnership
approach under NEPPS. He voiced the opinion that there may be a
need for training in NEPPS implementation among regional staff.
Another senior regional official said that some staff will only
take NEPPS seriously when their salaries are tied to their
performance in implementing the program. By the same token, our
interviews with senior state officials suggest that cultural
change is also needed at the state level if NEPPS is to achieve
its full potential. Specifically, several state officials said
that state program managers may not always be well-versed in
recognizing opportunities that would allow them to exercise their
responsibilities with greater flexibility. Some of them indicated
that there is resistance to NEPPS at the state program manager and
staff level because of the perceived threat to their programs. In
one state, in order to get the program directors' support for
participating in NEPPS, senior management made a commitment not to
make any large-scale shift of funds among or between programs.
Some regional staff and managers also commented that states have
not taken advantage of opportunities to seek more flexibility
under NEPPS, noting in particular that none of the states in their
regions attempted to move significant amounts of funds among
programs or across media lines. Challenges in EPA's
organizational structure poses additional challenges in
negotiating Communicating agreements that have the
full buy-in of all key EPA decision-makers. Requirements Through a
Headquarters interaction with the states is generally conducted
indirectly Multi-Level EPA through the regional
offices. National Program Managers set national Organizational
Structure strategic direction, and core program requirements
and priorities, for each of their environmental programs. The
managers establish overall national goals for their respective
programs based on a variety of factors, including the underlying
statutory mandates, congressional directives,
administration/administrator priorities, and their own view of
programs and policies that their programs should focus upon. The
managers also must develop an accountability system to ensure
program delivery by EPA's regions. The regional offices consult
with managers in determining national priorities and communicate
these priorities to the states. As such, Page 46
GAO/RCED-99-171 Performance Partnership System Chapter 4
Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
Been Modest the regional offices serve as the key EPA focal point
in negotiating with states on program priorities and oversight
arrangements to be reflected in NEPPS agreements. Importantly, the
states generally have little direct communication with the
managers. Thus, for example, if states wish to deviate from a
national core performance measure or priority, it is the regions
that consult with the managers. Buy-in by Key Decision Makers
As a consequence of this structure, according to the majority of
state and Mixed Messages Confuse program managers we
interviewed, it is not always clear that a Partnership States
Agreement between the state and the region has the full buy-in of
EPA's key headquarters managers. A senior official with Florida's
Department of Environmental Protection cited the example of the
state's "Joint Compliance and Enforcement Plan," negotiated under
the state's 1998-1999 Performance Partnership Agreement. Under the
plan, state and regional officials enter into a process that seeks
agreement, on the basis of industry compliance data, on what the
state's most important compliance problems are and which methods
(e.g., enforcement action, technical assistance) are most
appropriate to address them. The official said that while the
state has already invested significant time and effort into the
plan, and has had expressions of strong support from EPA's Atlanta
Regional Office, it does not know the extent to which EPA's
headquarters Office of Enforcement and Compliance Assurance
supports the effort or whether that office will ultimately give
its approval. Similar observations were made by other states'
officials, who indicated that provisions were sometimes added at
the request of EPA headquarters to Partnership Agreements after
they were negotiated. The timing of headquarters guidance and
special requests for input into Partnership Agreements was cited
by some state and regional officials as a key factor: final
headquarters guidance, or specific requests in some cases, often
come too late to be included in regional and state negotiations,
causing the need for some agreements to be renegotiated.8 State
officials also indicated that some headquarters requirements are
negotiated separately from the overall Partnership Agreement
negotiations. Officials with the Minnesota Pollution Control
Agency told us that after successfully negotiating its agreement
with regional program officials, the Office of Enforcement and
Compliance Assurance requested separate measures and a separate
section apart from the media programs 8In response to a 1997
survey by the Environmental Council of the States, participating
states commented that headquarters guidance should be finalized by
February of each year so that states and regions can meet in March
to set joint priorities and begin the Performance Partnership
Agreement process for the following fiscal year. Page 47
GAO/RCED-99-171 Performance Partnership System Chapter 4
Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
Been Modest in the Partnership Agreement. In the opinion of the
state officials, this process illustrated the difficulty in
getting all headquarters interests incorporated into the agreement
in a timely fashion. Officials in several other states cited
similar circumstances where enforcement provisions had to be
negotiated outside the scope of the Performance Partnership
Agreement, making it difficult to develop the kind of integrated
environmental program NEPPS is intended to encourage. EPA
Officials Acknowledge Officials in the four regional offices we
visited told us that sometimes Need for Clearer and More there
are inconsistencies between headquarters and regional offices,
Consistent Communication which complicates the message the
agency sends to the states. Boston regional officials cited one
instance in which Maine and Connecticut had proposed to
consolidate funds for their wetlands programs (1 of the 15
eligible programs) under a performance partnership grant and were
initially told by the regional office that the arrangement would
be acceptable. However, EPA's headquarters Water Office
subsequently objected to allowing all funds to be shifted from a
categorical grant to a Performance Partnership Grant on the basis
that a portion of the funds were supposed to be used in a
competitive bid process for nonprofit organizations (and other
eligible parties) to propose special projects. According to state
and EPA Boston regional program managers, EPA's Boston Regional
Office resolved the resulting confusion by brokering an agreement
to allow for some funding from each of the New England states'
wetlands grant programs to be set aside for special regional
wetlands pilots. Many EPA regional officials said that
headquarters officials sometimes view NEPPS negotiations as a
regional-state matter, and that headquarters offices do not view
themselves as "signatories" to the process. The officials noted
that it is only when there is a significant deviation on the part
of the state from a national priority that headquarters may become
involved with decisions related to NEPPS agreements. Most of the
headquarters managers that we interviewed acknowledged that EPA
headquarters input into the NEPPS negotiation and agreements
process is primarily left for the regions, to convey to the
states, with headquarters primarily engaged in setting the
national priorities and issuing national program guidance. These
headquarters managers acknowledged that headquarters input into
the NEPPS process can be improved, noting in particular that
headquarters guidance, initiatives, and special requests sometimes
arrive at the regions too late to be useful. In April 1999,
headquarters managers issued 2-year program guidance to help Page
48 GAO/RCED-99-171 Performance
Partnership System Chapter 4 Reductions in EPA's Oversight
Attributable to NEPPS Have Thus Far Been Modest address some of
the problems related to untimely requests. EPA intends that this
guidance will allow the regions and states to include national
program priorities earlier in the negotiating process for
Performance Partnership Agreements. At the same time, however, the
managers said certain circumstances that could affect a signed
agreement, such as those dealing with new regulations, are
sometimes out of their control. Agency Comments EPA provided
updated information about the concern that headquarters program
guidance often arrived too late to be of use in Performance
Partnership Agreement negotiations between states and their EPA
regional offices. Specifically, the agency noted that in April
1999, its headquarters National Program Managers issued 2-year
program guidance to the regional offices simultaneously and on
schedule, so that the information would be available prior to
Performance Partnership Agreement negotiations. EPA said that the
new procedure has been well received by the regional offices, and
that the introduction of 2-year guidance will allow regional
offices and states to extend their planning horizon without fear
that the priorities of the National Program Managers will change
dramatically on an annual basis. We have amended this chapter to
reflect this progress. At the same time, the chapter still conveys
agency officials' views that the guidance will not necessarily
prevent other circumstances, which are out of EPA's control, from
necessitating the reopening of an agreement. EPA also cautioned
that the report should more clearly distinguish between the terms
burden reduction and differential oversight. Burden reduction,
according to EPA, applies to activities, particularly information
exchanges, that both EPA and a state agree are unnecessary,
duplicative, or inefficient. In such cases, EPA believes that all
state programs should benefit from burden reduction. The term
differential oversight, according to EPA, means that oversight may
vary depending on how effectively a state program meets
performance expectations. The EPA comment draws a clear
distinction between issues associated with reporting burdens and
other issues that are more appropriately viewed as related to
EPA's oversight of state environmental programs. We acknowledge
that there are circumstances, separate and apart from EPA
oversight, in which EPA and a state collaboratively pursue
strategies to reduce reporting requirements that they both agree
are unnecessary, duplicative, or inefficient. However, the
distinction between this activity and oversight is not always so
clear. Specifically, in cases where states and EPA have disagreed
on the need for data not required by statute and viewed by states
as extraneous, and EPA Page 49
GAO/RCED-99-171 Performance Partnership System Chapter 4
Reductions in EPA's Oversight Attributable to NEPPS Have Thus Far
Been Modest has continued to require reporting of such data,
states have often characterized the issue as, in their view, a
questionable exercise of EPA oversight. Page 50
GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
of NEPPS Participation Cited, but Full Potential Has Yet to Be
Realized State participants' expectations for reduced EPA
oversight and greater program flexibility-major anticipated
benefits at the outset of NEPPS in 1995-have thus far met with
some disappointment. Yet while these participants expressed
disappointment at the rate of progress in achieving greater
autonomy and greater emphasis on state priorities, senior
officials and program managers from each of the six states that we
reviewed agreed that NEPPS has provided their programs with
worthwhile benefits. Among the benefits most frequently cited were
that NEPPS (1) provided a means of getting buy-in for innovative
and/or unique projects, (2) allowed states the option to shift
resources and funds under the Performance Partnership Grants
Program, (3) served as a tool to divide a burdensome workload more
efficiently between federal and state regulators, and (4) improved
communication and increased understanding among EPA and state
program participants about each other's program priorities and
other key matters. Officials in each of the four regions visited
substantially agreed with many of the benefits of NEPPS
participation cited by state officials. Yet while participants
from each state indicated that their participation in the
voluntary program would probably continue, they also shared a
consistent opinion that the benefits of the program should be
greater, that the program has yet to achieve its potential, and
that improvements are needed. To some extent, such an outcome
should not be surprising, given that the program (1) has been in
place for just a few years and (2) began as an experiment in which
participants were encouraged to try different tools and
techniques. Yet these early years of the program have also
provided a wealth of experiences as to what has worked well, what
has not worked, and how the program can be improved. The 1995
agreement anticipated the appropriateness of such reflection in
calling for a joint evaluation system for EPA and the states to
review the results of their efforts to ensure continuous
improvement. On the basis of our work, we believe that it is now
appropriate to begin such a joint evaluation process. Program
State officials in each of the six states we visited identified a
number of Improvements benefits to their air, water, and
waste programs, but frequently spoke of some benefits, such as the
ability to move funds toward the state's highest Attributed to
NEPPS priorities, as potential future benefits rather than as
benefits already realized. EPA regional staff acknowledged many of
the benefits identified by state participants, but were often
cautious in stating that additional flexibility could be exercised
only so long as states continue to meet the statutory and
regulatory requirements associated with their base Page 51
GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
of NEPPS Participation Cited, but Full Potential Has Yet to Be
Realized programs. Benefits identified related to the flexibility
to work on innovative and special projects; to use resources and
gain administrative efficiencies through the consolidated
environmental grant; to more efficiently divide the workload among
EPA and state regulators; and as a means of improving public
outreach and involvement in environmental policies and programs.
The additional benefit most frequently cited by state officials is
perhaps the most intangible one-that it helped to encourage a more
systematic and effective communication between EPA and state
officials on key issues and priorities, leading to increased
mutual understanding and improved relations. Although many of
these officials acknowledged that this progress has not yet
resulted in the more equal partnership with EPA to the extent
hoped for, the collaboration and negotiation fostered by the
process was viewed as a definite step in the right direction.
NEPPS Provides a Means The majority of EPA regional and state
officials we contacted cited the of Getting Buy-in for
ability to work on, and get buy-in for, innovative and/or unique
projects Innovative And/or Unique (such as those dealing with
cross-cutting issues or multimedia projects) as Projects
a tangible benefit under NEPPS. Among the examples cited was a
Quality Assessment Management Plan included in Florida's fiscal
year 1999 Performance Partnership Agreement, signed between the
state's Department of Environmental Protection and EPA's Atlanta
Regional Office. Once fully developed and implemented, the plan is
expected to provide the state with the ability to identify and
improve the quality of data provided by private laboratories. The
Florida project director spearheading the effort on behalf of the
state said that elevating the project as a priority in the
Partnership Agreement legitimized the concept and gained the
support of key EPA and state decision makers. The prototype or
model of the plan has been completed and submitted to state and
regional officials with the expectation that the project staff
will next move on to issues related to implementation. According
to the project director, the Partnership Agreement-as a document
signed by the senior officials at both the state and federal
level-was crucial in conveying top management buy-in. The project
director observed that the Agreement, in effect, provided the
"impetus to innovate" whereby state and regional leadership
formally endorsed a new way of doing business. Environmental
officials in Minnesota recently reorganized the state's pollution
control agency to eliminate its media-specific structure. The new
organization has three geographic divisions to handle most
environmental Page 52 GAO/RCED-
99-171 Performance Partnership System Chapter 5 Benefits of NEPPS
Participation Cited, but Full Potential Has Yet to Be Realized
issues and two divisions to handle environmental planning and
outcomes. The reorganization was undertaken because they believed
that an integrated approach to environmental management was needed
and because many problems transcend media boundaries. Agency
officials noted that the Performance Partnership Agreement between
the state and EPA's Chicago Regional Office was key to
establishing a new working relationship with EPA and to
Minnesota's efforts to find a better way to plan and carry out
their work. They added that the Partnership Agreement provides the
state the flexibility to go beyond reporting on media-based
program outputs towards linking, tracking, and measuring agency
activities with actual environmental results. Among other examples
cited, a program manager in EPA's Atlanta Regional Office pointed
to North Carolina's effort to use its Performance Partnership
Agreement to pursue a multimedia inspection project for metal
finishing plants. The inspections are conducted jointly from an
air, water, and waste perspective so that each media program does
not have to do its own separate inspection. The Partnership
Agreement provided program managers in the state environmental
agency with a recognized vehicle to propose and implement the
inspection initiative to share resources across media lines by
getting a formal buy-in from state and EPA officials through a
signed agreement. Flexibility to Shift As noted earlier in this
report, Performance Partnership Grants allow Resources and Funds
states the opportunity to combine individual categorical grant
funds into a Under NEPPS Grant consolidated grant. Once
included in the consolidated grant, the funds Agreements
essentially lose their category-specific identity and can be used
with considerably greater flexibility. Environmental agencies
within four of the six states included in our review (Maine,
Connecticut, Georgia, and Minnesota) have Partnership Grants with
their corresponding EPA regional offices. Importantly, officials
in these states told us that they have not been able to take
greater advantage of the ability to shift funds, primarily because
the programs covered by the Partnership Grant each have their own
base program requirements that must be funded.1 However, several
of the officials told us that the flexibility allowed under a
Partnership Grant to move funds where they are most needed remains
an important potential benefit of the program. For example, a
Georgia official said that they hoped to shift 1At a workshop
sponsored in July 1997 by the Environmental Council of the States,
two states reported plans to shift between 5 and 15 percent of the
funds under their Performance Partnership Grants to address
priorities such as pollution prevention. Also, one state in a 1997
Council survey of the Performance Partnership Agreement process
reported setting aside 5 percent of its funds from water grants to
address wetlands lakes and a new Performance Partnership Grant
coordinator position. Page 53
GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
of NEPPS Participation Cited, but Full Potential Has Yet to Be
Realized funds sometime in the future to address nonpoint sources
of water pollution and air quality in metropolitan Atlanta. This
official added that if an emergency were to arise, the Partnership
Grant would allow the state to move funds and staff quickly from
various programs to address the problem. A grant official with
EPA's Boston office noted that prior to the Partnership Grant
program, states in the region often complained about their
inability to shift funds from programs that had excess funds to
other programs that were short of funds. He noted that such
complaints have declined with the inception of the program.
Program officials in all four of the case study states having
Partnership Grants also cited administrative efficiencies from the
ability to consolidate their categorical grants. The officials
noted that the grants have allowed states to condense individual
work plans into a single consolidated work plan, and states have
gained additional flexibility in the way they account for staff
time. State environmental agency officials noted that they were
able to reduce the number of grant applications, budget documents,
and work plans required. Some added that they gained
administrative relief from not having to track staff time and
charges on a detailed, grant-by-grant basis. A Maine official, for
example, noted that under the traditional categorical grant
process, staff positions funded by multiple categorical grants
required controls to be in place to ensure that employees charge
their time to specific grants and budget categories. The
Partnership Grants provide the flexibility to accomplish necessary
work without worrying about which tasks are funded by which
categorical grants. EPA regions' responses to this increased
flexibility have been mixed. Regional program managers in the four
regions visited expressed concern about the flexibility of the
NEPPS agreement and grants process and said that there is a need
to retain or develop new state reporting requirements if EPA is to
retain proper program oversight. These program managers commented
that eliminating reporting requirements results in EPA losing its
ability to hold states accountable and argued for states to
provide predictive annual targets as to what they plan to
accomplish and to develop short-term or interim measures for
reporting states' progress towards measuring environmental
results. Several managers said that it is important for the states
to prove that work is actually being done and cited the
consolidation of grant funds under a Partnership Grant as an
example where EPA loses a level of control. Other regional program
managers, however, were more optimistic and comfortable with the
fact that states provide year-end reports on what they have done.
Page 54 GAO/RCED-99-171
Performance Partnership System Chapter 5 Benefits of NEPPS
Participation Cited, but Full Potential Has Yet to Be Realized A
Tool to Divide a In 1997, we reported on EPA's and states'
efforts to improve their Burdensome Workload management of
Superfund site cleanups,2 and cited innovative efforts in
Efficiently Between Minnesota and Washington where state and
regional officials experienced Federal and State substantial
efficiencies through work-sharing agreements. In Washington,
Regulators state and EPA officials reported that under
a formal written agreement signed by officials in EPA's Seattle
office and the state's Department of Ecology, responsibility was
formally divided for cleaning up the state's National Priority
List sites between the two agencies. Both EPA and state officials
reported that the formal, clearly articulated division of
responsibility between the two parties helped to reduce both the
acrimony and the duplication of effort that characterized their
past relationship. The state official reported a strong consensus
among the staff that the changes contributed to a significant
reduction in the number of staff resources needed to oversee
cleanups at NPL sites. Minnesota officials and Superfund officials
with EPA's Chicago office reported similar success with such a
work-sharing agreement. State and EPA regional officials cited
similar benefits of Partnership Agreements, as formal documents
that clearly articulate the obligations of both parties to the
agreement. State officials noted that in some instances in the
past, communication seemed to be one of EPA conveying its
expectations of the state, rather than the two-way communication
embodied in many Partnership Agreements. Even where the concept of
dividing responsibilities and identifying work-sharing
opportunities has been used, state officials indicated that a
formal Partnership Agreement brings a commitment and focus to the
need to share scarce resources and to formalize stated
commitments. Program managers in several of the regions and states
we visited cited a number of examples that illustrated the
benefits associated with the formal division of labor memorialized
in a Partnership Agreement. Connecticut's NEPPS coordinator, for
example, said that the Connecticut Department of Environmental
Protection negotiated with EPA's Boston Regional Office to pick up
some of the state's training work load, because EPA could provide
joint training for all the New England States at a lower cost than
would be the case if each state provided training individually.
The Connecticut Partnership Agreement specifically states that EPA
agrees to assist with training in several areas, such as measuring
and documenting the success of the state's compliance assistance
and enforcement activities. The state's fiscal year 1999 agreement
also 2Superfund: Stronger EPA-State Relationship Can Improve
Cleanups and Reduce Costs (GAO/RCED-97-77, Apr. 24, 1997). Page 55
GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
of NEPPS Participation Cited, but Full Potential Has Yet to Be
Realized documents coordination with the Boston office, indicating
that the region agrees to work with the state on helping to reduce
the state's reporting burden. According to the agreement, the
region was to assume some of the state's inspection workload or
streamline inspection requirements in order to free up state staff
resources for compliance assistance activities. Georgia's
Partnership Agreement includes provisions for EPA's Atlanta office
to assist the state in training, enforcement, and inspection
activities generally on an "as requested" basis. For example, the
region provided some expertise to the state and committed to
dedicating EPA resources to the training of compliance officers
within the state. Oregon's Partnership Agreement was similarly
used to address unmet needs in the state's water program.
Officials with EPA's Seattle Regional Office and the state's
Department of Environmental Quality agreed that the state's
program to identify and remediate heavily polluted waters was
understaffed and underfunded. Under the agreement, the regional
office agreed to provide the state with two staff to assist in the
program. Opportunity to Improve A key intended benefit and one
of the seven principal components of NEPPS Public Outreach and
in its May 1995 joint agreement is the opportunity to share
information Involvement with the public on state
environmental conditions, objectives, and performance. Officials
with the Environmental Council of the States commented that public
participation is a strong point of the NEPPS program-something
that rarely occurred under the formal traditional system where
public comment was generally sought on specific facilities or
sites only. At the time of the 1995 agreement, some states had
begun to share such information through their annual state of the
environment reports. The NEPPS process, however, offered greater
opportunities for constructive public involvement. EPA and state
officials told us that increased public participation and
involvement remains a principle benefit of the EPA-state NEPPS
process, but its full potential is largely unmet. State officials
have found that public interest and input into the NEPPS process
has varied but that, overall, it has thus far tended to be
limited. Minnesota officials, for example, said that they sent out
a press release and copies of their Partnership Agreement to about
400 entities comprised of industry, environmental, community, and
tribal groups and received only a handful of comments. Georgia
sought comments on its Partnership Agreement at a public meeting
and received limited comments, and Connecticut held an evening
meeting with an advisory board consisting of representatives for
the different media and Page 56
GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
of NEPPS Participation Cited, but Full Potential Has Yet to Be
Realized similarly obtained little feedback. In general, regional
and state officials said that it will take time to increase the
public's understanding and interest in focusing on the states' and
EPA's long-term environmental goals and performance, rather than
only on specific activities or conditions of more immediate
concern. Improved Communication Nearly all EPA regional and
state officials that we interviewed said that a Among Participants
About key benefit of NEPPS has been improved communications
among program Program Priorities and participants and the
fostering of a better federal-state working Other Key Matters
relationship. Members of the EPA Chicago Regional Office's NEPPS
coordinating committee (which represents all media and enforcement
programs), said that NEPPS has provided the region with a better
understanding of states' strategic plans, which has assisted the
states when negotiating a change with the region. In addition,
NEPPS has encouraged regional and state staff of all media
programs to discuss their programs jointly, a practice that has
helped program officials at both the state and regional level gain
a better understanding of each other's needs. According to state
and regional officials, this higher level of understanding has
been a major factor that has helped them to improve the way they
set priorities across programs. Headquarters enforcement officials
also point to regional efforts to try to use NEPPS as a vehicle to
more actively engage the states in joint enforcement planning and
priority-setting. EPA's Boston Regional Office, for example,
systematically arrayed a number of multimedia enforcement and
compliance assistance programs for discussion and possible
incorporation in states' fiscal year 2000 Performance Partnership
Agreements. In each case, the priority the agency attaches to the
program is indicated as well as the type of collaboration EPA
anticipates having with the states.3 Similarly, the officials
cited as another example a Chicago Regional Office's analysis of
its Performance Partnership Agreement with Minnesota which
describes, on a media-specific basis, the state's and EPA's
commitments to participate in mutually agreed-upon enforcement and
compliance assurance activities to realize jointly determined
environmental objectives. Senior officials and program managers in
all six states we visited also agreed that the NEPPS process has
improved EPA-state communication and overall relations. Many also
noted that NEPPS highlights and enhances 3EPA's Boston Regional
Office, "Assistance and Pollution Prevention Programs & Priorities
For Fiscal Year 2000 State/EPA Performance Partnership Agreement
(PPA) & Compliance Strategy Discussions" Page 57
GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
of NEPPS Participation Cited, but Full Potential Has Yet to Be
Realized communication among their own state media programs, as
well as among EPA regional media programs. Noting that improved
communications can solve 95 percent of their state-regional
problems, Minnesota officials have instituted routine monthly
conference calls with EPA's Chicago Regional Office to address
waste issues and are considering implementing the same process for
their other media programs. EPA Chicago Regional Office officials
told us that they are also relying increasingly on oral
communications with their states in an effort to encourage a more
collegial and efficient approach to resolving problems. Future
Prospects for Officials in each of the case study states that
we interviewed agree that the Success Depend on concept
behind NEPPS, and its potential for achieving a more effective
partnership between EPA and the states, is worth pursuing. Yet
while Further Progress acknowledging some benefit from
their participation, they also consistently expressed the view
that the benefits should be greater; that the program has yet to
achieve its potential; and that improvements are needed. Of
particular note, providing states with the incentives envisioned
initially under NEPPS, including the differential oversight as
discussed in chapter 4, was seen by almost all of the state
officials we interviewed as critical to the future success of the
program. This view is reinforced by the resource commitment that
some states feel has been required to take part in the program.
Oregon officials, for example, said that they invested a
significant amount of their resources in conducting a state
environmental self assessment and other activities to participate
in the NEPPS-Performance Partnership Agreement process. To date,
however, these officials noted that they have not gained the
advantages of reduced oversight leading to increased self
management of their delegated programs and greater autonomy to
focus on state priorities. Similarly, in explaining a major reason
for their decision not to participate in the program, the Deputy
Director of Michigan's Department of Environmental Quality noted
that the heavy investment cited by participating states and the
modest benefits achieved by those states has led to the
Department's decision to wait and see how NEPPS evolves. The
Deputy Secretary of Pennsylvania's Department of Environmental
Protection had similar reasons for that Department's
nonparticipation, noting that the department had several state
initiatives underway that were important and, therefore, they
would be reluctant to shift resources to NEPPS. The Deputy
Secretary said that Pennsylvania is reserving judgment as to its
future participation in NEPPS, noting that if greater Page 58
GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
of NEPPS Participation Cited, but Full Potential Has Yet to Be
Realized progress and benefits under NEPPS accrue over time, it
may become advantageous for the state to participate. For their
part, EPA officials acknowledge the states' desire for greater
program flexibility and autonomy, but believe they are not in a
position to grant it unconditionally. Specifically, the officials
maintain that additional program flexibility will have to be
accompanied by demonstrated, measurable assurances that statutory
and regulatory requirements and program objectives will still be
met. As we noted in chapter 4, both EPA and state officials have
pointed to the difficulty of developing specific, nationwide
criteria to be used in determining the appropriate level of
regional oversight of state programs under NEPPS. However, given
the importance to the program's future of making progress on this
issue, it may be helpful for EPA and state officials to
collaborate in developing some type of non-binding guidance that
could be used in guiding the negotiations of individual regions
and states on this sensitive issue. In addition to this overriding
concern about oversight, NEPPS participants believe that the
benefits that have accrued from their participation in NEPPS have
not reached their full potential. For example, many participants
have noted improvements in communication under NEPPS, but said
that further improvements are needed to ensure that all key EPA
offices provide timely input into Partnership Agreement and
Partnership Grant negotiations to help state agencies understand
whether their agreements have full buy-in of all EPA offices.
Similarly, while Partnership Grants allow for greater flexibility
in shifting funds among media programs, states have thus far taken
advantage of this opportunity to only a limited degree. To some
extent, the base program requirements under individual programs
combined with financial constraints have limited states'
flexibility in shifting funds as freely as they would like.
However, other factors may explain the problem as well, including
specific grant regulations, resistance by EPA headquarters and/or
regional staff, or similar resistance among state agencies
themselves. Joint EPA-State Evaluation These concerns pose
challenges for the future of the program. However, Process Needed
to we believe such challenges are to be expected in
the context of a new Improve NEPPS program that
strives to chart a new direction in the EPA-state relationship.
Importantly, the need to address such challenges was anticipated
by the 1995 Agreement that launched the program, which called for
a joint Page 59 GAO/RCED-99-171
Performance Partnership System Chapter 5 Benefits of NEPPS
Participation Cited, but Full Potential Has Yet to Be Realized
evaluation system for EPA and the states to review the results of
their efforts to ensure continuous improvement. To some extent,
such a joint evaluation process was undertaken to produce the core
performance measures. The intergovernmental committees that
developed these measures, composed of representatives of EPA and
state agencies, produced an initial set of measures for fiscal
year 1998 that was modified and improved in subsequent years. As
noted in chapter 3, the measures approved for fiscal year 2000 are
widely viewed as substantially improved by both EPA and state
officials. EPA's and states' recent efforts to improve their
working relationship in cleaning up priority Superfund sites may
offer another useful precedent for such an effort. Reflecting a
growing consensus among many in the administration, state
government, and the Congress that states should take on more
responsibilities for leading priority site cleanups, EPA and
representatives from different states formed a number of
intergovernmental workgroups to recommend ways to overcome the key
barriers toward this goal.4 For example, a "State Readiness
Workgroup," composed of representatives of EPA headquarters and
regional offices and state agencies, was charged with clarifying
the requirements and circumstances under which states could be
granted additional responsibilities to clean up these priority
sites. Similarly, an intergovernmental "Assistance Workgroup" was
also established to identify the technical financial,
administrative, and legal assistance needs of the states in their
efforts to take a lead role in successfully cleaning up Superfund
sites. According to the Director of the State, Tribal, and Site
Identification Center (within the Office of Solid Waste and
Emergency Response), the workgroups were particularly useful in
fostering collaboration among representatives of EPA's
headquarters and regional offices involved in the cleanups in a
manner that helped to identify where the key problems were and
what practices worked well to address them. The Director said that
the results of the workgroups have since been incorporated into
pilot projects in seven states (and their corresponding regional
offices) designed to increase states' responsibilities in leading
cleanups of these sites. Conclusions On the basis of
information that can be learned from experiences to date of a
number of states and their corresponding EPA regional offices, we
4State and EPA efforts to augment states' roles in leading
Superfund cleanups are discussed in our 1997 report, Superfund:
Stronger EPA-State Relationship Can Improve Cleanups and Reduce
Costs (GAO/RCED-97-77, Apr. 1997). Page 60
GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
of NEPPS Participation Cited, but Full Potential Has Yet to Be
Realized believe the systematic joint evaluation process called
for by the 1995 Joint Commitment to Reform Oversight and Create a
National Environmental Performance Partnership System should be
initiated. The goals of this effort should be to (1) identify best
practices among participating states for dealing with the most
challenging problems facing the program and (2) eventually obtain
agreement on actions that will improve and expand the program.
Such a process has already been used to develop and improve the
Core Performance Measures used in the NEPPS program, and has
served as a successful model elsewhere in EPA where new ideas have
been developed and tested, and agreement among diverse parties on
their implementation has been reached. We believe a similar
effort, which targets key issues affecting NEPPS progress and
which involves representation from EPA headquarters offices, EPA
regional offices, and participating state agencies, could
similarly help to expand both the participation in, and
effectiveness of, this important program. The precise format to be
used for this process (e.g., whether individual working groups
should be established or whether a single committee composed of
senior state and EPA officials should be used) should be
determined by EPA and state environmental leaders. Recommendations
We recommend that the Administrator of EPA work with senior-level
state officials to initiate a joint evaluation process that (1)
seeks agreement on the key issues impeding progress in developing
a more effective National Environmental Performance Partnership
System and (2) develops mutually agreeable remedies for these
issues. Among the issues such a process could focus on are *
developing a set of flexible guidelines, to be used as a tool by
state and EPA regional NEPPS negotiators, that could help to
clarify the appropriate performance expectations and other
conditions that states must meet to achieve reduced oversight in
carrying out their environmental programs and the type of reduced
oversight (e.g., reduced frequency of reporting, greater autonomy
in setting program priorities) that could be achieved; *
identifying what additional work is needed to address the
challenges in implementing the Core Performance Measures recently
negotiated by EPA and the Environmental Council of the States for
fiscal year 2000, including how these measures can best be
reconciled with the measures adopted by EPA under the Results Act;
* alleviating the resistance among some staff (both within EPA
offices and among participating state agencies) toward
implementing the National Page 61
GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
of NEPPS Participation Cited, but Full Potential Has Yet to Be
Realized Environmental Performance Partnership System, through
training and other strategies; * determining what appropriate
steps should be taken by EPA and the states to allow for greater
use by states of the flexibility envisioned under the Performance
Partnership Grant system to shift resources and funding among
their media programs; * determining how effective public
participation in the NEPPS process can best be ensured; * and
developing ways to improve communication among EPA's headquarters
and regional offices and participating states to ensure that
states are given a clear and timely indication on whether key
elements of their agreements pursuant to the system have the full
buy-in of major EPA offices. Agency Comments EPA agreed with
the report's recommendation that EPA and state efforts to improve
NEPPS should include training and other efforts to achieve the
"cultural change" necessary for greater success. The agency also
pointed out that it recently agreed with representatives of the
Environmental Council of the States on a basic outline of a joint
evaluation process. We acknowledge this milestone and note that
further progress on the details of such a process, including the
specific issues to be addressed and a timetable for addressing
them, will be important steps toward improving NEPPS. EPA also
commented on our recommendation that EPA and state environmental
leaders should develop guidelines that would help to clarify, for
EPA and state negotiators, the appropriate performance
expectations that states must meet to achieve reduced oversight in
carrying out their environmental programs and the type of reduced
oversight (e.g., reduced frequency of reporting, greater autonomy
in setting program priorities) that could be achieved. EPA noted
that while it agreed with this recommendation in principle, the
agency and the states believe that each state's Performance
Partnership Agreement should specify the degree of oversight
necessary to accommodate the unique environmental problems and
varied program capabilities of that state. We agree that oversight
arrangements should be negotiated between each state and its
corresponding regional office in a manner that accounts for that
state's unique circumstances, and that these arrangements should
be specified in the Performance Partnership Agreement. We continue
to believe, however, that nonbinding national guidance-to be
agreed upon in advance by EPA and state environmental leaders-
would be useful in Page 62
GAO/RCED-99-171 Performance Partnership System Chapter 5 Benefits
of NEPPS Participation Cited, but Full Potential Has Yet to Be
Realized introducing objective parameters to be considered by
regional and state negotiators as they seek agreement over this
sensitive issue. Page 63
GAO/RCED-99-171 Performance Partnership System Appendix I Comments
From the Environmental Protection Agency and Our Evaluation See
comment 1. See comment 2. Page 64 GAO/RCED-99-171 Performance
Partnership System Appendix I Comments From the Environmental
Protection Agency and Our Evaluation See comment 3. See comment 4.
See comment 5. Page 65 GAO/RCED-
99-171 Performance Partnership System Appendix I Comments From the
Environmental Protection Agency and Our Evaluation The following
are GAO's comments on the Environmental Protection Agency's (EPA)
letter dated May 20, 1999. 1. We have clarified, in the executive
summary and chapter 5, that EPA officials and representatives of
the Environmental Council of the States have recently agreed to
certain characteristics of a joint evaluation process, and that
further progress (including decisions on the specific issues to
address and a timetable for addressing them) would be important
steps in improving NEPPS. 2. We have amended the report to reflect
the agency's expectation that its April 1999 2-year guidance
should allow the regions and states to consider national program
priorities earlier in their partnership agreement negotiations,
and thus limit the need to renegotiate priorities that had been
previously established. At the same time, the report still conveys
agency officials' views that the guidance will not necessarily
prevent other circumstances, which are out of EPA's control, from
necessitating the reopening of an agreement. 3. Citing the
observation in chapter 3 that (1) EPA has focused on outputs to
meet its obligations under the Results Act while supporting a
transition to outcome-based management under NEPPS and (2) these
conflicting priorities have led to confusion that hinders
performance partnerships, EPA stated that, to the contrary, both
the Results Act and NEPPS encourage the development of outcome
measures and outcome-based management. We acknowledge the shared
objective of NEPPS and the act in focusing on results. The key
word, however, is implementation: as we have documented in other
recent work, the measures EPA has used in its implementation of
the Results Act have thus far been heavily output-oriented and
therefore convey priorities that are often in conflict with the
more outcome-oriented measures being employed under NEPPS. We
acknowledge the agency's ongoing efforts to orient its Results
Act-related measures increasingly toward outcomes, and believe
that further progress toward this end will help to alleviate this
problem. In addition, we modified our discussion of this issue in
Chapter 3 to note that the April 1999 Addendum to the Joint
Statement, co-signed by EPA and the Environmental Council of the
States, states that core performance measures and other current
reporting requirements will be relied upon to satisfy EPA's
Results Act-related data needs. 4. We agree that oversight
arrangements should be negotiated between each state and its
corresponding regional office in a manner that accounts Page 66
GAO/RCED-99-171 Performance Partnership System Appendix I Comments
From the Environmental Protection Agency and Our Evaluation for
that state's unique circumstances, and that these arrangements
should be specified in the state's Performance Partnership
Agreement. We continue to believe, however, that nonbinding
national guidance-to be agreed upon by EPA and state environmental
leaders-would be useful in introducing objective parameters to be
considered by regional and state negotiators as they seek
agreement over this sensitive issue. 5. EPA's comment draws a
clear distinction between issues associated with reporting burdens
and other issues related to EPA's oversight of state environmental
programs. We acknowledge circumstances in which EPA and a state
collaboratively pursue strategies to reduce reporting requirements
that both agree are unnecessary, duplicative, or inefficient; and
that such circumstances could be viewed as outside the two
parties' oversight arrangement. However, the distinction between
this activity and oversight is not always so clear. Specifically,
where states and EPA have disagreed on the need for data not
required by statute and viewed by states as extraneous, and EPA
has continued to require reporting of such data, states have often
characterized the issue as, in their view, a questionable exercise
of EPA oversight. Page 67
GAO/RCED-99-171 Performance Partnership System Appendix II GAO
Contacts and Staff Acknowledgments GAO Contacts Steven
Elstein, (202) 512-6515 Acknowledgments In addition to those
named above, Maureen Driscoll, Gerald Laudermilk, Susan McCartin
and Lisa Pittelkau made key contributions to this report. (160447)
Page 68 GAO/RCED-99-171 Performance
Partnership System Ordering Information The first copy of each GAO
report and testimony is free. Additional copies are $2 each.
Orders should be sent to the following address, accompanied by a
check or money order made out to the Superintendent of Documents,
when necessary. VISA and MasterCard credit cards are accepted,
also. Orders for 100 or more copies to be mailed to a single
address are discounted 25 percent. Orders by mail: U.S. General
Accounting Office P.O. Box 37050 Washington, DC 20013 or visit:
Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S.
General Accounting Office Washington, DC Orders may also be placed
by calling (202) 512-6000 or by using fax number (202) 512-6061,
or TDD (202) 512-2537. Each day, GAO issues a list of newly
available reports and testimony. To receive facsimile copies of
the daily list or any list from the past 30 days, please call
(202) 512-6000 using a touchtone phone. A recorded menu will
provide information on how to obtain these lists. For information
on how to access GAO reports on the INTERNET, send an e-mail
message with "info" in the body to: [email protected] or visit
GAO's World Wide Web Home Page at: http://www.gao.gov PRINTED ON
RECYCLED PAPER United States General Accounting Office
Bulk Rate Washington, D.C. 20548-0001 Postage & Fees Paid GAO
Permit No. G100 Official Business Penalty for Private Use $300
Address Correction Requested
*** End of document. ***