Environmental Protection: Assessing Impacts of EPA's Regulations Through
Retrospective Studies (Letter Report, 09/14/1999, GAO/RCED-99-250).
Pursuant to a congressional request, GAO provided information on efforts
to assess the economic impact of the Environmental Protection Agency's
(EPA) regulations to enhance environmental quality, focusing on: (1) to
what extent EPA's regulations have been the subject of retrospective
studies; (2) whether retrospective studies are viewed as useful or
difficult to do; and (3) ways to foster such studies in the future.
GAO noted that: (1) assessments of the costs and the benefits of EPA's
regulations after they have been issued have rarely been done; (2) of
the 101 economically significant regulations issued by EPA from 1981
through 1998, only five were the subject of retrospective studies; (3)
various sources--including Congress, EPA, and academia have provided the
impetus for these studies, which were all completed from 1997 through
1999; (4) these studies covered regulations designed to, among other
purposes, control acid rain, phase out chlorofulorocarbons, and reduce
air pollution through improved vehicle inspection and maintenance; (5)
authors of retrospective studies, EPA officials, and other experts told
GAO that these studies have been or can be useful but also pose a number
of difficulties for researchers; (6) these studies found that that
actual costs of reducing emissions were lower than initially estimated;
(7) based on these findings, a legislative proposal to further limit
emissions has been introduced; (8) retrospective studies are viewed as
difficult to do because, among other reasons, it is difficult to obtain
valid cost data from regulated entities; (9) also, it is extremely
difficult to quantify actual benefits; (10) one reason is that a
significant time lag usually exists between the elimination or major
reduction of pollutants and any corresponding change in illness rates;
(11) GAO found that a systematic approach by EPA would foster
retrospective studies in the future; (12) while all economically
significant regulations, by executive order, are subject to a
prospective economic assessment, EPA is not expected to calculate the
actual costs and benefits for all of these regulations; (13)
accordingly, EPA officials told GAO that they have used the agency's
limited resources on prospective economic assessments--which EPA must
perform--rather than on retrospective studies; and (14) based on
discussions with study authors, EPA officials, and other experts, GAO
found that the resource constraints and the difficulties in conducting
retrospective studies could be better addressed if EPA were to develop a
plan to foster studies of the actual costs and benefits of environmental
regulations.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: RCED-99-250
TITLE: Environmental Protection: Assessing Impacts of EPA's
Regulations Through Retrospective Studies
DATE: 09/14/1999
SUBJECT: Reporting requirements
Strategic planning
Economic analysis
Agency proceedings
Policy evaluation
Environmental policies
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to Congressional Requesters
September 1999
ENVIRONMENTAL PROTECTION -
ASSESSING THE IMPACTS OF EPA'S
REGULATIONS THROUGH RETROSPECTIVE
STUDIES
GAO/RCED-99-250
Retrospective Studies of EPA's Regulations
(160453)
Abbreviations
=============================================================== ABBREV
CFC - chlorofluorocarbons
EPA - Environmental Protection Agency
OMB - Office of Management and Budget
PACE - Pollution Abatement and Expenditures
RFF - Resoures for the Future
Letter
=============================================================== LETTER
B-283198
September 14, 1999
The Honorable Bud Shuster
Chairman
Committee on Transportation and Infrastructure
House of Representatives
The Honorable Frank R. Lautenberg
United States Senate
Over the past three decades, the Environmental Protection Agency
(EPA) has issued and enforced regulations, pursuant to federal
statutes, to enhance environmental quality. These regulations impose
costs, such as compliance expenses for the regulated entities, and
produce benefits, such as improved human health and recreational
opportunities. These costs and benefits are substantial. In 1997,
the Office of Management and Budget (OMB) estimated that federal
environmental regulations cost about $144 billion annually and
produced benefits of about $162 billion.\1
Under Executive Order 12866, EPA prepares detailed cost-benefit
analyses for all economically significant regulations--including,
among others, those expected to have an annual impact on the economy
of $100 million or more.\2 These economic assessments inform and
improve the regulatory process by prospectively estimating the future
costs and benefits of feasible alternatives. Also, under this order,
EPA periodically reviews existing regulations that are economically
significant to determine if they can be made more effective or less
burdensome. The order, however, does not specify the nature or the
extent of this review. One tool that can be used in considering such
regulatory changes is a retrospective study�an assessment of the
actual costs and benefits.
Concerns have been raised, however, that agencies have made only
limited attempts to assess the actual impacts of federal regulations.
For example, in May 1998, the Senate Committee on Governmental
Affairs reported that the review of existing rules, as directed by
executive order, �has met with varying degrees of failure. Clearly,
getting agencies to review existing rules is much easier said than
done.�\3 In light of these concerns, you asked us to (1) determine to
what extent EPA's regulations have been the subject of retrospective
studies, (2) identify in what ways retrospective studies are viewed
as useful or difficult to do, and (3) identify ways to foster such
studies in the future.
--------------------
\1 In a 1998 report, OMB estimated that the costs of federal
environmental regulations were between $120 billion and $170 billion
and the benefits could be as high as $3.3 trillion annually. The
controversy surrounding this estimate is discussed in our report,
Regulatory Accounting: Analysis of OMB's Reports on the Costs and
Benefits of Federal Regulation (GAO/GGD-99-59, Apr. 20, 1999).
\2 The executive order was issued on September 30, 1993. EPA is also
directed by the Unfunded Mandates Reform Act of 1995 to conduct
regulatory cost-benefit analyses under certain circumstances.
\3 The Congress has considered a legislative proposal that would
require agencies to review or �look back� at existing regulations.
To see our testimony on this proposal, see Regulatory Reform:
Comments on S. 981�The Regulatory Improvement Act of 1997
(GAO/T-GGD/RCED-97-250, Sept. 12, 1997).
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
Assessments of the costs and the benefits of EPA's regulations after
they have been issued have rarely been done. Of the 101 economically
significant regulations issued by EPA from 1981 through 1998, only
five were the subject of retrospective studies.\4 Various
sources--including the Congress, EPA, and academia�have provided the
impetus for these studies, which were all completed from 1997 through
1999. These studies covered regulations designed to, among other
purposes, control acid rain, phase out chlorofluorocarbons, and
reduce air pollution through improved vehicle inspection and
maintenance.
Authors of retrospective studies, EPA officials, and other experts
told us that these studies have been or can be useful but also pose a
number of difficulties for researchers. In terms of usefulness, for
example, retrospective studies have provided insights into a new,
market-based regulatory approach to reduce emissions that cause acid
rain. These studies found that the actual costs of reducing
emissions were lower than initially estimated. Based on these
findings, a legislative proposal to further limit emissions has been
introduced. Retrospective studies are viewed as difficult to do
because, among other reasons, it is difficult to obtain valid cost
data from regulated entities. For example, corporate accounting and
financial records do not typically capture the information necessary
to determine the incremental costs of complying with federal
environmental regulations, and companies have little incentive to
isolate and monitor these costs. Also, it is extremely difficult to
quantify actual benefits. One reason is that a significant time lag
usually exists between the elimination or major reduction of
pollutants and any corresponding change in illness rates.
We found that a systematic approach by EPA would foster retrospective
studies in the future. While all economically significant
regulations, by executive order, are subject to a prospective
economic assessment, EPA is not expected to calculate the actual
costs and benefits for all of these regulations. Accordingly, EPA
officials told us that they have used the agency's limited resources
on prospective economic assessments rather than on retrospective
studies. Based on discussions with study authors, EPA officials, and
other experts, we found that the resource constraints and the
difficulties in conducting retrospective studies could be better
addressed if EPA were to develop a plan to foster studies of the
actual costs and benefits of environmental regulations. We are
recommending that EPA develop a plan based on a systematic (1)
identification of candidate regulations for study, (2) assessment of
the feasibility of conducting retrospective studies of those
regulations, and (3) identification of the resources needed.
--------------------
\4 According to EPA, another 14 economically significant regulations
have been issued in 1999.
FEW OF EPA'S REGULATIONS HAVE
BEEN THE SUBJECT OF
RETROSPECTIVE STUDIES
------------------------------------------------------------ Letter :2
EPA's regulations have rarely been the subject of a retrospective
study.\5 According to EPA, the agency issued 101 economically
significant regulations from 1981 through 1998,\6 and five of these
regulations have been the subject of retrospective studies. In
addition, of the more than 2,600 environmental regulations issued
during this period that were not economically significant, 23 were
the subject of retrospective studies.
As listed below, the retrospective studies we identified ranged from
broad studies, such as a report on the impacts of the entire Clean
Air Act, to studies of individual regulations, such as the phaseout
of chlorofluorocarbons (CFC). All were completed from 1997 through
1999. Most of the studies had prospective cost estimates against
which actual costs could be compared. Several also had information
on actual benefits.
-- Section 812 of the 1990 amendments to the Clean Air Act required
EPA to provide information about the economic costs and benefits
and the health, welfare, and environmental impacts of the Clean
Air Act. EPA issued the Section 812 study in October 1997.\7
-- The 1990 amendments to the Clean Air Act directed an interagency
group to report every 4 years, beginning in 1996, on the costs,
the benefits, and the effectiveness of the acid rain program.
The group, known as the National Acid Precipitation Assessment
Program, issued the first retrospective study under this mandate
in 1998.\8
-- Other retrospective studies involving EPA's regulations include
23 studies on pesticides and individual studies on the phaseout
of CFCs, sulfur dioxide reductions, vehicle inspection and
maintenance, and reformulated gasoline. The sulfur dioxide
report was the only one of these studies to include information
on benefits.
Various sources provided the impetus for the retrospective studies we
identified. The Congress mandated the Section 812 and acid rain
studies. The latter study was funded under interagency agreements
with EPA and with other federal agencies.\9 EPA's Office of Policy
signed a research cooperative agreement with Resources for the Future
(RFF),\10 \11 which, in turn, initiated the 23 pesticide studies and
the vehicle inspection and maintenance study. EPA's Office of Air
and Radiation's Acid Rain Division partially funded the sulfur
dioxide study through cooperative agreements with the Massachusetts
Institute of Technology and RFF. Two retrospective studies were
completed without funding from EPA. The CFC phaseout study was
conducted by a faculty member of Harvard University and funded by
Harvard University, and the reformulated gasoline study was prepared
at the researchers' expense.
In addition to the retrospective studies that have been completed,
others are underway. EPA's Offices of Policy and Ground Water and
Drinking Water have contracted for a retrospective study of EPA's
drinking water standards for atrazine and nitrate.\12 EPA expects the
study to be completed by September 30, 1999. Also, under a
cooperative agreement with EPA, RFF is conducting another series of
retrospective studies on pesticide regulations.
--------------------
\5 We considered a study to be a �retrospective� study if, at a
minimum, it contained information on the costs of existing
regulations and focused on a specific regulation or a set of
regulations with common objectives.
\6 We used 1981 as the starting point because EPA began tabulating
the number of its regulations at that time.
\7 When counting the number of regulations that were the subject of
retrospective studies, we excluded the regulations covered in the
Section 812 study because it aggregated the costs for a number of
existing regulations and did not identify the incremental costs of
specific regulations.
\8 Title IV of the Clean Air Act amendments seeks to reduce the
adverse effects of acid rain through reductions in annual emissions
of sulfur dioxide and nitrogen oxides from utilities burning fossil
fuels.
\9 Agencies in the National Acid Precipitation Assessment Program
include EPA; the departments of Interior, Energy, and Agriculture;
the National Aeronautics and Space Administration; and the National
Oceanic and Atmospheric Administration.
\10 Generally, under this type of cooperative agreement, EPA agrees
to a research agenda proposed by the recipient and provides funds
without specifying how the research should be done. Also, the
recipient shares in the cost of the research.
\11 RFF is an independent, nonprofit organization engaged in research
and public education on natural resources and environmental issues.
\12 Atrazine is a herbicide and nitrate is an inorganic chemical
present in fertilizers and animal wastes. These chemicals can enter
drinking water by runoff into surface water and leaching into
groundwater.
RETROSPECTIVE STUDIES ARE
VIEWED AS USEFUL BUT ALSO
DIFFICULT TO DO
------------------------------------------------------------ Letter :3
According to the experts we interviewed, while retrospective studies
have provided valuable insights they also have been difficult to
perform.
RETROSPECTIVE STUDIES ARE
SEEN AS USEFUL
---------------------------------------------------------- Letter :3.1
Authors of retrospective studies, EPA officials, and other experts we
interviewed identified three primary areas where retrospective
studies have been or can be useful by providing (1) information for
consideration of legislative or regulatory modifications, (2)
insights into the need for methodological improvements in prospective
economic assessments, and (3) information for EPA's use in
implementing the Government Performance and Results Act (the Results
Act).
MODIFYING LEGISLATION OR
REGULATIONS
-------------------------------------------------------- Letter :3.1.1
Retrospective studies provide information used in considering
legislative and regulatory modifications�especially when a new
regulatory approach has been taken. Some time after implementation
of a regulation, a comparison of post- and pre-issuance estimates of
the costs and the benefits provides insights into whether that
regulation is working as planned, whether the burden on regulated
entities is appropriate, and whether unanticipated consequences are
occurring.
For example, title IV of the 1990 amendments to the Clean Air Act
authorized EPA to adopt a new regulatory approach to the acid rain
problem instead of the traditional �command and control� approach,
under which pollution sources are required to install certain
pollution control technologies or meet specific performance
standards. The new approach provided market-based incentives. It
allocated a number of allowances for sulfur dioxide emissions to each
utility under the program. Each allowance represents the limited
authority of a utility to emit one ton of sulfur dioxide. The Act
set an overall goal and capped the number of allowances to ensure an
overall reduction in emissions. Each utility may choose to reduce
its own emissions or to purchase unused allowances from another
utility. Furthermore, a utility may choose to �bank� its allowances
for sale at a later date. The allowances trade freely, as stocks do.
Retrospective studies indicate that the actual cost of reducing
emissions under title IV has been lower than initially estimated.
One reason for the cost overestimation has been the reduced cost of
pollution control technologies. Based on these findings and reports
that some ecosystems are not yet recovering from acid rain, a bill
(S. 172) was introduced in the U.S. Senate in January 1999 that
would, among other things, further limit sulfur dioxide emissions.
The retrospective studies also provided information on whether the
market-based approach used for addressing acid rain could be applied
to other regulatory areas, such as climate change and greenhouse
gases.
IMPROVING METHODOLOGIES
OF PROSPECTIVE ANALYSES
-------------------------------------------------------- Letter :3.1.2
Retrospective studies can also provide agencies with insights on how
to better conduct prospective cost-benefit analyses and prevent
methodological biases in these analyses. For example, some analysts
have suggested that EPA tends to overestimate costs because, among
other reasons, it does not anticipate and account for cost-saving
technological innovations. However, in its analysis of the accuracy
of prospective cost-benefit estimates, RFF found that two studies (on
reformulated gasoline and the Dinoseb pesticide) had overestimated
costs; two (on the CFC phaseout and sulfur dioxide) had accurately
estimated costs; and one (on the Aldicarb pesticide) had
underestimated costs.\13 Additional retrospective studies would
enable EPA to more fully explore concerns about what methodological
factors, if any, contribute to the over- or underestimation of costs.
According to an EPA official, even the best models for predicting the
costs and the benefits could be based on erroneous assumptions
because it is difficult to forecast technological improvements and
economic changes. The official explained that one conclusion from
the retrospective studies' results might be to include a �technology
improvement case� when estimating the costs of regulations.
--------------------
\13 RFF labeled a prospective study as �accurate� if the estimated
costs in the retrospective study were no more than 25 percent higher
or no less than 25 percent lower than the prospective estimates.
PROVIDING INFORMATION FOR
IMPLEMENTING THE RESULTS
ACT
-------------------------------------------------------- Letter :3.1.3
Retrospective studies of the costs and the benefits can also help EPA
in its efforts to implement the Results Act. The Results Act
requires agencies to set goals for program performance and to measure
results. Retrospective studies are one means of measuring results.
Such analyses aim to identify relevant results�usually expressed in
monetary terms--of existing regulations. The studies that identify
actual benefits can provide insights into the outcomes of EPA's
regulations. We reported in 1998 that the Results Act recognizes and
encourages the linkage between program evaluations--retrospective
studies are one type of such evaluations�and performance measures.
The Act directs federal agencies to provide a summary of program
evaluation findings along with performance measurement results in the
required annual performance reports.\14
--------------------
\14 Program Evaluation: Agencies Challenged by New Demand for
Information on Program Results (GAO/GGD-98-53, Apr. 24, 1998).
RETROSPECTIVE STUDIES ARE
DIFFICULT TO DO
---------------------------------------------------------- Letter :3.2
Researchers face a number of difficulties in conducting retrospective
studies: (1) determining the baseline against which the changes can
be identified and measured, (2) isolating the reasons for the actions
taken, (3) obtaining valid cost data from the regulated entities, and
(4) quantifying the benefits.
DETERMINING THE BASELINE
-------------------------------------------------------- Letter :3.2.1
According to OMB's 1998 Report to the Congress on the Costs and
Benefits of Federal Regulations, one of the primary problems with
estimating the incremental costs and benefits associated with
implementing a federal regulation is determining the baseline against
which the costs and the benefits should be compared. A baseline--how
things would have been if the regulation had not been issued--is
elusive because so many changes could have occurred while the
regulation was being implemented. As OMB's report states, "what
would have happened in the absence of regulation can only be an
educated guess since it never happened."
In a 1996 report on companies' regulatory burden, we found that it
can be extremely difficult for company officials to determine what
actions their company would have taken in the absence of the
identified regulations.\15 EPA and RFF officials took a more
optimistic view but contended that modeling would be needed to
establish a baseline. Such modeling can be costly and time-consuming
and requires making assumptions about economic behavior that may or
may not approximate reality.
--------------------
\15 Regulatory Burden: Measurement Challenges and Concerns Raised by
Selected Companies (GAO/GGD-97-2, Nov. 18, 1996).
DETERMINING THE REASONS
FOR REGULATED ENTITIES'
ACTIONS
-------------------------------------------------------- Letter :3.2.2
According to OMB's report, it is often difficult to attribute the
actions taken by regulated entities to specific federal regulations
or other external factors. These entities may be responding, for
example, to regulations from state and local governments. Voluntary
standards from organizations and public pressure also cause firms to
protect the public in the absence of federal regulations. In our
1997 report on the governmentwide implementation of the Results Act,
we found that regulatory agencies have had difficulty in sorting out
the effects of external forces.\16 One expert commented that external
factors often influence technological innovation and that isolating
the impacts of regulations from these other factors is difficult.
In 1997, EPA's Office of Water found these multiple factors to be a
problem when it attempted to conduct a retrospective study of
effluent guidelines. The Office never implemented the study because
it was too difficult to isolate the costs specifically related to
compliance with effluent guidelines from either self-initiated
technological improvements or state and local government
requirements.
--------------------
\16 The Government Performance and Results Act: 1997 Governmentwide
Implementation Will Be Uneven (GAO/GGD-97-109, June 2, 1997). Also,
see Managing for Results: Regulatory Agencies Identified Significant
Barriers to Focusing on Results (GAO/GGD-97-83, June 24, 1997).
OBTAINING VALID COST DATA
FROM REGULATED ENTITIES
-------------------------------------------------------- Letter :3.2.3
A number of studies have reported difficulties in obtaining valid
cost data. For example, our 1996 study on regulatory burden also
found that companies' accounting and financial records did not
capture the information necessary to determine incremental compliance
costs and that there was little incentive to isolate and monitor
these costs because such information had little business value.
Furthermore, companies were concerned about the disclosure of
information that is proprietary (e.g., information on their operating
expenses) or could potentially involve regulatory enforcement
actions. Similarly, in 1996, we found it difficult to determine the
effects of regulatory compliance on research grantees at the National
Institutes of Health.\17
Although members of the research community were confident that
compliance costs were high, they were unable to provide cost data
directly attributable to compliance activities.
Other research organizations have also had difficulties in data
collection. According to an expert in cost-benefit analysis at the
Massachusetts Institute of Technology, the university conducted a
telephone survey to collect actual cost data on scrubber technology,
but despite having "very good" industry connections, the survey's
results were somewhat disappointing because some companies declined
to participate. The Organization for Economic Cooperation and
Development�an international agency sponsored by industrialized
countries--is conducting a business survey to determine the costs
associated with implementing regulations in three general areas:
environmental compliance, tax administration, and employment. Its
survey is limited to small- and medium-sized businesses. According
to Organization officials, it is more difficult for large companies
that operate at multiple locations to determine the cost impacts of
regulations on their far-flung enterprises and to isolate these costs
from the costs attributable to other business decisions. They also
told us that asking businesses to self-report capital costs would not
be valid because the data would not be verifiable or consistent. In
their view, such data would have to be collected through very
detailed structured interviews at selected businesses.
Some data on the costs of environmental regulatory compliance were
once available. Until 1994, the Department of Commerce administered
the Pollution Abatement and Control Expenditures (PACE) survey. When
it discontinued the survey to focus its resources on higher
priorities, Commerce collected data annually from manufacturers on
the aggregate cost of implementing pollution control regulations.
Commerce, in collaboration with EPA, is working to resume the PACE
survey in 2000. According to EPA, reinstatement of this survey is
dependent on the availability of funds. While one expert we
interviewed suggested restarting the PACE survey, another raised
questions about the data's validity and usefulness. An official in
EPA's Office of Policy told us that his agency is aware of these
questions and, in developing the new survey, will attempt to address
them. EPA also pointed out that the agency's Science Advisory Board
and others have recommended reinstatement of the survey.
Another attempt at data collection is currently being undertaken. A
contractor is collecting cost data for the retrospective study of the
drinking water standards for atrazine and nitrate. EPA officials
told us that they believe the drinking water treatment facilities
will be able to isolate the costs of compliance for these chemicals.
They also expected both public and private treatment facilities to
cooperate.
--------------------
\17 Regulatory Compliance for NIH Grantees (GAO/HEHS-96-90R, Mar.
25, 1996).
QUANTIFYING BENEFITS
-------------------------------------------------------- Letter :3.2.4
The experts we interviewed agreed that quantifying the benefits, as
part of a retrospective study, is extremely difficult. One cited
reason is that data on benefits, which generally accrue outside the
regulated entities (e.g., to protected species or society at large),
are not available. Another reason, which was highlighted by the
author of the CFC retrospective study, is the difficulty in assigning
the benefits to a particular regulatory action versus other factors.
For example, he pointed out that although the benefits from a
reduction in occurrences in skin cancer could be attributed to the
phaseout of CFCs, other factors could have played a role. Finally,
considerable time may elapse�sometimes decades�before many benefits
are realized. For example, some diseases, such as cancer, can have
latency periods of decades. We emphasized this difficulty in our
1997 report on the Results Act, citing EPA officials who noted that a
significant time lag usually exists between the elimination of a
chemical hazard and any corresponding change in illness rates. This
lag makes it difficult to track the yearly progress of some of the
agency's regulatory actions. EPA's 1997 Strategic Plan noted how
difficult it is to assess the risks from pollutants; measure the
consequences of pollution to persons, plants, and animals; and assign
a dollar value to the health and environmental effects that are
remote in distance and time.
A SYSTEMATIC APPROACH WOULD
FOSTER RETROSPECTIVE STUDIES
------------------------------------------------------------ Letter :4
Based on discussions with study authors, EPA officials, and other
experts, we found that a systematic approach by EPA would foster
retrospective studies in the future. The agency, however, has not
targeted regulations for review after systematically (1) identifying
candidate regulations for study based on selection criteria, (2)
assessing the feasibility of conducting retrospective studies of
those regulations, and (3) identifying the resources needed.
IDENTIFYING CANDIDATES FOR
REVIEW
---------------------------------------------------------- Letter :4.1
While all economically significant regulations are subject, by
executive order, to a prospective economic assessment of their costs
and benefits before issuance, EPA is not required to calculate the
actual costs and benefits retrospectively for all of its rules.
Program offices within the agency have, on an ad hoc basis, initiated
the sulfur dioxide study and the ongoing study of the drinking water
standards. EPA has not, however, reviewed the full array of
environmental regulations and, based on criteria, selected certain
rules as candidates for study.
In identifying candidates for review, one criterion cited by the
experts we interviewed is the potential for learning something useful
that would improve the efficiency and the effectiveness of a
significant regulatory program. For example, the significance of
acid rain for the northeastern states and the concern that existing
regulations were not adequately controlling the problem played a role
when the Congress mandated a retrospective study. Other criteria
cited are the potential for gaining insights into a new regulatory
approach to environmental protection or the methodology for
prospective economic assessments. A retrospective study can, for
instance, examine assumptions made in prospective assessments where
uncertainty about regulatory costs exists because a new pollution
control technology is being used.
When identifying candidate regulations, timing is another
consideration. EPA would examine whether it would be more useful to
study the actual costs and benefits of certain regulations shortly
after implementation or to wait until a number of years or even
decades have elapsed. A major factor in the timing decision would be
the value of gaining insights into a regulation's impacts immediately
or after a more extended period of time.
ASSESSING THE FEASIBILITY OF
A RETROSPECTIVE STUDY
---------------------------------------------------------- Letter :4.2
The next action would be an assessment of the feasibility of
conducting a retrospective study of the candidate regulations. This
assessment would focus on the potential difficulties facing a
researcher. For example, to obtain valid cost data, EPA might find
it easier to address those regulations that rely on market forces
because price and cost information is in the public domain rather
than those regulations based on �command and control,� for which
collecting the original data is especially challenging. In some
command-and-control cases, however, the regulated entities could take
steps during the implementation of the regulation to help researchers
develop data on compliance costs. These steps could include asking a
sample of regulated entities to establish financial and accounting
mechanisms to measure the impact of the regulation as it is
implemented. In addition, to attribute changes in the actions of
regulated entities to specific EPA regulations, RFF officials
suggested that researchers employ the same models used in conducting
prospective economic assessments and, therefore, make the same
assumptions about regulated entities' taking actions in response to
the regulations. Of course, the researchers would attempt to
re-estimate the costs and the benefits after updating the model, for
example, with any data on the actual steps taken by regulated
entities.
IDENTIFYING NEEDED RESOURCES
---------------------------------------------------------- Letter :4.3
A planned approach would identify the resources needed to support the
retrospective study. Funding is, of course, critical. EPA receives
funding for analytical work in general. While EPA has occasionally
funded retrospective studies, EPA program officials noted that they
have used their analytical resources almost entirely for prospective
analyses�those that provide information for decisions on proposed
regulations. A 1997 Congressional Budget Office study found that EPA
incurred an average cost of $662,000 and a median cost of $376,000
(in 1995 dollars) on 65 economic assessments.\18 The program
officials told us that they had limited discretionary funds and
resources and needed to use them in developing new regulations. One
official said that once a regulation is issued, his office gears up
to develop any other regulations required by law.
The funding needed to do retrospective studies would be substantial
if a large proportion of the 101 economically significant regulations
issued since 1981 were the subject of retrospective studies. Where
we were able to obtain information on study costs, they varied
substantially�from $150,000 for the drinking water standards study,
to $433,000 for the acid rain report,\19 to $4 million for the
Section 812 study.
The method of funding as well as responsibility for a study can have
implications for its credibility. Authors of retrospective studies
told us that the credibility of these studies is enhanced when EPA
funds an independent analysis. The authors stated that it is better
not to have the same EPA offices that were responsible for conducting
the prospective cost-benefit analysis also responsible for conducting
the retrospective study. OMB officials stated that the best
estimates of the costs and the benefits of regulation are likely to
be retrospective studies done by individuals who do not have vested
interests but do have reputations as objective analysts to uphold.
One study author believed that the cooperative agreement arrangement
helped ensure that the researcher had freedom to maintain
independence. As discussed earlier, EPA has used cooperative
agreements with organizations, such as RFF and the Massachusetts
Institute of Technology. Another author suggested that if EPA's
program offices were directly involved in the study, the agency could
use some type of advisory committee or peer review process to oversee
the studies and ensure their credibility.
Resources also take the form of technical assistance, and EPA has
provided some. For example, the Office of Policy has built a library
of EPA's economic assessments of major regulations and related
reports and provides a limited number of them on the Internet.
Furthermore, EPA is now soliciting comments on a major revision of
its Guidelines for Performing Economic Assessments.\20 The agency
first issued these guidelines in December 1983 and has made a few
modifications to them over the past 15 years. The draft revision
provides, among other things, a summary of analytical methodologies,
empirical techniques, and data sources that can assist analysts in
performing economic assessments of environmental policies. While the
guidance is targeted more to prospective analyses, its principles can
also be applied to retrospective studies.
To foster retrospective studies, the experts we interviewed also
suggested that the availability and the accessibility of the
prospective economic assessments and supporting documentation would
have to be improved. For example, some of these authors who had used
the models employed in the prospective analyses told us they had
difficulty with the availability and the usefulness of the models.
One researcher wanted to run the model used for the prospective
economic assessment with different inputs. This would have served as
a useful way to determine to what extent and why the estimate of
actual costs and benefits differed from the prospective estimate. In
conducting his study, he had difficulty obtaining the complete model
from EPA. He was, however, able to obtain the complete model from
the contractor who performed the assessment. Another author found
that although he had access to the original model and supporting
documentation, it was very difficult to extract information.
Acknowledging concerns about the accessibility of models, the Office
of Air and Radiation maintains a web site that has placed all model
inputs in the public domain.
Finally, one author focused on the importance of capturing data on
the incremental costs of regulatory compliance from the regulated
entities. He noted that EPA would enhance its and others' capacity
to evaluate environmental regulatory programs if the agency collected
and maintained data on the costs immediately after issuing those
regulations it has targeted for retrospective study.
--------------------
\18 Regulatory Impact Analyses: Costs at Selected Agencies and
Implications for the Legislative Process (Mar. 1997). Costs
included time spent by EPA personnel. The majority of the economic
assessments in the study were completed between 1990 and 1996.
Because the scope and the complexity varied substantially, the costs
ranged from as low as $48,000 to as high as $6 million.
\19 From 1991 through 1996, the federal government spent $140 million
to underwrite the scientific research contained in the acid rain
report. This research included an extensive network of monitoring
stations that provided scientists with data sets.
\20 The Environmental Economics Advisory Committee of EPA's Science
Advisory Board has a major role in reviewing these guidelines.
CONCLUSION
------------------------------------------------------------ Letter :5
EPA's regulations, issued pursuant to federal statutes, cause this
nation to make a substantial investment in protecting the
environment. They also result in considerable benefits. While EPA
devotes substantial resources to cost-benefit analyses when
developing new regulations, the agency seldom looks back at the
actual costs and benefits after those regulations have been
implemented. An executive order and �good government� principles
recommend that EPA periodically reassess selected regulations to
determine if they are still needed and, if so, whether and how they
could be improved. Retrospective studies are difficult to do, but
they are a potentially valuable tool for such reassessments. We
believe that EPA's current ad hoc approach gives no assurance that
economically significant regulations will be subjected to review. A
systematic, planned approach would help EPA to foster retrospective
studies in the future.
RECOMMENDATION
------------------------------------------------------------ Letter :6
To help determine if existing environmental regulations need to be
retained or improved, we recommend that the Administrator, EPA,
establish a plan to study the actual costs and benefits of such
regulations, including when and how to use retrospective studies as
an integral part of its approach. The plan should target selected
regulations for review based on a systematic identification of
candidate regulations, an assessment of the feasibility of studying
those regulations retrospectively, and an identification of the
resources needed.
AGENCY COMMENTS
------------------------------------------------------------ Letter :7
We provided a draft of this report to EPA for its review and comment.
EPA commented that the report is accurate, provides a fair
characterization of the difficulties in performing retrospective
studies, and offers some useful suggestions on how the agency could
approach such studies in a more systematic manner.
EPA also suggested that the budget implications of conducting
retrospective studies need to be emphasized in the report because the
scarcity of budget and internal resources limits the number of such
studies that the agency could support without the Congress' providing
significant additional resources. In this report, we recognize that
EPA concentrates its analytical resources on prospective economic
assessments, which EPA is expected to perform, rather than on
retrospective studies. Executive Order 12866 states that EPA's
review of existing regulations should be consistent with the agency's
resources and regulatory priorities. We believe that if EPA were to
develop a plan, as we are recommending, the agency would be better
able to identify the additional resources needed to retrospectively
review selected regulations. EPA also provided specific technical
comments and clarifications, which we incorporated, as appropriate.
EPA's comments are contained in appendix I.
SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :8
To accomplish our objectives, we interviewed officials in EPA's
Office of Policy; Office of Air and Radiation; Office of Water;
Office of Solid Waste and Emergency Response; Office of Prevention,
Pesticides, and Toxic Substances; and Office of the Chief Financial
Officer and obtained and reviewed related legislation, memorandums,
and reports. We also interviewed all of the authors of the
retrospective studies identified in this report and several other
experts in cost-benefit analysis, including RFF and OMB officials,
and representatives from Harvard University, the Massachusetts
Institute of Technology, and the Organization for Economic
Cooperation and Development. In documenting the difficulties in
obtaining actual cost data from companies, we relied heavily on our
November 1996 report, Regulatory Burden: Measurement Challenges and
Concerns Raised by Selected Companies. To identify the universe of
retrospective studies, we used data developed by RFF for its January
1999 report, On the Accuracy of Regulatory Cost Estimates. We also
added the Section 812 report, the acid rain report, and pesticide
studies that RFF chose to exclude because they met our criteria for
containing information on the actual costs of existing regulations
and focusing on a specific regulation or a set of regulations with
common objectives.
We conducted our review from October 1998 through September 1999 in
accordance with generally accepted government auditing standards.
---------------------------------------------------------- Letter :8.1
We are sending copies of this report to Senator Fred Thompson and
Senator Joseph Lieberman, as Chairman and Ranking Member,
respectively, of the Senate Committee on Governmental Affairs;
Representative Dan Burton and Representative Henry Waxman, as
Chairman and Ranking Member, respectively, of the House Committee on
Government Reform; Representative James Oberstar as Ranking Member of
the House Committee on Transportation and Infrastructure; the
Honorable Carol Browner, Administrator, EPA; and the Honorable Jacob
Lew, Director of OMB. We will also make copies available to others
on request.
If you have any questions about this report, please contact me on
(202) 512-6111. Major contributors to this report were Jay Cherlow,
Ellen Crocker, Bob Levin, and Bruce Skud.
Peter F. Guerrero
Director, Environmental Protection Issues
(See figure in printed edition.)Appendix I
COMMENTS FROM THE ENVIRONMENTAL
PROTECTION AGENCY
============================================================== Letter
(See figure in printed edition.)
(See figure in printed edition.)
*** End of document. ***