Environmental Information: EPA Needs Better Information to Manage Risks
and Measure Results (Testimony, 10/03/2000, GAO/GAO-01-97T).
The Environmental Protection Agency (EPA) needs comprehensive and
accurate data to manage its programs more effectively. In reports going
back to 1988, GAO has identified many long-standing problems in the
agency's efforts to collect and use environmental data. This report
summarizes GAO's findings on: (1) EPA's need to set risk-based
priorities for its programs, and (2) develop outcome-oriented measures
of its programs' results. EPA's ability to assess risks and establish
risk-based priorities has been hampered by data quality problems,
including critical data gaps, databases that do not operate compatibly
with one another, and persistent concerns about the accuracy of the data
in many of EPA's data systems. To ensure future success in developing
outcome measures, however, EPA will need to make a long-term commitment
to overcome major challenges to obtaining the data needed to show the
results of environmental programs.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: GAO-01-97T
TITLE: Environmental Information: EPA Needs Better Information to
Manage Risks and Measure Results
DATE: 10/03/2000
SUBJECT: Risk management
Environmental policies
Environmental monitoring
Program evaluation
Information resources management
Performance measures
Data collection
Management information systems
Data integrity
IDENTIFIER: EPA Integrated Risk Information System
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-01-97T
ENVIRONMENTAL INFORMATION
EPA Needs Better Information to Manage Risks and Measure Results Statement
of Peter F. Guerrero, Director Environmental Protection Issues, Resources,
Community, and Economic Development Division
United States General Accounting Office
GAO Testimony Before the Committee on Environment and Public Works,
U. S. Senate
For Release on Delivery Expected at 9: 30 a. m., EDT Tuesday October 3, 2000
GAO- 01- 97T
Mr. Chairman and Members of the Committee: We appreciate the opportunity to
discuss our observations on the data that the Environmental Protection
Agency (EPA) needs to manage its programs more effectively. In reports going
back to our comprehensive general management review of EPA in 1988, 1 we
have identified numerous long- standing problems in the agency's efforts to
collect and use environmental data. Drawing from this work, I will discuss
today the limitations in the data that EPA needs to (1) set risk- based
priorities for its programs and (2) develop outcome- oriented measures of
its programs' results. Our observations are as follows:
� EPA's ability to assess risks and establish risk- based priorities has
been hampered by data quality problems, including critical data gaps,
databases that do not operate compatibly with one another, and persistent
concerns about the accuracy of the data in many of EPA's data systems. While
EPA's priorities should reflect an understanding of relative risk to the
environment and public health, good data often do not exist to fully
characterize risk. In the absence of reliable data, public perceptions of
risk can influence how EPA determines its priorities and allocates
resources. EPA has taken major steps during the past few years to improve
its data and to better inform the scientific community and general public of
environmental and public health risks. To finish this job, the agency will
need to expand its data improvement initiatives to fill key gaps in its
data, take advantage of opportunities to develop and implement data
standards to achieve compatibility among environmental databases, and ensure
the accuracy of its data.
� Measuring the results (outcomes) of its programs is critical to
determining EPA's effectiveness. Nevertheless, the agency historically has
relied on activity- based output measures, such as the number of inspections
performed, because of inherent technical difficulties in establishing sound
linkages among program activities, environmental improvements, and public
health. Spurred by the requirements of the
1 Environmental Protection Agency: Protecting Human Health and the
Environment Through Improved Management( GAO/ RCED- 88- 101, Aug. 16, 1988).
2 Government Performance and Results Act of 1993 (Results Act), EPA has made
progress in recent years in measuring the outcomes of its programs. To
ensure future success in developing outcome measures, however, EPA will need
to make a longterm management commitment to overcome major challenges to
obtaining the data needed to show the results of environmental programs.
Background
Since EPA's establishment in 1970, the federal government has developed a
complex system of laws and regulations to address the nation's environmental
problems. Over the years, as environmental threats were identified, the
Congress responded by enacting laws to address each problem, incrementally
adding to the statutory framework that sets EPA's agenda. However, these
laws were not coordinated or integrated to provide EPA with an overall
system for prioritizing problems so that the most serious problems can be
addressed first.
Impelled by budgetary constraints and a growing list of environmental
problems, EPA, in the late 1980s, began to consider whether its resources
were being spent on the problems that pose the greatest risks to public
health and the environment. The agency concluded that the nation actually
was devoting more resources to problems that had captured public attention
than to problems that were less well known but potentially more serious.
Subsequently, EPA began incorporating the concept of relative health and
environmental risk into decisions on environmental priorities and
emphasizing the need to identify the most serious risks and to keep the
public informed about the relative seriousness of various environmental
problems. To assess risks and deal with those likely to do the most harm,
EPA has recognized that it needs to have adequate environmental and
scientific data to conduct risk assessments, set standards, and develop
regulations. It also needs such data to identify and develop measures of
environmental quality and to assess the effectiveness of its programs by
linking program activities to changes in environmental conditions.
3
EPA Needs Better Data to Establish Risk- based Program Priorities
Establishing risk- based priorities for EPA's program activities requires
good data on the use and disposal of thousands of chemicals. To assess human
exposure to a chemical, EPA needs to know how many workers, consumers, and
others are exposed; how the exposure occurs; and the amount and duration of
the exposure. For environmental exposure, EPA needs to know whether the
chemical is being released to the air, water, or land; how much is being
released; and how wide an area is being affected. EPA's ability to make such
assessments is limited by (1) gaps in environmental and health data, (2)
databases that do not operate compatibly with one another, and (3) the lack
of an effective system for ensuring the accuracy of the agency's data.
Although EPA has implemented several agencywide initiatives to address these
problems, each of the initiatives has encountered obstacles that must be
overcome to substantially improve the agency's data.
Extensive Gaps Exist in EPA's Information About the Environment and Health
Risks Our work over the past few years has shown that very little is known
about the risks of potential exposure to chemicals and environmental
conditions for workers, the general public, and plant and animal life. For
example, we reported the following:
� EPA's Integrated Risk Information System, which is a database of the
agency's consensus on the potential health effects of chronic exposure to
various substances found in the environment, lacks basic data on the
toxicity of about two- thirds of the known hazardous air pollutants. 2
� EPA's National Water Quality Inventorydoes not accurately describe water
quality conditions nationwide. Only 19 percent of the nation's rivers and
streams were assessed for the 1996 Inventory( the latest report available at
the time of our review),
2 Major Management Challenges and Program Risks: Environmental Protection
Agency( GAO/ OCG- 99- 17, Jan. 1999).
4 as were 6 percent of ocean and other shoreline waters. Pollution of the
latter has
resulted in an increasing number of beach advisories and closures in recent
years. 3
� Of 1,456 toxic chemicals we recently reviewed, data on human exposure were
being collected for only about 6 percent. For example, of the 476 chemicals
that EPA identified as most in need of testing under the Toxic Substances
Control Act, only 10, or 2 percent, were being measured for human exposure.
(See table 1.)
Table 1: Extent to Which Human Exposure Data Are Collected for Potentially
Harmful Chemicals Through Surveys of EPA and the Department of Health and
Human Services
Priority chemicals Chemicals measured or being measured Number Description
of list In list Number Percentage
Chemicals found most often at the national Superfund 275 62 23 sites and of
most potential threat to human health EPA's list of toxics of concern in air
168 27 16 Chemicals harmful because of their persistence in the 368 52 14
environment, tendency to bioaccumulate in plant or animal tissues, and
toxicity Pesticides of potential concern as listed by EPA's 243 32 13 Office
of Pesticide Programs and the U. S. Department of Agriculture's Pesticide
Data Program Chemicals that are reported in the Toxic Release 579 50 9
Inventory; are considered toxic; and are used, manufactured, treated,
transported, or released into the environment Chemicals most in need of
testing under the Toxic 476 10 2 Substances Control Act (Master Testing
List)
Note: Our analysis was based on human exposure data collected through the
Department of Health and Human Services' National Health and Nutrition
Examination Survey or EPA's National Human Exposure Assessment Pilot Surveys
through 2000.
EPA has recognized that it has numerous and significant gaps in its data and
has initiated several efforts to fill at least some of the gaps. For
example, under its Environmental Monitoring and Assessment Program, EPA is
working with other federal agencies to develop information that the public,
scientists, and the Congress can use to evaluate the overall health of the
nation's ecological resources. EPA also recently launched its High
3 Water Quality: Key EPA and State Decisions Limited by Inconsistent and
Incomplete Data( GAO/ RCED- 00- 54, Mar. 15, 2000).
5 Production Volume Challenge Program, which asked chemical companies to
voluntarily
generate data on the effects of the chemicals they manufacture or import. As
of December 1999, over 400 participants had agreed to make public, before
the end of 2005, basic hazard data on over 2,000 of 2,800 high- production-
volume chemicals, which are chemicals manufactured or imported into the
United States in amounts equal to or greater than one million pounds per
year. Furthermore, EPA's new information office will be responsible for
encouraging the agency's program offices to reach out to other federal
agencies as well as to universities, research institutes, and other sources
of environmental information for data that EPA does not collect but that may
exist elsewhere. To date, however, such efforts have been hampered by
technological limitations imposed by the myriad of incompatible information
systems in use across the government.
Moreover, much of the information needed, such as environmental monitoring
data, will be expensive to obtain. Thus, it will be important for EPA to
work with the states and industry to reduce the reporting burden and to
encourage efforts to use data that may already have been collected by other
federal agencies or other entities. Likewise, as we recommended to EPA in
our September 1999 report on its information management activities, it will
be essential for the agency to develop a strategy that prioritizes its
requirements for additional data and identifies milestones and needed
resources. EPA can then use this information to support its budget requests.
Incompatible Data Systems Limit the Usefulness of Environmental Data Over
the years, EPA has developed and maintained “stovepipe” data
systems that are not capable of sharing the enormous amounts of data
gathered. EPA now recognizes that common data definitions and formats, known
as data standards, are essential to its efforts to integrate data from
various databases, including those of its state partners. EPA also considers
data standards as key to reducing the reporting burden on industry and the
states because such standards would permit integrated, and thus more
efficient, reporting of information to the agency. In recent years, EPA has
undertaken several efforts to develop standards for some of the data items
in its information systems.
6 According to the Office of Environmental Information, EPA recently
approved six data
standards and expects that all of these standards will be implemented in the
relevant data systems by fiscal year 2003.
EPA recognizes that its current data improvement efforts are only first
steps toward its goal of full data integration. For example, EPA has focused
primarily on the compatibility of its data with those of state environmental
agencies, rather than of other federal agencies and nongovernmental sources.
In a May 2000 report, we stated that improved collaboration among federal
agencies in meeting the needs for human exposure data is essential because
individual agencies have different capacities and skills and separate
attempts have fallen short of supporting the large efforts that are needed.
4 EPA's Science Advisory Board 5 has also recommended that EPA do more to
link the agency's databases with external databases. The Board noted that
“answering many health- related questions frequently requires linking
environmental data with census, cancer or birth registry data, or other data
systems (such as water distribution maps) to determine whether there is a
relationship between the environmental measures and health.” 6 EPA
officials acknowledge the importance of linking EPA's databases with those
of other agencies at all levels of government. However, they told us that
their actions to do so have been limited by resource constraints and by the
fact that EPA's statutes do not give the agency the authority to require
that other agencies collect or report data using formats compatible with
those used by EPA.
Concerns Persist About the Accuracy of EPA's Data In various reviews, we and
others have identified persistent concerns about the accuracy of the data in
many of EPA's information systems. EPA acknowledges that data errors exist
but believes that, in the aggregate, its data are of sufficient quality to
support its programmatic and regulatory decisions. However, EPA has not
assessed the accuracy of
4 Toxic Chemicals: Long- Term Coordinated Strategy Needed to Measure
Exposures in Humans (GAO/ HEHS- 00- 80, May 2, 2000). 5 The EPA Science
Advisory Board was created by the Congress to provide advice to EPA from
scientists outside the agency. 6 Science Advisory Board, Review of the
Agency- Wide Quality Management Program, EPA- SAB- EEC- LTR98-
003 (Washington, D. C.: EPA, July 24, 1998).
7 its information systems agencywide, and preventing errors and correcting
them once
they have been identified has proved daunting for the agency. For example,
in January 1998, an EPA advisory council on information management issues
described the difficulty of correcting errors in EPA's databases:
“Once an error is stored in one or more of the agency's systems,
making corrections to all those systems is an exercise in frustration and
futility. There is no simple way to ensure corrections are made to all
possible systems.”
To address such problems, EPA revised its agencywide quality system in 1998
to expand and clarify requirements for how environmental data are collected
and managed. Although the Science Advisory Board recently commended the
agency for its development of this system, the Board also found that its
implementation has been uneven within the agency. Moreover, the Board
reported that more than 75 percent of the states authorized to implement
EPA's environmental programs lack approved quality management plans for all
or some of these programs and thus are likely to be generating data of
unknown quality. We recently reported that EPA's National Water Quality
Inventory, which EPA uses as a basis for measuring progress under the Clean
Water Act, does not accurately describe water conditions nationwide. While
EPA prepares the Inventoryon the basis of data submitted by the states, the
states do not use a statistical sampling design that provides a
comprehensive picture of water quality. The Science Advisory Board has
pointed out that EPA programs that rely on data of unknown quality are
exposing themselves, the reliability of their decisions, and their
credibility to criticisms.
Correcting errors in the agency's data is an important responsibility for
the new information office. This office recently developed an Internet-
based system to identify, track, and resolve errors found in national
environmental databases. The system currently allows individuals to notify
EPA of suspected errors in some of the agency's major databases, and EPA
intends to implement the data correction system in additional databases
during the next two years.
8
Efforts to Develop Outcome- Oriented Performance Measures Are Constrained by
Data Limitations
Well- chosen environmental measures inform policymakers, the public, and EPA
managers about the condition of the environment and provide for assessing
the potential danger posed by pollution and contamination. They also serve
to monitor the extent to which EPA's programs contribute to environmental
improvement and can be used in future priority- setting, planning, and
budgeting decisions. EPA has been aware of the need for environmental
measures since the mid- 1970s. Nevertheless, the agency made little progress
in developing such measures until the Results Act mandated their use by
requiring federal agencies to report annually on their progress in meeting
performance goals. Under the Results Act, EPA has begun to set goals and
measures that are intended to help the agency, as well as the Congress and
the public, assess the environmental results of the agency's activities.
While EPA has made progress in adopting more measures that reflect the
environmental or health outcomes of programs, the overwhelming number of
EPA's measures reflect outputs, such as the number of inspections performed
or regulations issued, and additional progress is needed.
EPA considers getting the data needed to measure results its biggest
challenge in developing outcome- oriented performance measures. To date, EPA
and the states have made limited progress in developing such measures, as
these examples indicate:
� Of the 364 measures of performance that EPA has developed for use during
fiscal year 2000, only 69 (19 percent) are environmental outcomes; the other
measures reflect program activities, such as the number of actions taken to
enforce environmental laws. (See table 2.)
� Given inherent uncertainties about the results of research and development
activities, the problem of developing outcome- oriented measures is
particularly difficult for EPA's science activities. Of 36 measures related
to EPA's strategic goal of “sound science,” only 2 reflect
outcomes.
9
Table 2: EPA's Analysis of the Number and Type of Annual Performance
Measures for Its Strategic Goals for Fiscal Year 2000
Number of annual performance measures EPA's strategic goal Output Outcome
Total
Goal 1: Clean Air 19 14 33 Goal 2: Clean and safe water 65 17 82 Goal 3:
Safe food 16 1 17 Goal 4: Preventing pollution and reducing 28 14 42 risk in
communities, homes, workplaces, and ecosystems Goal 5: Better waste
management, 34 8 42 restoration of contaminated sites, and emergency
response Goal 6: Reduction of global and 27 7 34 cross- border environmental
risks Goal 7: Expansion of Americans' right to 28 3 31 know about their
environment Goal 8: Sound science, improved 34 2 36 understanding of
environmental risk and greater innovation to address environmental problems
Goal 9: A credible deterrent to pollution and 15 3 18 greater compliance
with the law Goal 10: Effective management 29 0 29
Total 295 69 364
Source: GAO's analysis of EPA data.
In addition to establishing output- and outcome- oriented performance
measures, EPA has adopted a framework for categorizing its performance
measures according to the type of outputs or outcomes to be achieved. As
shown in figure 1, most of the performance measures are outputs involving
either research and development efforts or actions by EPA, states, tribes,
or other governmental bodies, such as establishing standards for hazardous
levels of lead in paint, dust, and soil. The other categories represent
outcomes, including measures that focus on risks to ecology, health, or
welfare; pollutants absorbed by the body; and concentrations of pollutants
in the environment. Over time, EPA plans to increase the number of such
measures, as it is able to obtain better data linking its program activities
with changes in environmental and health conditions.
10
Figure 1: Number and Percentage of Performance Measures for Each Type of
Activity
Even with better data, it will be a major challenge for EPA to link its
environmental programs and activities to outcomes. Environmental conditions
may change because of a number of factors, including variables such as the
weather or economic activity, many of which are beyond the control of EPA
and its state partners. Likewise, it may be difficult to show the
relationship between EPA's annual program activities and some outcomes that
may not be apparent until many years later. For example, current EPA
activities to reduce the amount of polluting nutrients from fertilizers in
the ground may not result in improved water quality for a decade or more.
EPA program officials recognize that they need additional measures that show
the outcomes of programs, and they have recently taken actions that should
strengthen the agency's ability to develop them. For example, EPA is
developing processes and long
<1% 1%
3% 7%
8% 24% 57%
Environmental risks or impacts to ecology, health, or welfare (1)
Pollutants absorbed by the body (9)
Research and development (86) Discharges and emissions of
pollutants (30) Change in behavior of
regulated parties (24) Concentrations of pollutants (5)
Actions by EPA, states, tribes, or other governmental bodies (209)
Total: 364 performance measures
11 term strategies to improve the quality of performance measures and link
the activities of
program offices with environmental results. However, substantial resources
are required to identify and test the potential measures. Once the measures
are established, gathering and analyzing the data can be resource-
intensive, and it can take years to show environmental improvement.
Observations
Our prior work has identified numerous problems in the quality of EPA's data
and the way that the agency manages its data systems. These problems cut
across the various programs regulated by EPA and have limited the agency's
ability to assess risks and measure environmental results. To its credit,
EPA has initiated actions to improve its information management activities.
While EPA has made progress, its initiatives do not provide a long- term
strategy to ensure the completeness, compatibility, and accuracy of its
data. Furthermore, the initiatives have encountered obstacles that highlight
the difficulties facing EPA as it attempts to improve its information
management activities.
As we recommended in our September 1999 report, to substantially improve the
quality of the data used to set risk- based priorities and report on
progress toward improving environmental conditions and human health, EPA
needs to develop a strategy that reflects a long- term commitment to
resolving data problems. Such a strategy should include establishing
milestones and identifying the resources necessary to fill major data gaps,
identify and develop all needed data standards and implement them in key
databases, and coordinate the agency's data standardization efforts with
those of the states, federal agencies, and other organizations. This effort
would provide both senior agency managers and the Congress with what is now
missing-- the information they need to make the best decisions possible on
the costs, benefits, and trade- offs involved in providing scarce resources
to meet critical data requirements. Although EPA concurred with our
recommendation, the agency has made little progress toward developing and
implementing a comprehensive strategy. For example, EPA recently informed us
that it has not yet completed the first stage of a multi- phase effort to
develop an information
12 plan for the agency. EPA plans to complete the first stage by December
2000, which will
identify broad options for information management over the next several
years. ----
Mr. Chairman, I would be happy to respond to any questions that you or other
Members of the Committee may have.
Contact and Acknowledgments
For further information, please contact Peter F. Guerrero at (202) 512-
6111. Individuals making key contributions to this testimony included Ed
Kratzer and Cecilia Lee.
(360000)
Orders by Internet
For information on how to access GAO reports on the Internet, send an e-
mail message with “info” in the body to
info@ www. gao. gov or visit GAO's World Wide Web home page at http:// www.
gao. gov
To Report Fraud, Waste, and Abuse in Federal Programs
Web site: http:// www. gao. gov/ fraudnet/ fraudnet. htm E- mail: fraudnet@
gao. gov Automated answering system: 1- 800- 424- 5454
*** End of document ***