Human Health Risk Assessment: EPA Has Taken Steps to Strengthen
Its Process, but Improvements Needed in Planning, Data
Development, and Training (31-MAY-06, GAO-06-595).
Over 100,000 chemicals, pollutants, and toxic substances are used
in the United States and regulated by the Environmental
Protection Agency (EPA). EPA uses risk assessment to determine
the health risk from exposure to these substances, collectively
referred to as contaminants. In the last 12 years, independent
reviewers have examined this process and made recommendations for
how it could be improved. GAO was asked to (1) identify the
significant recommendations that have been made to improve human
health risk assessment; (2) describe what EPA has done to modify
its human health risk assessment process; (3) determine the
effects these past modifications have had on the preparation of
risk assessments; and (4) identify any additional actions experts
believe EPA could take to improve its process, and the barriers
it would face in doing so.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-595
ACCNO: A54902
TITLE: Human Health Risk Assessment: EPA Has Taken Steps to
Strengthen Its Process, but Improvements Needed in Planning, Data
Development, and Training
DATE: 05/31/2006
SUBJECT: Chemicals
Contaminants
Environmental research
Evaluation methods
Health hazards
Internal controls
Policy evaluation
Pollutants
Risk assessment
Toxic substances
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-595
* Results in Brief
* Background
* Independent Reviewers Have Recommended Improvements to EPA's
* Improve Planning and Review Process and Ensure an Appropriat
* Improve Planning and Adopt an Iterative Risk Assessment Appr
* Increase Peer Review and Stakeholder Involvement in the Risk
* Ensure EPA Has the Skilled Workforce Needed to Conduct Quali
* Improve Characterization of Risk and More Closely Link Risk
* Review Existing Data, Identify Data Gaps, Prioritize Researc
* More Clearly Indicate Use of Default Options and How They We
* Better Evaluate Available Methods and Models and Incorporate
* More Explicitly Analyze and Characterize the Sources of Unce
* Enhance Analysis of Variability in Exposure Levels and Healt
* Better Consider the Human Health Effects of Exposure to Mult
* EPA Has Strengthened Many Facets of Its Risk Assessment Proc
* EPA Has Enhanced Implementation of Key Aspects of Its Risk A
* EPA Has Issued Many Agencywide and Program Office-Specific G
* EPA Has Built Scientific Capacity through Increased Focus on
* EPA Has Begun to Incorporate Planning Activities and Stakeho
* EPA Has Refined Its Peer Review and Quality-Assurance Practi
* EPA Has Enhanced Its Strategic Planning and Refocused Staff
* EPA Restructured and Augmented Its Research Office
* EPA Has Enhanced Strategic Research Planning
* EPA Program Offices Have Established Collaborative Relations
* EPA Maintains Databases of Information Related to Risk Asses
* EPA Has Improved Its Choice and Communication of Default Opt
* EPA Has Taken Steps to Enhance Its Modeling Capabilities and
* EPA Is Introducing More Quantitative Characterization of Unc
* EPA Better Analyzes and Communicates Variability in both Exp
* EPA Has Begun to Consider the Combined Effects of Exposure t
* EPA Risk Assessors Responding to Our Survey Reported That Pr
* EPA Has Issued Numerous Guidance Documents That Have Been Ge
* Collaboration Has Improved EPA's Ability to Conduct Risk Ass
* EPA's Training Has Been Helpful, but Risk Assessors and Mana
* Risk Assessors Have Enhanced Their Skills through Training i
* Additional Training Is Needed
* Enhanced Risk Assessment Planning, Improved Analysis and Rev
* Enhance Planning by Increasing Focus on Data Needs and Invol
* More Thoroughly Evaluate and Transparently Document Analytic
* More Training Could Improve Risk Assessor, Risk Manager, and
* EPA Faces Barriers to Improving Its Risk Assessment Process,
* Conclusions
* Recommendations for Executive Action
* Agency Comments
* GAO Contact
* Staff Acknowledgments
* GAO's Mission
* Obtaining Copies of GAO Reports and Testimony
* Order by Mail or Phone
* To Report Fraud, Waste, and Abuse in Federal Programs
* Congressional Relations
* Public Affairs
Report to the Chairman, Committee on Environment and Public Works, U.S.
Senate
United States Government Accountability Office
GAO
May 2006
HUMAN HEALTH RISK ASSESSMENT
EPA Has Taken Steps to Strengthen Its Process, but Improvements Needed in
Planning, Data Development, and Training
GAO-06-595
Contents
Letter 1
Results in Brief 5
Background 8
Independent Reviewers Have Recommended Improvements to EPA's Risk
Assessment Process Since 1994 12
EPA Has Strengthened Many Facets of Its Risk Assessment Process Since
1994, and Efforts Are Ongoing 24
EPA Risk Assessors Responding to Our Survey Reported That Process
Modifications Have Helped Them Prepare Better Risk Assessments but That
Collaboration and Training Limitations Hamper Further Progress 51
Enhanced Risk Assessment Planning, Improved Analysis and Review, and Added
Training Could Further Improve EPA's Process, but Barriers Could Limit
Progress 62
Conclusions 67
Recommendations for Executive Action 69
Agency Comments 69
Appendix I Objectives, Scope, and Methods 71
Appendix II GAO Contact and Staff Acknowledgments 75
Figures
Figure 1: Typical Sequence of Risk Assessment and Risk Management
Processes 9
Figure 2: Helpfulness of Office-Specific Guidance 53
Figure 3: Risk Assessors That Generally or Strongly Agree That
Organizations Help Fill Data Needs 55
Figure 4: Areas in Which Training Has Moderately or Greatly Improved Risk
Assessor Knowledge and Skills 58
Figure 5: Types of Training Taken by Risk Assessors 59
Figure 6: Modifications in Training That Would Moderately or Greatly
Improve the Preparation of Risk Assessments 61
Abbreviations
ASPH Association of Schools of Public Health
ATSDR Agency for Toxic Substances and Disease Registry
BOSC Board of Scientific Counselors
Commission Presidential/Congressional Commission on Risk Assessment and
Risk Management
CREM Council for Regulatory Environmental Modeling
EPA Environmental Protection Agency
FQPA Food Quality Protection Act
GIS Geographic Information System
HEI Health Effects Institute
HPV High Production Volume
ILSI-RSI International Life Sciences Institute-Risk Science Institute
IRIS Integrated Risk Information System
NAS National Academy of Sciences
NCCT National Center for Computational Toxicology
NCEA National Center for Environmental Assessment
NCER National Center for Environmental Research
NERL National Exposure Research Laboratory
NHEERL National Health and Environmental Effects Research Laboratory
NHEXAS National Human Exposure Assessment Survey
NIEHS National Institute for Environmental Health Sciences
NRMRL National Risk Management Research Laboratory
OAQPS Office of Air Quality Planning and Standards
OMB Office of Management and Budget
OP Office of Pesticides
OPPT Office of Pollution Prevention and Toxics
ORD Office of Research and Development
OW Office of Water
STAR Science to Achieve Results
TRIM Total Risk Integrated Methodology
TSCA Toxic Substances Control Act
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office
Washington, DC 20548
May 31, 2006 May 31, 2006
The Honorable James M. Inhofe Chairman Committee on Environment and Public
Works United States Senate The Honorable James M. Inhofe Chairman
Committee on Environment and Public Works United States Senate
Dear Mr. Chairman: Dear Mr. Chairman:
Chemicals play an important role in people's everyday lives. Consumers use
products containing chemicals, such as cleansers and pesticides, and
companies use a variety of toxic substances as solvents or additives to
their industrial processes. Although these substances have beneficial uses
and are important in producing goods and services, some may adversely
affect human health. Over 100,000 chemicals, pollutants, and toxic
substances are used in the United States and regulated by the
Environmental Protection Agency (EPA). These substances, whether produced
in the United States or imported, include 187 hazardous air pollutants,
nearly 20,000 pesticide products on the market, and a growing number of
substances used in industrial processes-over 82,000 as of December 2005.
EPA uses a process known as risk assessment to estimate the health risk
from exposure to these substances, collectively referred to in this report
as contaminants.1 While highly technical by nature, risk assessments,
along with other relevant information, serve as a basis for regulatory
decisions that protect human health. EPA's risk assessments are shaped by
available scientific information and by provisions in major environmental
statutes, such as the Clean Air Act, and they affect a range of
stakeholders, including regulated industries, federal agencies,
environmental advocacy groups, academic and other researchers, and the
public. 21 2Chemicals play an important role in people's everyday lives.
Consumers use products containing chemicals, such as cleansers and
pesticides, and companies use a variety of toxic substances as solvents or
additives to their industrial processes. Although these substances have
beneficial uses and are important in producing goods and services, some
may adversely affect human health. Over 100,000 chemicals, pollutants, and
toxic substances are used in the United States and regulated by the
Environmental Protection Agency (EPA). These substances, whether produced
in the United States or imported, include 187 hazardous air pollutants,
nearly 20,000 pesticide products on the market, and a growing number of
substances used in industrial processes-over 82,000 as of December 2005.
EPA uses a process known as risk assessment to estimate the health risk
from exposure to these substances, collectively referred to in this report
as contaminants. While highly technical by nature, risk assessments, along
with other relevant information, serve as a basis for regulatory decisions
that protect human health. EPA's risk assessments are shaped by available
scientific information and by provisions in major environmental statutes,
such as the Clean Air Act, and they affect a range of stakeholders,
including regulated industries, federal agencies, environmental advocacy
groups, academic and other researchers, and the public.
1Although other regulatory agencies, such as the Food and Drug
Administration, also use risk assessments as part of regulatory decision
making, this report discusses only the risk assessment process used by
EPA.
2EPA's mission is to protect human health and to safeguard the natural
environment-air, water, and land-upon which life depends. This report
focuses only on EPA's efforts to protect human health.
In 1994, as part of a congressionally mandated review of the methods used
by EPA to estimate the risk of developing cancer from exposure to
hazardous air pollutants, the National Academy of Sciences (NAS) issued a
report entitled Science and Judgment in Risk Assessment. This report
focused primarily on hazardous air pollutants and provided an overview
assessment of EPA's risk assessment methods.3 In providing this overview,
the NAS committee identified several themes, largely focusing on the
quality and availability of essential information, that serve as a useful
framework for discussing the risk assessment process:
o Implementation. EPA faces certain overarching, institutional
issues that affect its implementation of the risk assessment
process.
o Data needs. Data necessary to complete scientifically plausible
risk assessments are often unavailable on such topics as the
actual levels of exposure to contaminants and how those exposures
affect human health.
o Default options. In the absence of convincing scientific
knowledge or data, EPA relies on assumptions, often conservative
in nature, about such questions as how exposure to low doses of a
contaminant affects human health.
o Method and model evaluation. The predictive accuracy of methods
and models, such as those used to predict how a contaminant will
be processed once it enters the body, is not always known.
o Uncertainty. The lack of precise knowledge about the type,
likelihood, and extent of adverse effects from exposure to a
contaminant results in uncertainty in risk assessment that can be
reduced only by advances in scientific understanding or the
collection of better data.
o Variability. Variability, in exposure or in the biological
differences among humans that determine how exposure to
contaminants affects health, can be better characterized with more
data but cannot be reduced or eliminated.
o Aggregate and cumulative effects. Aggregate effects are the
results from exposure to a single contaminant by any combination
of means-including inhalation, contact with the skin, and
ingestion; cumulative effects are the results from exposure to
multiple contaminants by any combination of means.
Science and Judgment in Risk Assessment was one of several reports
by NAS that discussed risk assessment in general and made
recommendations for ways to improve it.4 In addition, the
Presidential/Congressional Commission on Risk Assessment and Risk
Management issued a report in 1997 that reviewed risk assessment,
as well as risk management, in regulatory programs.5 Furthermore,
the Office of Management and Budget (OMB) issued guidance related
to peer review and ensuring the quality, objectivity, utility, and
integrity of information released to the public.6 More recently,
in January 2006, OMB issued a proposed bulletin that advocates
minimum standards for the scientific quality of risk assessments
to enhance their technical quality and objectivity.
In the context of these prior reviews, you asked us to (1)
identify the significant recommendations to improve human health
risk assessment that have been made since 1994; (2) describe what
EPA has done to modify its human health risk assessment process
over the same period; (3) determine the effects these past
modifications have had on the preparation of risk assessments; and
(4) identify any additional actions experts believe EPA could take
to improve its risk assessment process in the future, and the
barriers EPA would face in doing so.
To identify significant recommendations to improve human health
risk assessment since 1994, we reviewed EPA documents, including
those produced by EPA's Risk Assessment Forum, Science Policy
Council, Council for Regulatory Environmental Modeling, and
Science Advisory Board, as well as each of EPA's program offices.
We also reviewed our own reports and documents produced by NAS and
the Presidential/ Congressional Commission on Risk Assessment and
Risk Management. We also spoke with experts in the risk assessment
field, who identified many of these documents in the course of our
discussions and provided insight into some of the recommendations
made therein. To describe what EPA has done to modify its human
health risk assessment process, we interviewed program office
managers from EPA's Office of Air and Radiation, Office of
Pesticide Programs, Office of Pollution Prevention and Toxics,
Office of Water, and Office of Research and Development (ORD).
While we recognize that EPA's regional offices and the Office of
Solid Waste and Emergency Response conduct risk assessments, for
the most part their work focuses on site-specific assessments and
thus was not included in this effort. Within ORD, we interviewed
managers in two of EPA's laboratories (the National Health and
Environmental Effects Laboratory and the National Exposure
Research Laboratory) and three of EPA's research centers (the
National Center for Environmental Assessment, National Center for
Environmental Research, and National Center for Computational
Toxicology). Furthermore, we attended various EPA and stakeholder
group training sessions and meetings.
To determine the effects these modifications have had on the
preparation of risk assessments, we surveyed risk assessors from
the program offices mentioned above. We sought their views on the
usefulness of many of these modifications, as well as on aspects
of EPA's risk assessment process, including guidance documents,
training, organizational structure, and collaboration. The survey
and results can be viewed at
http://www.gao.gov/cgi-bin/getrpt?GAO-06-637SP . In addition, we
interviewed experts in the risk assessment field and obtained
their perspectives on the extent to which these modifications were
helpful to EPA's risk assessment process. To identify additional
actions experts believe EPA could take to improve its process and
to identify barriers to such actions, we contacted experts
representing a range of stakeholders in the process. Specifically,
we contacted risk assessment scientists; toxicologists; scientific
advisers to EPA; state officials; and representatives from
regulated industries, government agencies, and environmental
advocacy groups who have an expertise in risk assessment. We used
an iterative process (often referred to as the "snowball sampling"
technique) to identify these knowledgeable experts and selected
for interviews those who would provide us with a broad and
balanced range of perspectives on EPA risk assessment practices.
We used a standard set of questions to interview each of these
experts to ensure we consistently discussed each aspect of EPA
risk assessment policies and practices. (See app. I for a more
detailed description of the methodology we employed.) In addition,
we consulted with NAS's Board on Environmental Studies and
Toxicology at its June 2005 meeting to obtain input on the areas
in which EPA has made the most progress in improving its risk
assessment practices and areas EPA will need to focus on in the
future. We used the experts present at this meeting as a starting
point for our snowball sampling technique used to identify
subsequent experts. We conducted our work from February 2005
through March 2006 in accordance with generally accepted
government auditing standards.
Results in Brief
Independent reviewers, including NAS, have made a number of
significant recommendations to improve EPA's human health risk
assessment process since 1994 that can be grouped and discussed in
terms of the seven themes originally identified by NAS in 1994.
First, reviewers have said that EPA should improve its overall
implementation of the process with such steps as more thorough
planning of its risk assessments and greater use of independent
reviews by scientists (called peer reviews). Second, to improve
the quantity and quality of the data upon which risk assessments
are based, reviewers have recommended that EPA better utilize the
scientific data it has and identify additional data needs on the
potential adverse effects from exposure to contaminants, and
prioritize and support research to meet the data needs identified.
Third, because EPA lacks data to fully assess the health risk from
exposures, reviewers have recommended that when the agency relies
on assumptions-known as default options-it should, among other
things, more clearly indicate when it relies on default options
and how it chooses them. Fourth, to improve the accuracy of EPA's
risk assessments, reviewers have stated that EPA should better
evaluate the methods and models used in its analysis and
incorporate newer, more powerful tools when appropriate. Fifth, to
address the inevitable uncertainties associated with gaps in
scientific knowledge and general unknowns about model and data
accuracy, EPA should, according to reviewers, more explicitly
analyze and characterize the sources of uncertainty in its risk
assessments and, when possible, discuss the uncertainties both
descriptively (qualitatively) and numerically (quantitatively).
Sixth, reviewers have also recommended that EPA enhance its
analysis of variability in levels of exposures to contaminants, as
well as differences in individual reactions to exposure. Finally,
because people are typically exposed to a mixture of contaminants
through a variety of means, such as contact with skin and
breathing air, reviewers have recommended that EPA improve how it
considers the effects of combinations of contaminants and all
possible means of exposure.
EPA has strengthened many aspects of its risk assessment process
since 1994, and improvement efforts are ongoing. First, EPA has
improved implementation of its risk assessment process by, for
example, beginning to improve risk assessment planning and
creating scientific leadership positions, such as the Office of
the Science Advisor. Second, EPA has initiated actions to develop
missing or incomplete scientific data on the potential adverse
effects from exposure to contaminants. For example, EPA officials
told us they have begun to use the planning and review processes
to determine what data are needed and communicate these needs to
both EPA and outside researchers. Third, EPA has cited the need
for risk assessments to be more transparent about their use of
default options. For example, EPA has recently issued guidance
describing default options that are appropriate for certain
purposes and directing risk assessments to disclose the default
options used in a particular risk assessment. Fourth, EPA is
enhancing its use of models and embracing new methodologies. For
example, EPA has established a group-the Council for Environmental
Regulatory Modeling-to review models and provide guidance in model
selection. Furthermore, some program offices now use more advanced
models to predict how a contaminant will be processed once it
enters the body. Fifth, EPA generally characterizes uncertainty
descriptively, but has begun to incorporate quantitative
techniques into its uncertainty analyses. For example, EPA
typically uses a tiered approach to conducting uncertainty
analysis, starting as simply as possible, describing uncertainty
qualitatively, and sequentially employing more sophisticated
analysis, such as probabilistic analysis, as warranted. Sixth, EPA
has made progress in describing variability due to differences in
both the exposure and the susceptibility of exposed individuals to
contaminants. For example, when determining a level of exposure
that is unlikely to be harmful, EPA includes adverse effects for
people who might be at increased risk because of their age or the
state of their health. Finally, EPA has begun to take steps to
consider the combined effects of exposure to multiple contaminants
through multiple means of exposure. For example, EPA has developed
a framework to assess the combined, or cumulative, risk and has
directed its program offices to include cumulative risks when
planning major risk assessments.
EPA risk assessors believe the modifications EPA has made over the
past 10 years-particularly issuing additional guidance-have
generally helped them improve risk assessments, but collaboration
and training could be improved. Most EPA risk assessors believe
that agencywide guidelines and policy and reference documents have
helped them prepare risk assessments by, for example, providing
useful frameworks for evaluating potential harm from chemicals,
and have resulted in greater consistency among risk assessments
prepared by different offices within EPA. Furthermore, EPA's
collaboration with external researchers, including other federal
research entities, academia, and industry, has improved EPA's
ability to conduct risk assessments by providing expertise and
research not always available within the agency, according to risk
assessors. In addition, risk assessors reported that internal
collaboration among EPA offices is moderately to very effective,
but could be improved. For example, a few risk assessors commented
that the program offices' knowledge of the regulatory context in
which research will be used helps the researchers structure their
work. However, risk assessors also reported that collaboration is
hindered within EPA by, for example, conflicting priorities among
the various offices and the poor communication between some of
them. Finally, while risk assessors said improved training has
also helped them gain relevant skills and knowledge, over 70
percent of the risk assessors responding to our survey stated that
more in-depth or relevant training would improve their ability to
prepare risk assessments. In addition, they believe training for
risk assessors and managers in specific technical and scientific
areas, such as emerging scientific issues and the use of newer
models, is lacking.
While the experts we spoke with said the modifications EPA has
made over the past 10 years have been beneficial overall, they
identified additional actions EPA could take to improve its risk
assessment process, recognizing that EPA may face barriers to
doing so. Specifically, experts said EPA could improve the
planning it undertakes prior to starting a risk assessment by
better focusing on what data are needed for the assessment and by
involving stakeholders early in this planning process. For
example, several experts said that increased involvement with a
broad range of stakeholders early in the planning process would
help identify alternative methods and models and obtain
stakeholder concurrence with the agency's approach. In addition,
experts said EPA could more thoroughly evaluate methods and
models, transparently document its analytic choices, and enhance
internal review. For example, several experts said that EPA should
more transparently communicate which default assumptions were used
in risk assessments, why the defaults were chosen, and what
judgments EPA was making when it employed certain methods.
Finally, experts said EPA could provide additional training for
risk assessors, managers, and stakeholders on all elements of the
risk assessment process, such as how to use and apply models and
how to interpret data from emerging scientific fields. While these
efforts would further improve the risk assessment process, experts
pointed out inherent barriers that EPA may face in carrying them
out, such as the scientific complexity of risk assessment, the
difficulty of obtaining and applying data, and a cultural
resistance to deviating from established methods. For example,
several experts said that EPA's risk assessments have grown more
technically challenging and require risk assessors and managers to
have different skills from what they had in the past.
To further improve the risk assessment process, GAO recommends
that EPA enhance early planning of each risk assessment, identify
and communicate data needs to the research community, and support
development and implementation of in-depth training for risk
assessors and managers.
We provided EPA with a draft of this report for review and
comment. EPA neither agreed nor disagreed with our findings and
recommendations. However, the agency provided specific comments to
improve the report's technical accuracy, which we incorporated as
appropriate.
Background
Risk assessment, as used in public health and environmental
settings, is inherently a complex and highly technical process
that provides a systematic scientific description of potential
adverse effects from exposure to contaminants. Risk assessments
are conducted to estimate whether and how much harm can be
expected from exposure to a given contaminant or mixture of
contaminants and to help determine whether the harm is significant
enough to require regulatory or other corrective action. The
adverse effects from exposure can be acute, such as respiratory
illnesses or poisonings, or chronic, such as cancer or
reproductive or genetic abnormalities. The result of a risk
assessment is a statement of the likelihood that an exposed
population will be harmed and to what extent. EPA's human health
risk assessment process generally consists of the four stages
recommended in 1983 by the National Academy of Sciences (NAS): (1)
hazard identification (determining whether a substance could cause
adverse affects and what those adverse affects would be), (2)
dose-response assessment (determining the relationship between the
magnitude of exposure to a contaminant and the probability and
severity of adverse effects), (3) exposure assessment (identifying
the extent to which exposure is predicted to occur), and (4) risk
characterization (combining the information from the preceding
analyses into a conclusion about the nature and magnitude of
risk).
Risk assessment plays a critical role in EPA's regulatory
decisions, but the characterization of risk that results from a
risk assessment is only one piece of information used to inform
decisions on regulatory standards for protecting human health.
This analysis is integrated with other information, such as
economic information on the costs and benefits of mitigating the
risk, technological information on the feasibility of managing the
risk, and the concerns of various stakeholders. The combined
analysis informs agency officials who ultimately make regulatory
decisions. This overall process is generally called risk
management. (See fig. 1.) According to NAS, the risk assessment
process should be conceptually distinguished from how the results
of that process are used in the risk management process. However,
in practice, maintaining this separation is difficult because the
two processes are fundamentally linked by the complementary needs
of the risk assessors and the risk managers. Specifically, risk
assessors need to be aware of the context in which the assessment
will be used and to communicate their key findings, as well as
their confidence in them, in a way that risk managers can
understand and apply. Risk managers need to ensure that the risk
assessment has been properly performed and can be integrated with
other information to make and justify a sound regulatory decision.
Figure 1: Typical Sequence of Risk Assessment and Risk Management
Processes
EPA's human health risk assessment process is limited to a great
extent by the amount of scientific data available on contaminants
and by the level of scientific understanding about how
contaminants act in the body to produce adverse effects. Data are
often unavailable, and science cannot always provide definitive
answers. The complex and sophisticated analytic tools used in risk
assessment cannot overcome basic scientific unknowns. As a result,
EPA's risk assessments almost always include assumptions about
potential adverse effects. Some assumptions are conservative-that
is, they are intended to help ensure that the agency does not
underestimate health risks. Some critics of EPA's risk assessment
practices believe EPA uses assumptions that are unjustifiably
conservative, given new scientific data and methods, and thereby
produces estimates that overstate actual risks. Other
stakeholders, however, criticize some agency assumptions for not
being conservative enough in the face of scientific uncertainties.
These criticisms often result in legal challenges to EPA's
decisions.
EPA's human health risk assessment process is also greatly
dependent on its degree of knowledge about the population's level
of exposure to contaminants. This knowledge includes the extent to
which people are exposed to potentially harmful contaminants in
their daily lives, the chemicals to which they are most often
exposed, the levels of such exposures, how exposures change over
time, and the sources of exposure. Risk assessors, researchers,
and policymakers must often rely on estimates of human exposure
that are often derived from data showing the extent the chemicals
are found in the air, water, food, or other environmental media
and assumptions about how and at what rate the body absorbs the
chemicals it contacts. In addition to estimates from models,
extrapolations from experiments involving animals, and
measurements of chemicals in the environment, EPA also relies on
more direct methods to measure exposure and more accurately assess
exactly how much of a contaminant has been absorbed in the body.
For example, EPA uses population activity models-models based on
actual human behavior, such as the time spent outdoors or, for
children, the amount of time spent on the floor-to better estimate
an individual's true exposure.
Risk assessment activities at EPA are carried out by both the
agency's Office of Research and Development (ORD)-its principal
scientific and research arm-and its program and regional offices,
including the Office of Air and Radiation, Office of Pesticide
Programs, Office of Pollution Prevention and Toxics, Office of
Solid Waste and Emergency Response, and Office of Water. ORD
carries out all steps of highly complex, precedent-setting risk
assessments for specific contaminants, such as dioxin. In
addition, ORD often has responsibility for the first two steps of
the risk assessment process-hazard identification and
dose-response assessment-in support of the program offices. In
such cases, the last two steps-exposure assessment and risk
characterization-are the responsibility of the various program
offices. Three notable exceptions exist to this division of
responsibility. The Office of Pesticide Programs and the Office of
Pollution Prevention and Toxics often conduct all steps of risk
assessments independently, in part, because the relevant statutes
for these program offices place strict time frames on decision
making and the confidential nature of data provided to the agency
under these programs. Furthermore, the Office of Water also does
all of the stages for some of the assessments for purposes of the
Safe Drinking Water Act.
EPA's approach to risk assessment varies across program offices,
often as a result of different regulatory and legal requirements.
For example, a branch of the Office of Pollution Prevention and
Toxics assesses data submitted by industry applicants on
approximately 2000 new chemicals annually under the Toxic
Substances Control Act (TSCA) of 1976. TSCA generally requires EPA
to evaluate the chemicals within 90 days, but does not require all
applicants to conduct laboratory tests on the potential hazards
and risks of the chemicals. In contrast, the Clean Air Act
requires EPA to establish National Ambient Air Quality Standards,
to review the scientific basis for those standards at least every
five years, and to revise the standards as appropriate. As part of
this process, ORD summarizes the most current scientific
information on the pollutant in question. ORD's findings and
conclusions are then combined with other exposure and risk
analyses to determine what, if any, revisions should be made to
the standards.
Risk assessment has been the center of numerous reports, analyses,
and regulations over the years. One of the earliest was the
National Academy of Sciences' 1983 report Risk Assessment in the
Federal Government: Managing the Process, often referred to as the
"Red Book" because of the color of its cover. Subsequently, NAS
released several related reports, including Science and Judgment
in Risk Assessment (1994), Understanding Risk: Informing Decisions
in a Democratic Society (1996), and Strengthening Science at the
U.S. Environmental Protection Agency (2000). In addition,
Congress, via the Clean Air Act Amendments of 1990, created the
Presidential/Congressional Commission on Risk Assessment and Risk
Management (Commission) and required it to investigate the policy
implications and appropriate uses of risk assessment and risk
management in various regulatory programs designed to protect
people from cancer and other chronic health effects that may
result from exposure to hazardous substances. The Commission
published its two-volume final report in 1997.
In addition, the Office of Management and Budget (OMB) has issued
guidelines, reports, and bulletins that have affected the practice
of risk assessment. For example, in October 2001, OMB issued its
Guidelines for Ensuring and Maximizing the Quality, Objectivity,
Utility, and Integrity of Information Disseminated by Federal
Agencies, which required agencies to issue their own guidelines to
ensure the quality of information being disseminated. Furthermore,
in September 2003, OMB issued a report to Congress on the costs
and benefits of federal regulation. As part of this effort, OMB
sought public comment on the practice of risk assessment, many of
which were directed at, and passed along to, EPA. OMB also issued
the Final Information Quality Bulletin for Peer Review in December
2004 that provided guidance to federal agencies on what
information is subject to peer review and defined a planning
process for peer review. Most recently, on January 9, 2006, OMB
released a draft bulletin on procedures agencies should use to
conduct risk assessments and information that should be included.
Public comment on this draft will be accepted until June 15, 2006.
Independent Reviewers Have Recommended Improvements to EPA�s Risk
Assessment Process
Since 1994Independent reviewers, including the National Academy of Sciences
(NAS) and the Presidential/Congressional Commission on Risk
Assessment and Risk Management have made a number of significant
recommendations to improve EPA's human health risk assessment
process since 1994. These recommendations cover a range of
actions, including improving planning of what will be required to
complete a particular risk assessment and what key stakeholders
need to be involved, supporting development of new scientific data
on the potential adverse effects from exposure to the contaminant
under review, and enhancing analysis of the potential risk and its
uncertainties. All of them can be summarized and discussed in
terms of the seven themes-implementation, data needs, default
options, method and model evaluation, uncertainty, variability,
and exposure to multiple contaminants and routes of
exposure-reported in NAS's report Science and Judgment in Risk
Assessment.7
Improve Planning and Review Process and Ensure an Appropriately
Skilled Workforce to Conduct Risk Assessment
Independent reviewers made a number of recommendations to EPA to
improve the overall implementation of its risk assessment process.
The reviewers recommended that EPA improve its planning of risk
assessments before beginning the process and adopt an iterative
risk assessment approach. Reviewers also stressed that EPA should
increase its use of risk assessment reviews by independent
scientists, known as peer reviews; increase stakeholder
involvement in the risk assessment process; and ensure it has the
workforce needed to conduct quality risk assessments. Finally, the
reviewers recommended that EPA improve the way it characterizes
the risk it finds and strengthen the link between risk assessment
and risk management.
Improve Planning and Adopt an Iterative Risk Assessment Approach
In 1996, NAS reviewers stressed the importance of planning from
the beginning of a risk assessment to incorporate the perspectives
and knowledge of interested and affected parties.8 Because of the
large number of stakeholders interested in the results of a risk
assessment, it is important to ensure that the risk assessment
addresses all of the stakeholders' needs. NAS pointed out that one
way to do this is to be aware of the stakeholders' concerns from
the outset and incorporate them into the analysis and
characterization of risk. By involving stakeholders early, risk
assessors can ensure that they ask the right questions, make
appropriate assumptions, and determine the best way to summarize
information, thereby potentially making the resulting message more
credible to these parties.
In addition to improved planning, NAS suggested in 1994, and the
Commission concurred, that EPA develop the ability to conduct risk
assessments iteratively-that is, begin with a screening analysis
to ascertain the potential risk and refine that analysis as
appropriate. Such an approach would start with relatively
inexpensive screening techniques and move to more
resource-intensive data gathering and modeling as the particular
situation warranted. To guard against the possibility of
underestimating risk, these screening techniques should be
constructed to err on the side of caution where there is
uncertainty.
Increase Peer Review and Stakeholder Involvement in the
Risk Assessment Process
Each of the NAS reports we reviewed, as well as the report by the
Commission, recommended ways EPA could improve its peer review
process and better involve stakeholders in risk assessment. These
independent reviewers said peer review is critical to evaluate the
accuracy and appropriateness of technical information, scientific
observations, and interpretations used in regulatory decision
making. The Commission observed that peer review can also help in
the planning stages of a risk assessment to put a problem in
context and draw on the knowledge of experienced researchers,
public health officials, and scientists.
In particular, NAS recommended in 2000 that EPA change its peer
review policy to ensure greater independence of these reviews from
the control, or appearance of control, of the program managers.9
EPA's Science Advisory Board had expressed concern about potential
conflicts of interest because agency policy allowed the same
individual to serve as both project manager for a particular work
product and peer review leader.10 The NAS committee concurred with
this concern.
In addition, NAS's 1994 report recommended that EPA collaborate
more with outside parties to improve the overall risk assessment
process. For example, EPA could use external advisory groups, such
as its Science Advisory Board, to help ensure that risk assessment
decisions use the best science and incorporate full public
discussion and participation from the scientific community. Such
collaboration could also include a process for public review and
comment. The Commission also said that agencies should use
advisory groups, composed of stakeholders, to periodically
evaluate the use of technical information. In addition, the
Commission recommended that agencies establish clear, written
guidelines for peer review and match the use of peer review to the
importance of the decision to be made.
In 1996, NAS commented on the importance of stakeholder
involvement in the risk assessment process. NAS suggested that the
risk assessment process involve the spectrum of interested and
affected parties, including decision makers and specialists in
risk analysis, at each step of the process. According to NAS, such
involvement will help ensure that as much important, relevant
knowledge as possible enters the process and that the process
remains open and inclusive. NAS also mentioned another benefit of
involving these stakeholders in the process: the participation of
a spectrum of people helps ensure that the process will be framed
correctly and the resulting risk characterization will be
accurate, balanced, and informative.
Ensure EPA Has the Skilled Workforce Needed to Conduct
Quality Risk Assessments
Each of the NAS reports we reviewed contained recommendations
stressing the importance of an organization's ability to ensure
that it has the expertise and leadership needed to conduct risk
analyses. In addition, according to current workforce models,
agencies need to ensure that they have effective training and the
capability to identify what skills and competencies the employees
and the organization need. In 1994, NAS recommended that EPA
acquire additional expertise, as needed, to better interpret data
and reach sound conclusions about the risks to human health from a
contaminant. For example, many of the conclusions regarding the
potential risks from a particular contaminant rely upon risk
assessment models based on animal data or, when available, human
evidence from epidemiological studies. The quality of the risk
assessment will depend upon how well these data are interpreted to
predict health effects in human populations. NAS recommended that
EPA acquire staff with specialized skills in fields such as
toxicology to successfully complete this type of analysis.
NAS also recommended in 2000 that EPA continue to place high
priority on ORD's graduate fellowship and postdoctoral program.
The report stated that to achieve scientific and technical
excellence, EPA must first attract, retain, and properly support a
dedicated professional staff. NAS also noted that while EPA has
many outstanding scientists and engineers, ORD's workforce is
aging and many staff may retire in the relatively near future.
Periodic hiring freezes, combined with high competition from the
private sector, had made it difficult for ORD to recruit the new
talent needed to sustain and enhance the research workforce. NAS
concluded that ORD's graduate fellowships and postdoctoral
programs brought a stream of fresh scientific and technical talent
into EPA's research program and helped train future researchers in
environmental science, engineering, and other disciplines. NAS
urged EPA to continue to place a high priority on these programs.
In 2000, NAS made a series of recommendations related to
scientific leadership and talent at EPA. Specifically, NAS
recommended that EPA establish a new position-deputy administrator
for science and technology-to address concerns that science at EPA
was not perceived to be strong and that many EPA scientists
believed their scientific knowledge and resources were not
effectively used. Traditionally, EPA's most senior science
official had been the assistant administrator for research and
development, but that official lacked agencywide responsibility or
authority to oversee the scientific and technical basis for
regulatory and policy decision making. In addition, EPA's program
offices were not required to follow scientific advice from ORD.
Consequently, the NAS panel concluded that EPA needed an
appropriately qualified science official at a sufficiently high
level with both the authority and responsibility for agencywide
scientific performance.
Improve Characterization of Risk and More Closely Link Risk
Assessment with the Needs of Risk Managers
Finally, reviewers recommended ways EPA could improve its
characterization of risk and better link its risk assessment and
risk management processes to harmonize the scientific aspects of
risk assessment and the broader policy objectives of risk
management. NAS's 1996 report, which primarily focused on risk
characterization and the role it plays in risk assessment,
stressed that successful risk characterization can result in
better and more widely accepted risk assessment and risk
management decisions. The Commission echoed NAS's position and
recommended that risk characterizations should include information
useful to everyone in the risk management process, such as
information on who is at risk, how they might be affected, what
the severity of an adverse effect might be, and how confident the
risk assessors are about their predictions.
Reviewers also suggested that EPA better link its risk assessment
and risk management processes. EPA had tried to separate these
processes entirely to avoid the possible perception that EPA made
scientific judgments on the risk posed by a contaminant on the
basis of its willingness to regulate the substance rather than on
the science. However, NAS pointed out that while some degree of
judgment is necessary, the science-policy judgments EPA makes in
the course of risk assessment should more clearly be informed by
the agency's risk management priorities and goals. According to
NAS, better linkage between risk assessment and risk management
will help ensure that the risk assessments will be more accurate
and relevant to risk managers. The Commission concurred, stating
that risk assessment should be guided by an understanding of the
issues important to risk managers' decisions about how to protect
public health and the environment.
Review Existing Data, Identify Data Gaps, Prioritize Research Needs,
Foster Development of New Data, and Improve Strategic Research
Planning and Collaboration
In 1994, NAS found, in general, that EPA did not have the full
range of data needed to conduct adequate risk assessments to
estimate the health risks associated with most contaminants-for
example, data related to the effect a contaminant has on the body
and the extent of exposure to the contaminant. These data are
essential to ensure the accuracy and precision of the risk
assessment. In addition, NAS recommended that EPA increase its
collaboration with researchers and improve the transparency of its
research priorities.
To help address EPA's dearth of data, NAS recommended, among other
things, that EPA review its existing databases, such as the
Integrated Risk Information System (IRIS), and identify data
gaps.11 As part of its database review, NAS recommended that EPA
compile an inventory of the chemical, toxicological, and
epidemiological literature in the databases to determine what gaps
exist. NAS further suggested that EPA prioritize its future
research on the basis of the significance of the missing data to
risk assessments, convey its data needs to other public and
private researchers, such as the Agency for Toxic Substances and
Disease Registry,12 and develop incentives to expedite the
generation of needed data by these entities.
Furthermore, in 2000, NAS recommended, among other things, that
EPA expand the multiyear research planning by its Office of
Research and Development (ORD). For example, ORD had developed
plans for research on topics such as particulate matter, endocrine
disruptors,13 and drinking water. The NAS committee pointed out
that these planning efforts will contribute to research program
continuity and the achievement of strategic goals. In addition,
NAS recommended that the plans be both continued for
problem-driven research-research targeted at a particular
environmental problem-and expanded to include core
research-broader, more generic research that will help improve
current and future understanding of a scientific issue. Research
should be balanced between these two types, since core research
will help ORD better understand and anticipate environmental risks
and will enable ORD to provide better problem-specific research
and technical assistance to the agency.
NAS has also made recommendations to EPA in the area of research
accountability. Specifically, NAS recommended that EPA improve the
documentation and transparency of the decision-making process ORD
uses to set research and technical assistance priorities and
allocate funds. For example, NAS pointed to an EPA Science
Advisory Board finding that ORD's lack of transparency in its
process for setting research priorities made it difficult to
evaluate the adequacy of proposed budgets. NAS also recommended
that EPA expand on its efforts to create an inventory of science
projects and programs across EPA by documenting and publishing a
comprehensive and detailed inventory. The inventory should include
information such as goals and objectives of each project,
milestones, schedules, and staff allocations and should be used to
ensure that science activities are properly coordinated through
the agencywide science planning and budgeting process and are
appropriately peer reviewed.
Once the research strategy has been established, research
collaborations can help execute it. NAS suggested that EPA should
recognize the limits of its own research capabilities and develop
a strategy to obtain outside support in research areas where ORD
is not pre-eminent. Specifically, NAS recommended EPA enhance its
research collaborations by developing and implementing a strategy
to stimulate, acquire, and apply the results of research conducted
or sponsored by other federal agencies, state agencies,
universities, and industry in this country and abroad. NAS said
such collaboration is especially important given the budget
constraints EPA and the federal government face and the breadth of
knowledge required to conduct the full range of risk assessments.
In addition, NAS recommended that EPA develop additional
mechanisms to promote and facilitate research interactions among
grantees in EPA's Science to Achieve Results (STAR) grant program
and ORD research staff.14 For example, NAS suggested that grant
applicants could identify in their proposals how their research
might complement or supplement ongoing or planned research in the
ORD laboratories. NAS also recommended that EPA increase its
efforts to disseminate ORD's research products, explain their
significance, and assist others inside and outside the agency in
applying the research. NAS noted that while EPA's policy and
regulatory work receives a great deal of public attention, the
agency's science research typically receives a similar degree of
attention only when the scientific basis for a decision is
questioned. They also pointed out that even internal EPA offices
may be unaware of important benefits from ORD's research program.
More Clearly Indicate Use of Default Options and How They Were Chosen
When EPA needs to rely on assumptions-known as default
options-because it lacks data to fully assess the potential health
risks from exposure to a contaminant, reviewers said EPA should,
among other things, more clearly indicate that it relied on these
defaults and how they were chosen. EPA's defaults, which are based
on general scientific knowledge and policy judgment, are, for the
most part, conservative.15 One example of a commonly used default
option is that laboratory animals are a surrogate for humans in
assessing health risks: An adverse effect in the animals is taken
as evidence of a chemical's potential to harm humans.
Independent reviewers agreed with EPA's use of default options as
a reasonable way to cope with the lack of available data. However,
NAS recommended that EPA be clearer about the scientific and
policy basis for each default option and when it uses a default.
Furthermore, reviewers said that EPA should fully explain under
what conditions they would depart from these default options.
According to NAS, giving greater formality to the criteria for a
departure would result in greater guidance for the public and
lessen the possibility of undocumented departures that would
undercut the scientific credibility of a risk assessment. The
Commission concurred, stressing that the defaults used in risk
assessments and the uncertainty associated with their results
should be clearly identified and justified.
Better Evaluate Available Methods and Models and Incorporate More
Advanced Tools When Appropriate
The overall accuracy of a risk assessment largely depends on the
validity of the various analytic methods and models EPA uses to
assess the toxicity of, and exposure to, a particular contaminant.
However, according to NAS, EPA often does not clearly understand
the extent to which the methods or models it chooses accurately
predict the toxicity or exposure to a contaminant. For example,
because of limited scientific information on how contaminants
actually work in the body to produce adverse effects in humans,
EPA frequently uses a method that relies on studies involving
laboratory animals to understand the toxicity of a substance. The
concentrations of the contaminant introduced into the animals as
part of these studies are higher and administered for shorter
periods of time than humans would normally experience.
Consequently, to determine the expected response in people, EPA
extrapolates the response from laboratory animals to humans. While
such extrapolations are useful to predict potential harmful
effects of a contaminant, different analytic methods may better
predict the effect.
In 1994, NAS recommended, among other things, that EPA evaluate
the accuracy of its methods and models for assessing toxicity and,
when appropriate, incorporate more advanced tools. For example,
models that are based on the underlying mechanisms at the cellular
or molecular level can more accurately estimate the dose of the
contaminant that would have an adverse biological effect on a
specific part of the body. Regarding the risk of cancer from a
particular contaminant, NAS recommended that EPA continue to both
use methods involving animal studies to evaluate the possibility
of adverse effects in humans and explore, when appropriate,
mechanistic models. Furthermore, to better convey the cancer risk
associated with a particular substance, NAS recommended that EPA
develop a classification scheme that provides narrative statements
regarding the hazards posed by carcinogens and a descriptive
evaluation of the strength and nature of the evidence used to
estimate the substance's potential for causing cancer.
Regarding exposure, NAS noted that EPA had traditionally
characterized exposure according to two criteria: exposure of the
total population and exposure of a specified highly exposed
subpopulation. While these two criteria can help assess whether
any particular exposure might occur above a regulatory threshold,
only considering the highly exposed subpopulation is likely to
overestimate the exposures of most of the population.
Consequently, NAS recommended, and the Commission concurred, that
EPA consider the entire range of a population's exposure, rather
than just the exposures of a highly exposed subpopulation. The
Commission also recommended that EPA identify and evaluate highly
exposed populations separately. Moreover, NAS recommended that EPA
use population-activity models-models based on actual human
activity patterns-to better estimate an individual's true exposure
and expand efforts to use personal monitoring data to better
understand actual exposures and variances across the population.
More Explicitly Analyze and Characterize the Sources of Uncertainty in
Risk Assessments
To address the inevitable uncertainties associated with gaps in
scientific knowledge and general unknowns about model and data
accuracy, reviewers recommended that EPA more explicitly analyze
and characterize the sources of uncertainty in its risk
assessments. Numerous gaps in scientific knowledge exist regarding
the health effects of various contaminants, such as the exact
amount of exposure to a particular contaminant that can cause an
adverse effect or the biological effect of a contaminant on the
body. In addition, knowledge is often lacking about which model or
method might be most appropriate to estimate risks to human health
from a particular contaminant. NAS recommended that EPA develop
guidelines for how to analyze and report the different types of
uncertainty, both for the overall assessment and for the different
stages of the risk assessment, such as hazard identification and
exposure assessment. For example, during hazard identification,
uncertainty can be related to the quality, type, and results of
scientific studies; however, during exposure assessment,
uncertainty can be related to model choice or available data.
NAS also recommended in 1994 that EPA conduct uncertainty analysis
of the risk estimate and present the identified uncertainties as
explicitly, accurately, and fully as feasible. Analysis of the
effects of uncertainties can help inform EPA decision makers and
the public about the extent of uncertainty associated with the
risk assessment. The analysis can also show where additional
research might resolve major uncertainties and where it might not.
NAS recommended that the uncertainty analysis be presented both
descriptively (qualitatively) and, where possible, numerically
(quantitatively). For example, some sources of uncertainty, such
as those related to estimating exposures, can be reduced through
the use of more advanced statistical methods. Other types of
uncertainty, such as those associated with extrapolating data from
animal testing to predict the effect on humans, are more difficult
to quantify. The Commission concurred and recommended that risk
characterizations include narrative descriptions of the primary
reasons for uncertainty, as this information is likely to be more
understandable and useful than quantitative estimates or model
results.
Enhance Analysis of Variability in Exposure Levels and Health Risks to
Exposed Individuals
Variability, which refers to the natural diversity in a
population, can be better understood or described, but not
reduced. To address the two main categories of variability-one
related to differences in levels of individual exposures to
contaminants and the other related to differences in individual
reactions to exposure-reviewers recommended that EPA enhance its
analysis of both types and carefully state in each risk assessment
what assumptions it made about what is and is not accounted for.
Specifically, variability related to different exposures depends
on the various concentrations of a contaminant as it disperses in
the environment, different breathing rates, and different food
consumption and personal activity patterns. For example, infants
and children generally consume more fruits, vegetables, and fruit
juices per body weight than adults, and some people, such as
agricultural workers, are more exposed to pesticides through
breathing and skin contact. The Commission recommended that risk
assessments identify groups of people who are likely to have
higher exposures to contaminants and consult these groups in the
early stages of an assessment to obtain information about all
known sources of exposure.
Reviewers also recommended that EPA revise the way it estimates
how long a person is exposed to a contaminant. NAS and the
Commission recommended that EPA move away from estimates of
exposure based on a hypothetical "maximally exposed individual,"
who was assumed to be the person at greatest risk in a worst-case
scenario, because these estimates do not account for a number of
other factors that may affect exposure patterns and rates, such as
the time the person spends indoors or going to work. Furthermore,
estimates based on the hypothetical maximally exposed individual
likely overestimate the exposures of most of the population and
underestimate the exposures of subpopulations, such as
agricultural workers, who may be more highly exposed than the
general population. While EPA's 1992 exposure assessment
guidelines suggest the use of ranges and high-end exposure
estimates chosen from the high end of those ranges, according to
NAS, EPA had not sufficiently documented the reliability of such
estimates when data are limited.
The second type of variability-differences in human
susceptibility-is related to inherent differences among people,
such as age, physiologic characteristics, lifestyle, genetics,
sex, and ethnicity. Reviewers found that EPA's approach for
reducing risks associated with chemical exposures generally did
not include information on differences in individual
susceptibility or encourage gathering evidence to identify these
differences. The reviewers recommended, among other things, that
EPA consider this "interindividual" variability and adopt a
default option for differences in susceptibility among humans. In
addition, NAS recommended that EPA assess risks to infants and
children whenever it appears that their risks might be greater
than those of adults. For example, the developing brains of
infants and young children have an increased susceptibility to
contaminants that harm the nervous system, such as lead. NAS
specifically recommended that EPA sponsor research to examine the
causes and extent of interindividual variability in susceptibility
to cancer and the possible connection between susceptibility and
age, race, ethnicity, and sex. The Commission generally concurred,
adding that risk assessments should also identify especially
susceptible subpopulations, such as people with asthma who may
have an increased responsiveness to allergens and respiratory
irritants. The Commission also stated that, where possible,
available information about the range of the population's
susceptibility should be considered and used in place of default
assumptions.
Better Consider the Human Health Effects of Exposure to Multiple
Contaminants and Routes of Exposure
Reviewers recommended that EPA improve how it considers the
effects of combinations of contaminants (cumulative exposure) and
all possible paths of exposure to a single contaminant (aggregate
exposure). People are typically exposed to a mixture of
contaminants through a variety of pathways, such as contact with
skin or eating food, each of which might be associated with an
increased probability of one or more health effects. However, most
risk assessments address a single contaminant and often focus on a
single pathway of exposure, such as inhalation. As a result, NAS
recommended that EPA should consider all possible exposure
pathways. For example, when assessing risk from mercury, EPA
should consider the risk to residents from inhaling mercury
emitted from a nearby industrial smoke stack, as well as the
possibly greater health risk of consuming mercury that has
accumulated in the tissue of fish that are caught and eaten
locally after mercury from the smoke stack was deposited into
water.16
When assessing the risks of exposure to chemical mixtures, most
risk assessments estimate the risks from individual contaminants,
then calculate the combined risk by simple addition. However, this
method ignores potentially synergistic interactions that may make
the effects more damaging to human health than anticipated or
antagonistic interactions that may make the effects less damaging
than anticipated. Consequently, this method could either under- or
overestimate the total risk. NAS said that simple addition of the
risks from multiple contaminants may be appropriate for
screening-level risk estimates. However if a more refined
quantitative estimate is needed, EPA should consider using
statistical procedures to combine the risks from exposures to
multiple contaminants, which would help produce a more
comprehensive estimate of risk. The Commission concurred but
stressed that combining risks may not always be feasible; the risk
analyses for exposure to each contaminant may not be compatible
because the risk assessments may differ in accuracy. Further, the
Commission recommended that for risk assessments involving
exposures to low concentrations of multiple chemicals, the risks
from each exposure should be added in the absence of information
on exactly how the chemicals affect the body. However, if the
multiple chemicals affect the body in different ways-for example,
if one chemical affected development while another affected the
nervous system-the impact of each chemical on the body should be
considered independently and not added together.
EPA Has Strengthened Many Facets of Its Risk Assessment Process
Since 1994, and Efforts Are Ongoing
EPA has modified its human health risk assessment process since
1994 in several ways. First, the agency enhanced implementation of
its risk assessment process by, for example, issuing guidance and
realigning staff resources. In addition, EPA has taken steps to
identify the scientific data it has on the potential adverse
effects from exposure to various contaminants and has established
collaborative relationships with external researchers to foster
the development of needed additional data. Furthermore, EPA has
begun to improve its use of default options, enhance its modeling
capabilities, and explore new methodologies. EPA has also begun to
characterize uncertainty quantitatively and analyze and
communicate variability more thoroughly. Finally, EPA is more
often considering the combined effects of exposure to multiple
contaminants through multiple pathways. Most of these efforts are
ongoing and can be discussed in terms of the themes presented in
the previous section.
EPA Has Enhanced Implementation of Key Aspects of Its Risk
Assessment Process
EPA has taken a number of steps to improve implementation of its
risk assessment process. Specifically, EPA has developed guidance
and policy documents at the agency and program office levels;
built scientific capacity; modified components of its approach to
risk assessment; and refined its peer review and quality-assurance
practices.
EPA Has Issued Many Agencywide and Program Office-Specific Guidance and
Policy Documents to Improve Risk Assessment Practices
The guidance and policy documents EPA has issued over the past
decade were intended to help staff develop and use risk
assessments and to provide basic information to the public about
EPA's risk assessment methods. While some of these documents have
remained unchanged over time, many have been revised, or will be
revised, as science, knowledge, and analytic methods have
improved. Among these documents are the guidelines issued by EPA's
Risk Assessment Forum, a committee of senior EPA scientists
established to promote agencywide consensus on risk assessment
issues, which cover such topics as neurotoxicity,17 exposure
assessment, and carcinogenic risk assessment. Each of the original
five guideline documents created in 1986 has been updated at least
once, and some are slated to be revised again. For example, the
exposure guidelines were revised in 1992 and revisions are
currently being planned. In addition, the forum has issued two
entirely new sets of guidelines since 1994-Guidelines for
Reproductive Toxicity Risk Assessment and Guidelines for
Neurotoxicity Risk Assessment-and has developed a number of
policies on, among other things, risk characterization, peer
review, and evaluating risk to children. EPA has also issued
interim policy memorandums and position papers on scientific
issues such as genomics and endocrine disruption.
Similarly, much of EPA's agencywide guidance issued since 1994 has
undergone revision and has its origins in earlier policy
documents. For example, EPA's Risk Characterization Handbook,
issued in 2000, has its roots in the 1995 Policy for Risk
Characterization. The handbook stresses that risk characterization
should be transparent, clear, consistent, and reasonable. EPA's
policy on the use of peer review was originally issued in 1994 and
was followed up by issuance of peer review handbooks in 1998 and
again in 2000.18 The peer review policy was recently updated in
January 2006. EPA has also issued technical guidance, such as its
review of the processes to estimate a daily or continuous exposure
to humans that is likely to be without appreciable adverse effects
during a lifetime.
In addition to the EPA-wide guidance, many of the program offices
have also issued guidance documents that support their particular
risk assessment efforts. In general, the office-specific guidance
documents provide risk assessors with analytic tools and exposure
scenarios pertinent to the statutory responsibilities of the
office. For example, the Office of Pollution Prevention and Toxics
created guidance in the form of an analytic tool to screen
chemicals in the absence of data, which frequently occurs because
of the lack of a requirement for industry to develop extensive
data on new chemicals or new uses of existing chemicals. In
addition, the Office of Pesticide Programs has issued
pesticide-specific guidance documents. For example, to help ensure
consistency in pesticide chemical risk assessments, the Office of
Pesticide Programs issued guidance for developing residential
exposure assessments and developed a template for making and
documenting registration eligibility decisions.
EPA Has Built Scientific Capacity through Increased Focus on Scientific
Leadership, Greater Reliance on Research Advisory Groups, and
Development of Future Scientific Talent
EPA enhanced its scientific leadership through the creation of the
position of science advisor in 2002 and the Office of the Science
Advisor in 2003, the increased reliance on research advisory
groups composed of senior EPA scientists and external experts, and
the continuation of its research fellowship programs. The
overarching responsibility of the science advisor is to coordinate
and oversee the scientific activities of the program offices at
EPA to ensure the best use of science. The Office of the Science
Advisor provides further leadership by establishing specific
mechanisms to ensure that scientific results, combined with
technical evaluation and peer review, play a prominent role in
regulatory decisions and that EPA staff interpret and enforce
regulations consistent with the science supporting them.
In addition to enhancing its scientific leadership, EPA has also
increased its reliance on research advisory groups since 1994. The
Science Policy Council and the Risk Assessment Forum play key
roles in advancing the practice of risk assessment at EPA. The
council reviews the adequacy of existing policies, establishes
science policy as needed, and coordinates EPA efforts related to
methods, modeling, risk assessment, and environmental technology.
The Science Policy Council staff facilitate ad hoc work groups,
encourage communication and consensus building within the agency,
and participate in technical work-group activities and
deliberations.
The Risk Assessment Forum is a standing committee of senior EPA
scientists established to promote agencywide consensus on
difficult and controversial risk assessment issues and to ensure
that this consensus is incorporated into guidance. According to an
agency official, the forum is designed as a venue where staff can
meet and discuss common risk assessment issues across program
offices. One of the forum's main contributions to risk assessment
at EPA has been the issuance of a series of risk assessment
guidelines. The forum is currently working on new guidelines, such
as one related to adverse effects on the immune system. When more
specificity is needed on an existing guideline, the forum issues
companion pieces, known as "purple books" because of the color of
their cover, that provide additional or updated information.
The Board of Scientific Counselors (BOSC) provides objective and
independent advice, information, and recommendations about ORD's
research program to ORD's assistant administrator. BOSC is
composed of scientists and engineers from academia, industry, and
environmental organizations who are recognized as experts in their
fields. In 1998, BOSC completed a peer review of ORD's
laboratories and centers.19 BOSC completed a second review of the
laboratories and centers in 2002 and 2003 that identified key
accomplishments of the laboratories and centers, as well as areas
for future improvement. In addition, after EPA's Office of the
Science Advisor issued its 2004 staff paper,20 it asked BOSC to
host a workshop for EPA staff and other interested stakeholders,
such as industry, environmental groups, and researchers, to
provide feedback to refine EPA's current practices and to suggest
alternative approaches for specific aspects of risk assessment.
EPA has also worked to foster scientific excellence and enhance
the skills of its existing workforce through its graduate and
postdoctoral fellowship programs. One such program, the EPA/ORD
Postdoctoral Fellowship Program, began in 1998, and, as of May
2003, 205 individuals had participated in the program. One benefit
of the program is that it helps provide a ready pool of talented
candidates for EPA vacancies. In fact, according to a study of the
program conducted by the National Council for Science and the
Environment, nearly half of the former postdoctoral participants
had taken permanent positions at EPA. The Science to Achieve
Results (STAR) fellowship program is designed to encourage masters
and doctoral students to pursue careers in an environmental field.
The STAR fellowship program has provided new environmental
research in the biological and health sciences, two fields related
to the development of human health risk assessment. EPA has also
partnered with the Association of Schools of Public Health (ASPH)
to offer 1-year placements of graduates in EPA laboratories,
centers, and program offices to work on public health issues. In
the announcement of opportunities for the 2006 ASPH Fellows
Program, a dozen fellowships are being offered in areas related to
human health risk assessment. For example, the National Center for
Environmental Assessment is offering a fellowship to develop
health assessments for various chemicals, and the National Center
for Environmental Research is offering a position for someone
interested in working on developing models or analyzing
uncertainty in risk assessments.
EPA Has Begun to Incorporate Planning Activities and Stakeholder
Input into Its Risk Assessment Approach
To improve planning, which is a part of all risk assessments to
some degree, EPA has issued various guidance documents and held
workshops for staff. One of EPA's earliest related guidance
documents,21 released in 1997, was designed to help risk assessors
and risk managers plan and document the scope of risk assessments
and to consider input from appropriate stakeholders and experts,
especially in those assessments involving the effects of
combinations of contaminants.22 EPA followed up this guidance with
workshops to help staff apply it in risk assessments. In January
2002, EPA issued a handbook to reflect some of the lessons learned
from implementation of the 1997 guidance to make risk assessments
more useful to decision makers and other stakeholders. These
lessons, conveyed through case studies, include that planning can
be particularly valuable when the assessment is complex,
controversial, or precedent setting, and that explaining
uncertainty to stakeholders can help develop trust, credibility,
and support for the decision-making process.
The 2000 Risk Characterization Handbook also strongly advocates
the use of planning and presents a number of topics for both risk
managers and risk assessors to consider, such as identifying the
stakeholders in the process; scope of the effort; relevant
management goals, issues, and policies; available data; and data
needs. For example, during planning, risk assessors, risk
managers, and stakeholders need to identify the key data gaps and
discuss how best to fill them, such as whether to use existing
data or conduct additional short- or long-term tests to evaluate
exposure and effects. The Office of Air and Radiation recognized
the need for planning and developed planning guidance as part of
its Air Toxics Risk Assessment Reference Library, issued in 2004.
EPA acknowledged in its 2004 staff paper that it needs to continue
to stress the importance of concerted and conscious planning with
risk assessors and risk managers before a risk assessment is
started. According to EPA, risk assessors need to outline early in
the development of a risk assessment what will and will not be
addressed and how they will develop the risk assessment.
Stakeholders and the public play a key role in the planning, as
well as at later stages in the development of a risk assessment.
Stakeholders, at various levels and in various forms, can help
ensure better understanding of the risk assessment results and may
promote support for the selected risk reduction strategies.
Program offices involve stakeholders in various ways. For example,
the branch of the Office of Air Quality Planning and Standards
(OAQPS) responsible for setting certain air quality standards for
six principal pollutants solicits input from stakeholders in the
planning phase of its periodic updates to the standards it sets.23
In addition, the public may officially comment on draft air
quality standards once they are publicly released. The Office of
Water pursues stakeholder and public involvement that includes
working with the environmental community, industry, trade
associations, risk assessor organizations, states, and bordering
countries. In addition, the office's periodic reviews of water
quality standards and other nonregulatory actions, such as health
advisories, are all open processes that allow for public input on
various stages of the analysis.
For risk assessments involving the reregistration of pesticides,
the Office of Pesticide Programs (OPP) established a process that
provides several opportunities for public participation.24
Depending on the potential health risks posed by a pesticide
product, the public has anywhere from one to four separate
opportunities to comment. For example, if risk assessors estimate
that the product poses little risk to human health, the public
will have one opportunity to comment before OPP decides whether to
approve the pesticide product. For higher-risk products, the
public will have as many as four opportunities to comment. The
first opportunity to comment occurs after OPP has completed a
preliminary risk assessment. This preliminary assessment contains
all of the elements of a risk assessment and has undergone
internal review, but is not yet finalized. Notice of the
opportunity to comment is distributed to people who have elected
to sign up for such notifications, as well as through a "notice of
availability" published in the Federal Register. The public can
also comment on risk assessments prepared by the Office of
Pesticide Programs through the office's Science Advisory
Panel-which holds periodic public meetings on pesticide-related
risk assessment issues, such as methods to assess skin sensitivity
to exposure to pesticides or models used to estimate dietary
exposures.
EPA has also adopted an iterative approach to many of its risk
assessments. An iterative approach begins with a screening
assessment and progressively grows in depth and scope in relation
to the estimated risks to human health. When a screening
assessment identifies a potential for a nontrivial risk, EPA
decides if pursuing that risk is appropriate based on its current
priorities and available resources. If EPA decides to pursue the
risk, a more detailed, refined risk assessment is performed. The
degree of refinement is based on the type of decision, the
available resources, and the needs of the risk manager. After
refinement of the estimate, EPA reviews it to see if it will be
sufficient to answer the questions posed. Refinements proceed
iteratively until the assessment provides an adequate answer for
the decision maker within the resources available. Both the
revised cancer guidelines and EPA's 1995 Policy for Risk
Characterization support an iterative approach to risk assessment.
Some program offices have also adopted an iterative-or
tiered-approach to risk assessment. For example, the air toxics
program follows a tiered approach, beginning with an analysis that
includes few data and many conservative assumptions. If this
analysis indicates that the risk may be relatively high, assessors
pursue more intensive analysis to determine if the risk is
realistic or an artifact of the lower tier's conservative
assumptions. Despite this move toward greater use of an iterative
approach, EPA acknowledges it could be clearer about when it is
taking such an approach. For example, EPA could be more
transparent about when and why it makes a risk management decision
based on a screening-level assessment rather than a more detailed
assessment.
EPA Has Refined Its Peer Review and Quality-Assurance Practices
In the years since the issuance of Science and Judgment in Risk
Assessment, EPA has made strides to improve its peer review
practices. EPA uses peer review to help ensure the quality of its
risk assessments and keep them as objective and consistent as
possible. EPA's Peer Review Policy states that scientifically and
technically based work products related to agency decisions should
be peer reviewed. In 2000, EPA issued its revised Peer Review
Handbook, an update of the original 1998 edition. In the
intervening 3 years, EPA's Science Advisory Board, EPA's Office of
Inspector General, the National Research Council, and GAO
scrutinized the peer review process. In response, in part, to
recommendations made by many of these groups, EPA issued the
current edition of the handbook. Among other things, it instructs
EPA to balance peer review panels in terms of expertise and biases
to help ensure a reasonable and scientific review.
In addition to issuing the Peer Review Handbook, EPA has
undertaken a number of actions to help ensure the quality of its
data and information. In May 2000, EPA established an agencywide
quality-assurance system and issued the EPA Quality Manual for
Environmental Programs. Key components of this system include
assigning a quality-assurance manager to conduct independent
oversight of data quality, developing a management plan, and
conducting an annual assessment of the quality system. In
addition, the system calls for an assessment of the data EPA used
to support agency decisions to verify that they were of sufficient
quantity and quality for their intended use. In 2002, EPA
developed its information-quality guidelines in response to those
issued by OMB, which stated that agencies must ensure the quality,
objectivity, utility, and integrity of information released to the
public.25 EPA's guidelines outline its policy and procedures to
ensure and maximize the quality of the information it disseminates
and describe the mechanisms by which EPA reviews information prior
to dissemination. EPA issued a complement to these guidelines in
2003 to raise awareness among the public about EPA's ongoing
interest in high-quality data and to serve as an additional
resource for staff as they evaluate the quality and relevance of
information.26
EPA Has Enhanced Its Strategic Planning and Refocused Staff Resources
to Initiate Review of Existing Data and Development of Needed Data
Since 1994, EPA has initiated a number of actions to develop
missing or incomplete scientific data on the potential adverse
effects from exposure to contaminants, including refocusing and
expanding the Office of Research and Development human health risk
assessment program, improving strategic research planning,
increasing collaboration with outside researchers, and enhancing
databases.
EPA Restructured and Augmented Its Research Office
In 1995, EPA restructured ORD to more effectively generate and
gather information needed for the risk assessment process,
combining 12 existing laboratories into 3 national laboratories
and 2 national centers-the National Health and Environmental
Effects Research Laboratory (NHEERL), National Exposure Research
Laboratory (NERL), National Risk Management Research Laboratory
(NRMRL), National Center for Environmental Research (NCER), and
National Center for Environmental Assessment (NCEA).27
Specifically, one laboratory, NHEERL, conducts research on the
effects of various exposure routes and rates, dose levels and
duration, and cumulative doses on health outcomes. Its main
research areas, all of which help improve risk assessments, target
(1) the use of mechanistic information-how a substance causes an
effect at a biological level-to reduce uncertainties in risk
assessment; (2) the cumulative risks posed by exposure to
contaminant mixtures; (3) the health risks to particularly
susceptible populations; and (4) the evaluation of public health
outcomes to determine the effectiveness of actions designed to
reduce health risks. This research is developed in coordination
with the program offices to target issues in risk assessment for
air pollutants, safe foods and pesticides, water, and contaminated
lands.
A second laboratory, NERL, researches and develops improved
methods and models to assess and predict human exposures to
harmful contaminants in air, water, soil, and food. For example,
it developed methods to better characterize pollution sources, and
models to quantify the effects on exposure from various individual
behaviors. NERL works closely with the program offices to set
research priorities and help ensure that its results are useful to
the program offices. For example, NERL coordinated with OAQPS to
format data from several studies on airborne particulate matter in
a comparable manner that allowed NERL and OAQPS staff to develop
more powerful analytic results than would have been possible from
the individual studies alone. NERL also developed tools to enhance
the exposure assessment portion of the risk assessment, such as a
database of human activities by age, sex, and location, to better
characterize exposure risk based on personal activities. The third
laboratory, NRMRL, focuses its research more on ways to minimize
exposure to contaminants that cause health risks than on ways to
improve the preparation of risk assessments. For example, NRMRL
researches sources of chemicals that disrupt the endocrine system
and strategies to minimize exposure to such chemicals.
The two centers, NCER and NCEA, serve as focal points for external
researchers and risk assessors, respectively. NCER funds
innovative environmental research by academic scientists to reduce
uncertainty in risk assessment. Specifically, NCER has sponsored
research to develop data for use in models, thereby helping to
reduce the model's uncertainties. NCER also coordinates with EPA's
laboratories and program offices to develop its research topics.
The Science to Achieve Results (STAR) program, one of NCER's grant
programs, funds competitive research proposals and graduate
fellowships in environmental science and engineering fields to
complement ORD's research in its strategic and research plans,
such as the health effects of contaminants on airborne particulate
matter and in drinking water, and, more generally, on children's
health. For example, the STAR program has funded research to
generate data on human exposures that will improve risk estimates.
NCER has also established academic research centers in such areas
as children's health. NCEA, EPA's national resource center for
human health risk assessment, supports EPA's work in three main
ways. First, NCEA conducts risk assessments of national
significance-for example, assessments of dioxin and diesel
emissions-and prepares the air quality criteria documents that
reflect the state of the science for six principal air pollutants.
Second, it develops scientifically sound, defensible risk
assessment methods to improve the use of science in risk
assessment, such as software to estimate a benchmark dose-the dose
that produces change in the risk of an adverse effect. Third, it
provides guidance and support to risk assessors and risk managers
through such means as its management of the Integrated Risk
Information Systems (IRIS), a database of the potential human
health effects of exposure to various chemicals in the
environment.
In addition to the six labs and centers, EPA in 2005 established
the National Center for Computational Toxicology (NCCT) to
coordinate and implement EPA's research on computational
toxicology, a cutting-edge field that uses mathematical models to
predict adverse effects and to better understand the mechanisms
through which a given contaminant causes harm. Given advances in
such newly emerging disciplines as the study of genes and their
functions, computational toxicology offers the potential for
scientists to develop a more detailed understanding of the risks
posed by a much larger number of chemicals than is currently
possible. NCCT research is designed to develop tools to conduct
quantitative risk assessments more rapidly and to improve the
identification of chemicals that may pose substantial health
risks. NCCT staff have begun to collaborate with other ORD
laboratories and centers to effectively target their research
efforts.
EPA Has Enhanced Strategic Research Planning
Since 1994, EPA has undertaken a number of strategic activities to
more closely link the data needs of program offices to research
agendas of EPA and ORD.28 In connection with the goals presented
in EPA's and ORD's strategic plans, ORD defined 16 high-priority
research areas such as human health, endocrine disruption,
airborne particulate matter and other air pollutants, and safe
pesticides. Research strategies for each area are either in place
or under development. For example, EPA's Human Health Research
Strategy identifies four broad, overarching research areas to
guide ORD's human health research over the next 5 to 10 years.29
To carry out its 16 research strategies, ORD began in 2000 to
develop a multiyear implementation plan for each. The first plans
emerged in 2001, with most finalized by 2003 following widespread
participation and input from many stakeholders as well as review
by senior ORD managers. ORD invites input on the plans from ORD
and program office staff, federal research partners, and outside
peer groups, such as EPA's Science Advisory Board, the Board of
Scientific Counselors, and the National Research Council. These
plans establish the short- and long-term goals and timelines
required for ORD's laboratories or centers to implement each of
the strategies. The National Program Director, a newly established
position for each major ORD research area including human health,
helps ensure that ORD's time and staff resources are used
strategically and that the overall planning effort links to the
needs of the program offices.
Program offices use various planning and review approaches to
determine data needs. For example, the Office of Air Quality
Planning and Standards develops a research-needs paper at the
conclusion of each periodic review of the air quality standards it
establishes to inform the research agenda for the next review,
which occurs about every 5 years. Each paper helps ensure that
research in key areas will be available for the next review. In
contrast, the data needs of the Office of Pesticide Programs are
defined by law.30 Applicants who wish to register a pesticide
product must submit the data defined in the statute and
regulations, and OPP staff determine whether the data are of
sufficient quality and quantity to assess the risk from the
pesticide product.31 This list of required data is currently
undergoing revision and additional data requirements may be added.
EPA Program Offices Have Established Collaborative
Relationships with External Researchers
Since 1994, EPA has strengthened and formalized collaboration with
a range of other federal researchers to better leverage its
limited research dollars and foster the development of data to
improve human health risk assessments. Specifically, EPA has
developed relationships with agencies such as the National
Institute for Environmental Health Sciences (NIEHS) and the Agency
for Toxic Substances and Disease Registry (ATSDR). For example, in
1998, EPA established a cooperative agreement with NIEHS to
develop a body of research on the relationship between exposures
and children's health. This collaboration jointly funded
Children's Environmental Health Research Centers at seven U.S.
universities and one medical center to research children's asthma
and other respiratory diseases, as well as ways to reduce farm
children's exposure to pesticides. In addition, EPA works closely
with ATSDR to help fill research gaps and develop
chemical-specific toxicological assessments used in risk
assessments. In 2004, EPA and ATSDR entered into a formal
agreement to ensure close collaboration to avoid duplicating
efforts to fill data gaps. Under the agreement, the two agencies
formed a work group to coordinate their efforts to develop
toxicological assessments for ATSDR's work at specific highly
contaminated locations and for EPA's Integrated Risk Information
System (IRIS) database.
EPA, NIEHS, and ATSDR also jointly develop and annually review a
list of approximately 275 hazardous substances commonly found at
the nation's highly contaminated sites and for which ATSDR will
prepare toxicological assessments. At each annual review, agency
staff may add chemicals to the list and identify priority research
to fill gaps in knowledge. Of these 275 chemicals, approximately
150 have been identified by EPA as high-priority needs. Each
toxicological assessment contains almost everything that is known
about the chemical, including its potential to harm human health
or the environment. A key difference between these toxicological
assessments and the ones in EPA's IRIS database is that ATSDR
includes chronic cancer and noncancer effects, as well as acute
effects, while IRIS generally includes only chronic cancer and
noncancer effects.
In addition, EPA has begun to establish collaborative
relationships with scientific and industry-related researchers.
For example, EPA has cooperative agreements with the International
Life Sciences Institute's Risk Science Institute (ILSI-RSI), an
organization that researches critical scientific issues in risk
assessment, such as the development of risk assessment
methodologies.32 These cooperative agreements were specifically
designed to engage the scientific community and bring together
scientists from different affiliations (including academia, other
parts of government, and the private sector including industry) to
address risk assessment issues. Under one agreement, ILSI-RSI is
to research risk assessment approaches for cumulative and
aggregate exposures. In addition, EPA has used research provided
by CIIT Centers for Health Research, a chemical research
laboratory funded by EPA, industry, and other federal agencies, to
provide information for its formaldehyde IRIS assessment.
Furthermore, EPA and industry jointly fund the Health Effects
Institute (HEI)-an organization that researches the health effects
of various air pollutants, including airborne particulate matter
and ozone. HEI has provided data for risk assessments and convened
panels of experts to review and issue reports related to risk
assessment, recently on diesel exhaust. In its 2004 staff paper,
EPA noted that it needs to continue to encourage development of
the specific data necessary to more accurately assess potential
risks, particularly with researchers responsible for generating
appropriate data, such as those seeking approval to manufacture or
use a chemical.
The Office of Pollution Prevention and Toxics has two programs to
work with industry to develop data on contaminants that can be
used to better understand risks. The first is the High Production
Volume (HPV) Challenge Program. This program was officially
launched in late 1998 to ensure that a baseline set of data would
be made available to the public on approximately 2,800 chemicals
that are manufactured or imported in amounts greater than 1
million pounds per year. Diverse stakeholders, including the
American Chemistry Council, Environmental Defense, and the
American Petroleum Institute participate in the program. The HPV
Challenge Program provides an opportunity for all stakeholders,
including the public, to comment on the tests and data summaries
from the chemical sponsors-companies and consortia that
volunteered to make publicly available screening-level data that
allow EPA, industry, and other stakeholders to more effectively
gauge the potential hazards of HPV chemicals. All comments are
publicly available on the World Wide Web. As of January 2006, EPA
had commitments from industry sponsors to provide data for 2,247
of the chemicals. The second program, the Voluntary Children's
Chemical Evaluation Program, is designed to provide data that will
allow the public to better understand the potential health risks
to children associated with certain chemical exposures. EPA asked
companies that manufacture or import 23 chemicals that have been
found in human tissues in various biological monitoring programs
to voluntarily sponsor the evaluation of specific chemicals in a
pilot program. Thirty-five companies and 10 consortia volunteered
to sponsor 20 chemicals. This program was developed only after
considering comments and concerns from stakeholders. Of the 23
chemicals chosen for this pilot, data gathering has been completed
for 9 and is under way for another 11. The remaining 3 chemicals
in the pilot program have no sponsors.
EPA Maintains Databases of Information Related to Risk Assessment
EPA maintains both an agencywide and several program-specific
databases of information to help in the development of risk
assessments. The primary database used by risk assessors in the
program offices is the Integrated Risk Information System (IRIS),
an electronic database of descriptive and quantitative information
on human health effects that may result from exposure to various
chemicals in the environment. Program office staff combine the
IRIS data with exposure information they develop to characterize
the health risks of a given chemical in a given situation.
Since 1994, EPA has changed the IRIS assessment process in several
ways. For example, each IRIS file now contains a discussion of the
key studies, as well as a description of the decisions and default
assumptions used in the assessment. EPA has also expanded the
review that IRIS assessments undergo. For example, internal peer
reviewers, including EPA senior health scientists representing
program offices and regions, review the IRIS summary and
accompanying detailed technical information. After this review,
ORD releases the document for external peer review. EPA makes
draft assessments available to the public at this time and,
following peer review, the IRIS assessment discusses the key
issues reviewers raised and EPA's response. In addition, EPA has
added a tracking system that allows IRIS users to readily
determine where an individual assessment is in its development.
In September 2003, EPA completed a congressionally requested
review to assess the need to update information in IRIS, based on
concerns that EPA and state regulators rely on potentially
outdated scientific information. Input from EPA program and
regional offices, the public, and other stakeholders indicated
that EPA should, among other things, increase the number of new or
updated assessments completed each year to 50. To date, EPA has
fallen considerably short of this goal. According to a program
official, EPA completed 8 IRIS assessments in 2005, plans to
complete 16 in 2006, and has approximately 75 assessments under
way. EPA officials said a number of factors, such as the
complexity of the assessment process, resource limitations, and
extensive peer review, had limited EPA's ability to complete more
assessments last year. EPA has increased the number of staff
working on IRIS assessments from 6 to 23 and may ultimately
increase the number to 29. The review also indicated that EPA
needs to assign staff to develop health assessments for IRIS, and
provide funding for extramural research and contracts to develop
IRIS files and subject them to external peer review.
EPA also changed how it sets priorities for which chemicals need
new or updated IRIS assessments.33 Annually, EPA asks its program
offices, regions, and the public to identify contaminants for
which it should develop or revise IRIS assessments. EPA publishes
the list in the Federal Register and requests the public and
scientific community to submit any relevant data on substances
undergoing review. EPA is currently reviewing ways to increase
coordination with other governmental agencies that develop
chemical assessments, outreach to stakeholders earlier in the
development of IRIS assessments, and consultation with independent
external reviewers. In 2004, the IRIS program also initiated a
review of available scientific literature for the 460 chemicals in
the database that are not under active reassessment to determine
whether a reassessment based on new literature could significantly
change existing toxicity information. For 63 percent of the
chemicals reviewed, no major new health effects studies were
found. Such literature reviews will be conducted annually and the
findings noted in the IRIS database.
In addition, some program offices maintain databases to enhance
their risk assessments. For example, the Office of Air Quality
Planning and Standards (OAQPS) maintains a database of
dose-response values developed by various sources, including IRIS,
ATSDR, and the California Environmental Protection Agency, as an
aide for its risk assessors. OAQPS staff update this database as
better data become available. As part of its National Air Toxics
Assessment-an ongoing comprehensive evaluation of hazardous air
pollutants in the United States-EPA assessed 32 air pollutants
plus particulate matter in diesel exhaust in 1996. The national
assessment is designed to identify air pollutants with the
greatest potential to harm human health, and the results will help
set priorities for collecting additional data. As part of its
assessment, EPA compiled a national emissions inventory of
hazardous air pollutants from outdoor sources, estimated
population exposures to the pollutants, and characterized the
potential cancer and noncancer health risks from breathing the
pollutants.
ORD also maintains personal monitoring data on the chemicals in
the air, foods and beverages, water, and dust in an individual's
personal indoor and outdoor environments. For example, in its
National Human Exposure Assessment Survey (NHEXAS) program, which
was completed in 1998, ORD collected human exposure data from
hundreds of subjects from several areas of the country. NHEXAS
provided data on background levels of total exposure to
environmental contaminants that can be used as a baseline in
exposure and risk assessments to estimate whether specific
populations are exposed to increased levels of environmental
contaminants.
EPA Has Improved Its Choice and Communication of Default Options
EPA has explicitly stated the need for risk assessors to identify
when they relied on a default option, why they chose it, and when
they departed from using a standard default option, but the agency
acknowledges more could be done. To carry out its mission to
protect human health, EPA's risk assessment procedures, including
its default options, are protective of human health. In three
recent guidance documents-the 2004 staff paper, the 2005 cancer
guidelines, and the risk characterization handbook-EPA advocated
more transparency in the choice of default options. These
documents summarize a significant change in EPA's approach.
Specifically, EPA first critically examines all relevant and
available data to assess health risks, then uses the default
options only in the absence of adequate contaminant-specific data.
EPA also states in its staff paper that it bases its default
assumptions on peer reviewed published studies, empirical
observations, extrapolation from related observations, and
scientific theory. Moreover, the cancer guidelines include an
appendix that defines the basis for each of the default options
that may be used in a cancer risk assessment. The Risk
Characterization Handbook notes that risk assessments should
describe the full range of default options that were used,
including ones to address uncertainty. Moreover, the handbook
states that when defaults are used, the risk assessment should
reference the relevant EPA guidance that explains them.
EPA program offices also advocate greater transparency when using
default options. Specifically, the majority of IRIS assessments
completed since 1997 describe the defaults used in the analysis
and any departures from those defaults. The Office of Air Quality
Planning and Standard's Air Toxics Risk Assessment Reference
Library contains a number of references to defaults that should be
used in the course of preparing a human health risk assessment.34
For example, to estimate an individual's exposure to an air
pollutant, the guidance presents the default option to use for
exposure in a screening-level assessment, namely that the
individual remains at a single location and continuously breathes
polluted air.
Despite the increased focus on more transparency in the use of
defaults, EPA acknowledges it could more consistently describe how
the default was developed and explain why it is a reasonable
assumption. In its staff paper, EPA acknowledges it needs to
ensure that the defaults are supported by the best available data
and should look for opportunities to increase certainty and
confidence in the defaults and extrapolations used. EPA also
acknowledges it may need to re-examine older risk assessments that
relied on defaults that can now be replaced with relevant data. To
a large degree, the use of defaults is intertwined with EPA's
ability to get the data it needs. As was discussed previously, EPA
has targeted research, both within EPA and through its grant
programs, to understand variability and uncertainty in the data
derived from studies of laboratory animals, and this research may
further reduce EPA's need to rely on default options.
EPA Has Taken Steps to Enhance Its Modeling Capabilities and Embrace
New Methodologies
Since 1994, EPA has taken several steps to enhance its modeling
capabilities and embrace new methodologies for risk assessment
through improved guidance and workshops. To help improve its
models, EPA's Agency Task Force on Environmental Regulatory
Modeling published a report that concluded a need existed for,
among other things, training and technical support, agency
guidance on external peer review of environmental regulatory
modeling, and creation of a Committee on Regulatory Environmental
Modeling.35 Also in 1994, EPA's Risk Assessment Forum developed a
draft protocol to evaluate models for exposure assessments.36 In
1997, ORD and program offices conducted an agencywide conference,
called the Models 2000 Workshop, to facilitate adherence to
existing guidance on modeling, to define and implement
improvements in how the agency developed and used models, and to
recommend an implementation plan for improving modeling within the
agency.
EPA followed up these activities in 2000 by creating the Committee
on Regulatory Environmental Modeling (CREM) to promote consistency
and consensus within the agency on modeling issues (including
modeling guidance, development, and application) and to enhance
internal and external communications on modeling activities. CREM
supports and enhances the existing modeling activities in the
program offices and provides EPA with tools to support
environmental decision making. CREM also provides the public and
EPA staff with a central point of inquiry about EPA's use of
models. In 2000, CREM launched agencywide activities designed to
enhance the development, use, and selection of regulatory
environmental models at EPA. One such activity-a workshop to
facilitate discussion of good modeling practices-resulted in the
development of modeling guidance.
In 2003, CREM developed guidance and created a database-called the
Models Knowledge Base-of the models most frequently used in EPA.37
The guidance recommends best practices to help determine when a
model, despite its uncertainties, can be appropriately used to
inform a decision. Specifically, it recommends that model
developers and users subject their model to credible, objective
peer review, assess the quality of the data they use as inputs,
and perform sensitivity and uncertainty analysis to determine
which of the model inputs has the greatest impact on the modeled
results.
EPA has also incorporated efforts to improve models in its
research strategies and implementation plans. For example, in its
plan for research on hazardous air pollutants, EPA established a
long-term goal to reduce uncertainties in risk assessments through
methods, data, and models of acute and chronic exposures and
exposures through multiple pathways at both the national and
regional levels. In addition, one of ORD's laboratories
established an exposure modeling research branch and develops
population exposure models, such as the Stochastic Human Exposure
and Dose Simulation model for inhalation and exposures of general
and sensitive subpopulations through multiple pathways. EPA has
also begun to use geographic information systems (GIS) to present
risk information spatially. For example, a GIS system is being
developed that maps all of the drinking water intakes in the
United States and their associated watersheds, so that the agency
can better assess risks to drinking water supplies stemming from
activities in the related watershed. For risk assessments of
hazardous air pollutants, GIS can display and analyze data during
planning, scoping, and problem formulation, during the exposure
assessment, and during the characterization of risks. GIS can also
help communicate information to risk managers and other
stakeholders.
In addition to models, EPA is beginning to embrace such new risk
assessment methodologies as probabilistic risk assessment and mode
of action analysis. Probabilistic risk assessment characterizes
the variability or uncertainty in risk estimates as the range or
distribution of the number of times each possible outcome will
occur. In probabilistic risk assessment, one or more variables in
the risk equation, such as the exposure rate, is defined as a
distribution rather than as a single number. A primary advantage
of probabilistic risk assessment is that it provides a
quantitative description of the degree of variability or
uncertainty. EPA's 1997 policy states that probabilistic
techniques, such as Monte Carlo analysis, can be viable
statistical tools to analyze variability in risk assessments, when
they are based on adequate supporting data and credible
assumptions.38 The guidance presents a general framework and broad
set of principles to ensure the use of good scientific practices
when conducting probabilistic analyses of variability and
uncertainty. EPA currently uses a number of models that include
probabilistic analyses and is developing a new modeling framework,
known as the Multimedia Integrated Modeling System, that will
further enhance the agency's ability to probabilistically model
uncertainty.
EPA's recently revised cancer guidelines advocate the use of a
mode of action analysis-based on the sequence of biological events
that must occur to produce a harmful effect-to improve the
accuracy of risk assessments. As a general rule, EPA assumes that
toxic responses observed in laboratory animals indicate that the
same responses are likely to occur in people even though
differences in such areas as metabolic rates can result in
different sensitivities between laboratory animals and humans.
Mode of action analysis will more clearly indicate whether a
difference exists between animals and humans in their response to
contaminants. In addition, the guidelines present a new cancer
characterization system consisting of five summary descriptors, to
be used in conjunction with narrative, to describe the extent to
which available data support the conclusion that a contaminant
causes cancer in humans and to justify the summary descriptor
selected. For noncancer risk assessments, EPA has used, and
continues to refine, the benchmark dose methodology, which
identifies the dose or concentration of a contaminant that
slightly increases the likelihood of an adverse effect.
EPA Is Introducing More Quantitative Characterization of
Uncertainty into Its Risk Assessments
Uncertainty is inherent in all phases of risk assessment, from
hazard identification through risk characterization. Over the
years, EPA has relied more on qualitative, or descriptive,
characterizations of uncertainty and less on quantitative, or
numeric, characterizations. EPA's practice now is to use a tiered
approach to analyze uncertainty. That is, EPA starts with a simple
description of uncertainty and sequentially employs more
sophisticated quantitative analysis, such as sensitivity analysis,
provided the additional analysis reduces the uncertainty. To
characterize risk quantitatively, EPA has typically used
approaches that produce a single number to characterize the risk
in terms of the level of a contaminant that does not cause harm,
as opposed to presenting a range of possible values.
Although EPA is beginning to use probabilistic approaches in
exposure assessments, and has done so for six principal air
pollutants, it does not typically do so to analyze uncertainty in
its dose-response analyses, though its Science Advisory Board
encouraged development of such approaches in 2000. EPA
acknowledged in its 2004 staff paper that probabilistic risk
assessment could be used more frequently and could provide useful
information beyond screening-level assessments. However, the staff
paper said probabilistic analysis may not be appropriate in all
situations and the accuracy of the analysis will depend largely on
the availability and quality of the data used in the analysis.
Another quantitative approach to uncertainty is the use of
uncertainty factors to account for such unknowns as variation in
sensitivity among members of the human population or the
appropriateness of extrapolating animal data to humans. EPA
routinely uses uncertainty factors when it estimates the daily
exposure to the human population that is likely to be without
appreciable risk of adverse effects during a lifetime. This daily
exposure estimate is called a reference dose for contaminants that
are consumed and a reference concentration for inhaled
contaminants. EPA states in its 2004 staff paper that it applies
uncertainty factors in health assessments based on available data
and the scientific judgment of EPA risk assessors and peer
reviewers. According to EPA, most IRIS toxicological assessments,
which contain a chemical's reference dose and reference
concentration, provide justifications for the uncertainty factors
applied to a particular chemical. Moreover, the factors undergo
rigorous internal, and independent, external scientific peer
review before being used in IRIS assessments.
EPA has issued a number of documents that delineate the need to
clearly and consistently characterize uncertainty in risk
assessments. In 1995, EPA issued a risk characterization policy
that stated that the risk assessment should fully, openly, and
clearly characterize risks, and should disclose the scientific
analyses, uncertainties, assumptions, and policies underlying the
decisions. This policy was followed in December 2000 by the Risk
Characterization Handbook, which includes guidance on how to
address, among other things, uncertainty in risk assessment and
describes the need and methods to present the sources and
magnitude of uncertainty to the risk manager. More recently, the
2005 revised cancer guidelines discuss each of the major
uncertainties, such as model uncertainty or uncertainty related to
human variation, and stress that assessments should discuss the
significant uncertainties encountered in the analysis. For
example, the guidance calls for the assessments of hazard,
dose-response, and exposure to have accompanying technical
characterizations covering the strengths and limitations of data
and a discussion of uncertainty.
EPA Better Analyzes and Communicates Variability in both Exposure and
Susceptibility
EPA has made progress in describing variability due to differences
in both the exposure individuals receive and the susceptibility of
exposed individuals to adverse effects. A key document EPA risk
assessors use to account for variation in exposure is its Exposure
Factors Handbook, originally issued in 1989 and revised in 1997.
The handbook summarizes data on human behaviors and
characteristics that influence exposure to environmental
contaminants and recommends values to account for those factors in
assessing risk. Specifically, the handbook contains a series of
over 150 data tables that provide information on how much time
individuals spend at various activities and in various
environments. Assessors can use these data to develop exposure
duration estimates for exposure scenarios. For example, the tables
contain statistics-broken down by age, gender, race, education,
and some medical conditions, such as asthma or emphysema-for time
spent in various outdoor locations. The handbook also provides
general guidance to risk assessors on the types of variability
relevant to a risk assessment and ways variability can be analyzed
and addressed.
All program offices address exposure variability in their risk
assessments, although they do so in different ways. For example,
risk assessors in the Office of Air Quality Planning and Standards
who set certain air quality standards for six principal pollutants
said they consider individual activity patterns for sensitive
populations like children or asthmatics in exposure modeling by
including a distribution of breathing rates to reflect variability
inherent in the population. Furthermore, by modeling to protect
the most sensitive or at-risks groups, they are assured of
protecting the rest of the population. Variability in exposure to
the six principal pollutants is generally described qualitatively
in scientific summaries for each pollutant. The Office of Water
includes an analysis of risks to various subpopulations and a
narrative discussion of the strengths and weaknesses of the
studies it used to estimate exposure, but generally does not
include a quantitative analysis. The Office of Pesticide Programs
considers 24 different population subgroups in its exposure
estimates, including differences in age, gender, ethnicity, and
geographic dispersion. When data allow, the Office of Pesticide
Programs develops a distribution of exposures and risks for its
more refined risk assessments.
To further its understanding of variability in exposure, EPA has
undertaken a number of research projects. For example, one of
ORD's laboratories conducted the National Human Activity Pattern
Survey to provide detailed human exposure information for specific
populations and allow EPA to better understand actual human
exposure to pollutants in real-world situations. The survey
results are stored in the Consolidated Human Activity Database to
help risk assessors estimate the time that exposed people spend in
various environments and their inhalation, ingestion, and dermal
absorption rates while in those environments. This laboratory also
conducts research to define, quantify, and reduce the uncertainty
associated with the exposure and risk assessments, to develop
improved methods to more accurately measure exposure and dose, and
to develop technical information and quantitative tools to predict
the nature and magnitude of human exposures to environmental
contaminants. A recent EPA study was designed to identify
chemicals commonly used in homes or day care centers, and whether
children in these environments encountered the chemicals in the
course of their daily activities.39 The research sought to
identify the major routes (i.e., breathing and ingestion) and
sources (i.e., dust, food, air, soil, and water) through which
children come into contact with chemicals.
Variability also exists with regard to susceptibility to adverse
affects because of inherent differences among humans. EPA most
recently addressed variability in susceptibility in the 2005
revision of its cancer guidelines, which describe the importance
of separate risk assessments for all potentially sensitive life
stages, including adults and children. The supplementary cancer
guidelines for children address issues pertaining to cancer risks
associated with early-life exposures. Legislation can also require
EPA to consider potentially susceptible populations and life
stages. For example, the Safe Drinking Water Act Amendments
mandate that EPA consider risks to groups within the general
population that are at greater risk of adverse health effects,
including children, the elderly, and people with serious
illnesses. In addition, the Food Quality Protection Act contains
special provisions for the consideration of risks to children from
pesticides. In 1995, EPA's Science Policy Council called for EPA
to consider the risks to infants and children consistently and
explicitly as part of its risk assessments. In 1997, the White
House issued an executive order that required EPA and other
federal agencies to identify and assess environmental health and
safety risks that may disproportionately affect children and to
ensure that policies, programs, activities, and standards address
such disproportionate risks.40
In its 2004 staff paper, EPA acknowledges that characterizing
variability for susceptible populations and life stages is an area
where it could improve, although the absence of data limits its
efforts. In 2002, EPA pointed out that data are limited to
identify susceptible populations and life stages for many
contaminants.41 In these situations, EPA typically relies on
default options, such as the use of uncertainty factors to account
for variations in susceptibility. Many of the exposure assumptions
and default values used to assess plausible current and future
exposure scenarios can be found in EPA's 1997 Exposure Factors
Handbook, and recent updates to the handbook are available online.
Another way EPA addresses variability is through research. One of
ORD's four strategic research directions in its Human Health
Research Strategy is designed to improve the understanding of why
some people and groups are more susceptible and highly exposed
than others. According to this strategy, ORD's research on
subpopulations will focus on three factors-life stage, genetic
factors, and pre-existing diseases-that have been identified by a
program office and the scientific community as having a high
priority for risk assessment. In 2000, ORD released its Strategy
for Research on Environmental Risks to Children to strengthen the
scientific foundation of risk assessment and management decisions
that affect children and guide EPA's research needs and priorities
over the following 5 to 10 years. Approximately 75 percent of the
funding for this strategy will be dedicated to research grants
under the STAR program, such as those designed to evaluate
children's exposure to pesticides.
EPA Has Begun to Consider the Combined Effects of Exposure to
Multiple Contaminants through Multiple Pathways
To help risk assessors analyze the health effects of exposure to
multiple contaminants (cumulative exposure) and through multiple
routes (aggregate exposure), EPA has issued guidance, developed
methods and models, and supported research. In 1997, EPA's Science
Policy Council issued guidance on cumulative risk assessment. This
guidance directs each office to consider cumulative risk in
planning major risk assessments and, where relevant data are
available, to broaden the scope of the assessment to integrate
multiple sources, effects, pathways, stressors, and populations
for cumulative risk analyses. The guidance also highlights the
need to ensure that the public and other stakeholders have an
opportunity to help define the way EPA assesses an environmental
or public health problem and calls for ongoing communication and
coordination among EPA's risk assessors, risk managers,
economists, engineers, and other technical experts.
In 2000, EPA updated its 1986 guidance on chemical mixtures, to
generate a consistent agencywide approach to assess health risks
from exposures to multiple chemicals.42 The guidance is organized
according to the type of data available to risk assessors, ranging
from data-rich to data-poor situations, to help risk assessors
select an appropriate methodology. For example, if data are of
poor quality or quantitative data are very limited on chemical
mixtures, the risk assessor may choose to perform a qualitative
analysis of the potential human health impacts from exposure to
the mixture. The guidance also contains procedures to develop
toxicity equivalency factors, based on the toxicity of components
of the mixture, to assess the risk from mixtures in the absence of
data on the specific mixture.
In 2003, EPA's Risk Assessment Forum developed a simple, flexible
framework to help risk assessors consistently conduct and evaluate
cumulative risk assessments.43 The framework is conceptually
similar to the one used in human health assessments of a single
contaminant in that it follows a three-stage approach of (1)
planning, scoping, and problem formulation; (2) analysis; and (3)
risk characterization. In addition, the document also highlights
needed areas of research and methods development that may be
important to the evaluation of cumulative risks, such as
understanding how the sequence and timing of exposure may
influence the ultimate risk for effects.
EPA risk assessors use a number of models and methodologies to
analyze multiple routes of exposure. For example, the branch of
the Office of Air Quality Planning and Standards that regulates
hazardous air pollutants employs the Multiple Pathways of Exposure
model to assess and predict the movement and behavior of chemicals
in the environment. For example, the Multiple Pathways of Exposure
model includes procedures to estimate human exposures and health
risks that result from the transfer of pollutants from the air to
soil and surface water bodies and the subsequent uptake of the
pollutant by plants, animals, and humans. The model specifically
addresses exposures from breathing; consuming food, water, and
soil; and contact with skin. More recently, EPA developed the
Total Risk Integrated Methodology (TRIM) and created the TRIM
Fate, Transport, and Ecological Exposure model that describes the
movement of air pollutants emitted from any type of stationary
source as well as their transformation over time in water, air and
soil.
Some program offices have also taken steps to explicitly consider
the risks associated with more than one route of exposure or more
than one chemical. Specifically, the Office of Pesticide Programs
issued guidance in 2001 and 2002 in response to statutory
requirements to assess the risk of aggregate exposure-exposure to
a single chemical by multiple pathways and routes.44 The first set
of guidance focuses on how to assess aggregate risk in those cases
where more extensive data and more sophisticated exposure
assessment methods and tools are available; this guidance also
emphasizes, when data are available, the use of distributional
data-aggregate exposures of many individuals in the population of
interest-for all pathways of exposure.45 This approach allows the
risk assessor to more fully evaluate exposure and resulting risk
across the entire population, rather than the exposure of a single
highly exposed individual. The second set of guidance describes a
framework to assess potential human health risks from all pathways
of exposure to multiple pesticides that share a common mechanism
of toxicity-that is, the pesticides produce a similar toxic effect
on the same organ or tissue.46
In addition, the branch of the Office of Air Quality Planning and
Standards that regulates hazardous air pollutants developed the
Integrated Urban Air Toxics Strategy, which is used to consider
cumulative risks presented by exposures to hazardous air
pollutants emitted from various sources. Staff can assess risk at
both a national and an urban or a neighborhood scale. Furthermore,
the hazardous air pollutant office developed guidance on
multipathway risk assessments that are particularly important for
hazardous air pollutants, such as mercury and dioxins, because
human exposure occurs both from breathing air containing the
toxins and from consuming plants, water, and soil where the
pollutants were deposited. Moreover, these pollutants persist in
the environment for long periods of time and may also accumulate
in the tissues of commonly consumed plants and animals to levels
that are harmful to humans.47 The guidance, maintained in the Air
Toxics Risk Assessment Reference Library, describes how to plan,
scope, and formulate the problem, conduct the analysis, and
characterize the risk for such cases.
The extent to which program offices assess the effects of
cumulative and aggregate exposures is related to the regulatory
responsibilities of each office and by the availability of data.
For example, the hazardous air pollutant office routinely analyzes
a mix of chemicals from various emitting sources, such as
petroleum refineries, to regulate hazardous air pollutants.48
Similarly, as mentioned above, the Office of Pesticide Programs is
required to consider exposure to pesticides from various pathways,
such as food, drinking water, and residential uses, and various
routes, such as eating, breathing, and contact with skin. In
contrast, the Toxic Substances Control Act does not require the
Office of Pollution Prevention and Toxics to assess the risks of a
new chemical that may occur through its interaction with other
chemicals. The office also assesses the risks of existing
chemicals but cannot conduct cumulative risk assessment for
classes of chemical that share a common mode of action because no
data exist. Program managers hope such data will become available
in the future.
In its 2004 staff paper, EPA commented that while it has increased
its emphasis on evaluating cumulative risks, it needs to expand on
approaches to do so, and it needs to produce a rigorous scientific
base to support such evaluations. To that end, one of ORD's four
strategic research directions in its Human Health Research
Strategy is to improve assessments of aggregate and cumulative
risks. Specific research objectives are to develop exposure models
and methods, provide a scientific basis to predict interactive
effects of contaminants in mixtures, and determine the most
appropriate approaches to combine effects and risks from mixtures.
EPA Risk Assessors Responding to Our Survey Reported That
Process Modifications Have Helped Them Prepare Better Risk
Assessments but That Collaboration and Training Limitations
Hamper Further Progress
EPA risk assessors responding to our survey reported that some
modifications to its risk assessment processes, such as new or
updated EPA guidance issued over the last 10 years, have been
helpful. They also said that although collaboration among internal
and external researchers has improved, problems remain with
communication and coordination. Finally, risk assessors said that
the training they've taken in the last 5 years has been
beneficial, but they need additional training on analytic tools,
such as modeling, and on other scientific disciplines related to
risk assessment.
EPA Has Issued Numerous Guidance Documents That Have Been Generally
Helpful to Risk Assessors
At least two-thirds of risk assessors responding to our survey who
reported using guidelines or reference documents indicated that
these documents were moderately to very helpful in preparing risk
assessments.49 In addition, between one-third and two-thirds of
respondents who reported using policy documents said these
documents were moderately to very helpful in preparing risk
assessments. More specifically, many risk assessors said
agencywide guidelines and reference documents provide a framework
to assess risks to human health that help make risk assessments
more consistent. For example, some risk assessors noted the
usefulness of agency reviewed or approved procedures to support
their assessments. In addition, some risk assessors said the
guidelines and reference documents helped clarify issues, and
several assessors said they were a good source for data needed to
conduct assessments. Risk assessors responding to our survey cited
the Guidelines for Carcinogen Risk Assessment as the document most
frequently used when preparing human health risk assessments. More
specifically, several risk assessors noted that the carcinogen
guidelines provide a useful framework for preparing risk
assessments. Many risk assessors commented that agencywide
guidelines and reference documents are helpful or provide useful
examples. For example, a few risk assessors stated that the
Exposure Factors Handbook helps provide consistency among EPA
offices that conduct exposure assessments because it defines
standard values for exposure, and the rationale behind those
values. Another assessor said that the Review of the Reference
Dose and Reference Concentration Processes provides comprehensive
guidance on setting reference values and contains a case study
that serves as a model for concise and well-written hazard
identification. Although risk assessors responding to our survey
reported that guidance documents are generally helpful, many
expressed concerns about them. For example, some risk assessors
consider the documents too general or too difficult to decipher.
In addition, 82 percent of the risk assessors whose offices have
office-specific guidance said that the guidance is very or
moderately helpful with regard to preparing risk assessments. (See
fig. 2.) According to many risk assessors, office-specific
guidance provides information in a format relevant to each
office's specific needs. For example, the Office of Pesticide
Programs periodically issues "hot sheets" that describe how to
apply general guidance to pesticide product risk assessments. In
addition, the Office of Air and Radiation created the Air Toxics
Risk Assessment Reference Library that provides information on how
to analyze the risks from hazardous air pollutants. Over 65
percent of risk assessors reported that EPA and program offices
were moderately to very effective at disseminating guidance.
Figure 2: Helpfulness of Office-Specific Guidance
Note: Percentages do not total to 100 because of some risk
assessors did not respond to this question.
Collaboration Has Improved EPA�s Ability to Conduct Risk Assessment,
but Problems Remain
To prepare risk assessments, EPA relies on external peer review
and scientific data from a variety of sources on the potential
health risks from exposure to contaminants. Collectively, this
collaboration has improved EPA's ability to conduct those
assessments. For example, 35 percent of risk assessors responding
to our survey reported that external peer review, which is often
conducted by independent researchers, has definitely helped
improve the overall quality of EPA's risk assessments, with an
additional 48 percent noting that it has probably helped improve
the overall quality. Furthermore, at least 32 percent noted that
peer review has definitely helped ensure that the science used in
risk assessments is appropriately characterized, helped advance
the use of new scientific methods or models, and helped EPA
produce risk assessments that are more defensible. Another 38
percent said that peer review has probably helped with these
improvements. In addition, responding EPA risk assessors rely
primarily on other federal research entities, academia, and
industry, and to a slightly lesser extent, ORD's research
laboratories, to meet their offices' research needs. (See fig. 3.)
Figure 3: Risk Assessors That Generally or Strongly Agree That
Organizations Help Fill Data Needs
More specifically, 63 percent of risk assessors generally or
strongly agreed that they relied on other federal research
entities, such as the National Toxicology Program; 50 the National
Institute for Environmental Health Sciences (NIEHS) and the
National Cancer Institute-both within the National Institutes of
Health; and the Agency for Toxic Substances and Disease Registry
(ATSDR) to help fill their offices' needs for scientific data. EPA
has established formal collaborative agreements with both NIEHS
and ATSDR to research children's health and to develop
toxicological data useful to both agencies, respectively. In
addition, EPA
has also collaborated with the U.S. Geological Service to identify
contaminants in ground and surface waters.
In addition to federal research entities, over 57 percent of
respondents generally or strongly agreed that their offices rely
on research from academia and industry to meet their research
needs. For example, EPA has formal agreements with the
International Life Sciences Institute's Risk Science Institute and
the Health Effects Institute to develop research on approaches to
analyze cumulative and aggregate exposures and the health effects
of various air pollutants, respectively. EPA also relies heavily
on industry-generated research on specific chemical substances.
For example, under the laws that govern registration of pesticide
products and new chemicals, applicants must supply specific data
for relevant EPA offices to review when deciding whether to
approve the pesticide products or chemicals in question.
Furthermore, 46 percent of risk assessors said they relied on
ORD's laboratories to generate research that helps fill scientific
data needs. For example, some risk assessors said scientists from
ORD's laboratories provide useful technical guidance on scientific
issues or the risk assessment process. Another risk assessor
commented that ORD's expertise is very useful to help interpret
unusual findings or to advise on emerging issues. One risk
assessor stated that ORD helped develop a specific model to use in
probabilistic risk assessments. Other types of collaboration with
ORD's laboratories include help to develop models for assessing
dose and response relationships, to interpret toxicity data, to
conduct epidemiological studies, and to develop scientific
summaries for risk assessments of priority air pollutants.
Overall, of the risk assessors who said they often or always
collaborate with other EPA offices, at least 46 percent said that
the collaboration was very effective. A few risk assessors
commented that collaboration has become more effective in recent
years, in part because staff in the various offices have more
contact with one other, established scientist-to-scientist
relationships, or learned whom to contact to address a particular
question. For example, one risk assessor pointed out that
cross-agency workgroups help to facilitate agencywide
collaboration. Furthermore, a few risk assessors commented that
the program offices' knowledge of the regulatory context in which
research will be used helps ORD's researchers structure their
work. For example, NERL collaborated with the Office of Air to
fund studies that coordinated the format of data produced by
different researchers to enhance the consistency of research
approaches, which created more powerful results and made the data
more useful to program offices. Finally, because some chemicals
may be assessed by more than one office, collaboration across
program offices helps ensure the consistency of risk assessments
across EPA. For example, the Office of Pesticide Programs and the
Office of Water may both prepare risk assessments for certain
contaminants that may be found on food and in drinking water.
Despite the improvements to collaboration at EPA, some risk
assessors pointed out two barriers that limit collaboration.
Specifically, assessors noted that conflicting priorities or goals
among EPA offices and poor communication between some offices
hinder the effectiveness of collaboration. For example, although
some chemicals are studied by more than one office within EPA, the
approaches and timelines differ among offices because the laws and
responsibilities for each program office can differ significantly.
As a result, what may be a priority chemical in one program office
may not be a priority in another, thereby hindering timely
collaboration. Furthermore, a couple of risk assessors found
collaboration challenging because they could not find the right
person in another office to communicate with on a specific issue.
Several risk assessors suggested ways to improve and increase
communication among program offices, ORD, and non-EPA
organizations. For example, some risk assessors suggested more
interagency work groups or meetings as a way to address research
needs and foster information exchange on the development of
methods. A few risk assessors suggested that a central library of
risk assessment information would facilitate collaboration and
avoid duplicating work already done by others. Specifically, one
risk assessor said EPA could provide centralized databases of work
conducted by different agencies and organizations, such as
chemical-specific toxicity data, specific exposure or other
values, and points of contact at each office.
EPA�s Training Has Been Helpful, but Risk Assessors and Managers
Lack Sufficient Training on Analytic Tools and Emerging Issues
At least 80 percent of risk assessors responding to our survey
said that the training they received, whether on the job,
self-directed, office specific, or agencywide, was moderately to
very useful. Moreover, over half of these risk assessors said that
training improved their ability to prepare risk assessments.
Nevertheless, risk assessors surveyed and agency officials
interviewed reported that both risk assessors and managers would
benefit from more in-depth training on subjects such as analytic
tools and emerging scientific issues.
Risk Assessors Have Enhanced Their Skills through Training in
Various Areas
Over half of the risk assessors reported that training had
moderately or greatly improved their abilities in at least seven
different risk assessment skill and knowledge areas. More
specifically, over 75 percent of risk assessors reported that
training has helped them learn about a particular risk assessment
method or model, enhance the quality of risk assessments or risk
characterizations they prepare, and maintain or refresh their
existing knowledge (see fig. 4). To a slightly lesser extent,
training also helped risk assessors apply EPA guidance and
policies, more effectively prepare risk assessments, and
understand EPA's four-stage paradigm for risk assessment.
Figure 4: Areas in Which Training Has Moderately or Greatly
Improved Risk Assessor Knowledge and Skills
Over three-quarters of risk assessors reported that they
participated in on-the-job (82 percent) or self-directed (77
percent) training to enhance their skills. (See fig. 5.) According
to risk assessors in the office that reviews new chemical
applications, training is primarily on the job, largely because
the office makes risk assessment decisions under tight time
frames, which limits the time available for formal training,
according to program officials. Similarly, workload constraints
affect opportunities for formal training at the Office of Water,
which instead relies primarily on self-directed or on-the-job
training. Some on-the-job training takes the form of mentoring,
such as in the Office of Research and Development, where senior
staff are expected to mentor newer staff.
Figure 5: Types of Training Taken by Risk Assessors
In addition, over 70 percent of risk assessors who responded to
our survey reported that they participated in office-specific
training to enhance their risk assessment skills. For example, the
Office of Pesticide Programs (OPP) has a comprehensive risk
assessment training program focusing mostly on scientific issues,
but also on other issues to improve the overall quality of risk
assessments. This training features speakers, including some from
outside OPP, such as from other EPA offices, academia, and
industry. These biweekly sessions are broadcast live over the
office's internal computer system and taped for future use as well
as broader distribution, so that staff can access them as needed.
Moreover, OPP's training program includes having experienced risk
assessors nearing retirement share their knowledge with other
staff. Of the risk assessors responding to our survey that work in
OPP, over 90 percent took advantage of its office-specific
training. A similarly high percentage of staff in the hazardous
air pollutant branch of the Office of Air and Radiation reported
that they took that office's training on preparing risk
assessments.
In addition, 50 percent of risk assessors said they attended
external training, such as professional society meetings sponsored
by the Society for Risk Analysis, Health Effects Institute, and
the Society of Toxicology and university courses related to
scientific methods and disciplines. Finally, 46 percent of risk
assessors said they participated in agencywide training. According
to an EPA official, the agency offers a broad array of agencywide
risk assessment training. For example, EPA's Risk Assessment Forum
develops training to accompany the agency's risk assessment
guidelines. More specifically, forum staff conducted briefings and
orientation sessions on the 2005 cancer risk assessment
guidelines. In addition, the forum sponsors colloquia
approximately twice a year for staff to exchange information on
risk assessment issues. Recent colloquia dealt with dermal
exposure assessment and whether additional guidance is needed on
the use of Monte Carlo analysis. Colloquia usually result in
reports that summarize the findings and may inform future guidance
documents.
Additional Training Is Needed
While risk assessors reported taking advantage of and benefiting
from various forms of training, they also said additional training
would improve their ability to prepare risk assessments, a belief
echoed by agency officials we spoke with. Over 70 percent of risk
assessors stated that more in-depth or relevant training would
improve their ability to prepare risk assessments. More
specifically, some risk assessors expressed the need for training
on analytic tools, such as modeling the dose-response relationship
and statistical analysis and software. Several risk assessors also
expressed a strong interest in training on scientific topics,
including toxicology and such emerging issues as genomics, as well
as nonscientific topics, such as public communications and public
relations. Some risk assessors suggested EPA provide formal,
comprehensive training for new risk assessors to educate them on
how to use the guidance documents and more advanced courses for
experienced risk assessors. Furthermore, many risk assessors
reported that more time (69 percent) and more funds (65 percent)
for training would moderately or greatly improve their ability to
prepare risk assessments. (See fig. 6.)
Figure 6: Modifications in Training That Would Moderately or
Greatly Improve the Preparation of Risk Assessments
In addition to our survey respondents, some program officials we
interviewed pointed out the need for additional training for both
risk assessors and risk managers. For example, risk assessors need
training in emerging issues, such as genomics, so that they can
use these types of data in risk assessments, according to one
agency official. In contrast, risk managers need training on the
elements of risk assessment, so that they will be better able to
interpret the information provided and apply it to risk management
decisions. According to one program manager, risk managers often
want the "bottom line" (numerical) results of a risk assessment
without understanding the nature of the uncertainties in the
assessment, or the potential value of obtaining more information
to help clarify those uncertainties.
Finally, the changing nature of the workforce may have
implications for training at EPA. According to an agency official,
many experienced risk assessors who possess years of institutional
knowledge are retiring or nearing retirement age. Consequently,
the agency needs to educate newer risk assessment staff as quickly
and thoroughly as possible to help ensure that the agency's
ability to accurately and effectively produce risk assessments
does not decline. While recently hired risk assessors come to EPA
with specialized knowledge in fields related to risk assessment,
they may not understand the broader context of risk assessment.
For example, a new employee with degrees in biology and toxicology
may not know how to integrate that knowledge with other scientific
information to prepare a risk assessment. Furthermore, one EPA
official noted that the agency has no formal training course, or
set of courses, to help develop staff's ability to prepare risk
assessments.
Enhanced Risk Assessment Planning, Improved Analysis and Review, and
Added Training Could Further Improve EPA�s Process, but
Barriers Could Limit Progress
The experts we spoke with, including representatives of federal
and state agencies, regulated industry, environmental advocacy
groups, and outside researchers and consultants, said the
modifications EPA has made over the past 10 years have been
beneficial overall. However, they identified additional actions
EPA could take to improve its risk assessment process, recognizing
that EPA would face barriers to doing so. Specifically, EPA could
improve its planning process of what will be required to complete
a risk assessment by better identifying the scientific data it has
and data it needs on the potential adverse effects from exposure
to a contaminant and by seeking stakeholder input early in this
planning process. In addition, EPA could more thoroughly evaluate
methods and models, transparently document its analytic choices,
and enhance internal review. Finally, experts believe EPA could
provide additional training for risk assessors, managers, and
stakeholders. While these efforts would further improve the risk
assessment process, EPA could face barriers in carrying them out,
such as the scientific complexity of risk assessment, the
difficulty of obtaining and applying data, and a cultural
resistance to deviating from established methods.
Enhance Planning by Increasing Focus on Data Needs and Involving a
Broad Range of Stakeholders
In order to ensure that EPA has the data needed for risk
assessment, it needs to better identify data that are available,
prioritize its data needs, and collaborate with the external
research community during the planning phase. For example, several
experts said EPA should generate a searchable database of studies
conducted by different agencies and organizations related to the
chemicals being evaluated, so that researchers and risk assessors
could more easily identify what studies are available and what
additional research is needed. Experts also suggested several ways
for EPA to prioritize its data needs. For example, sensitivity
analysis can be used within individual risk assessments to
determine which data gaps are the most critical to the risk
assessment result. Some experts also said EPA could better
prioritize its data needs by increasing its use of data on the
amounts of contaminants in people's bodies to help concentrate its
research on the chemicals to which humans are actually exposed.
Finally, several experts suggested that EPA increase its
collaboration with external researchers, in part because the
agency lacks the resources to independently generate all of the
data that are needed. If EPA more effectively collaborated with
other federal research organizations, such as the National
Toxicology Program and the National Institute of Environmental
Health Sciences, federal research dollars could be better
harnessed to help EPA protect the public from exposure to
contaminants. For example, the National Toxicology Program has the
technology to assist the Office of Pesticide Programs with its
screening of inert ingredients, which are all of the "other"
chemicals in a pesticide product. In addition, experts said EPA
should use all relevant data, including data from industry
research laboratories, provided EPA takes steps to ensure the data
are generated in an unbiased and scientifically defensible way.
For example, experts suggested that EPA could subject studies to
independent peer review and evaluate the sufficiency of data
produced by these organizations to increase confidence about using
these data in EPA risk assessments.
Experts also said that EPA could improve the quality of risk
assessments if the agency enhanced its planning by more
consistently involving stakeholders, especially early in the
process. Several experts said that increased involvement with a
broad range of stakeholders early in the planning process would
help identify alternate methods and models to use and obtain
stakeholder concurrence with the agency's approach. Although all
stakeholders might not agree on the methods chosen, some experts
believe that by seeking stakeholder input on these issues early in
the process, EPA may minimize arguments later. In addition to
stakeholder involvement with early planning, several experts
recommended that EPA increase coordination with stakeholders
throughout the process. For example, one expert said EPA could
more transparently acknowledge and address comments from the
public and other stakeholders, regardless of whether the agency
planned to implement their suggestions.
More Thoroughly Evaluate and Transparently Document Analytic
Choices, and Increase Internal Review
EPA could more thoroughly evaluate its analytic choices and
incorporate or develop a wider variety of analytic tools. Some
experts said that, with regard to EPA's use of default options,
risk assessors should more thoroughly document why available data
are insufficient to allow EPA to use another analytic approach and
commented that the revised cancer guidelines may provide a useful
framework for making this decision. Furthermore, several experts
said EPA could use a tool like sensitivity analysis to assess and
clearly communicate the extent to which the choice of a method or
default assumption affects the risk assessment outcome. For
example, if sensitivity analysis demonstrated that the impacts
were significant, EPA could use the analysis to identify the
critical areas where additional studies might reduce the need to
rely on a default assumption for that assessment. One expert
pointed out that the Office of Pesticide Programs uses sensitivity
analysis in its exposure assessments to estimate which uses of a
pesticide present the greatest risk to workers and how to mitigate
those risks. In addition, several experts recommended that EPA
make better use of existing analytic tools and develop new ones,
where needed. For example, some experts said EPA should more
frequently employ probabilistic analyses in risk assessments and
incorporate the latest scientific tools, such as genomics and
computational toxicology, to better assess uncertainty and
variability. In addition, several experts noted that EPA needs to
develop tools and methodologies to better analyze certain aspects
of risk assessment, such as the combined effects of exposure to
multiple chemicals through multiple pathways.
Experts also said EPA risk assessments should clearly describe the
sufficiency of the data and the scientific basis for its choice of
a default assumption, method, or model. Some experts pointed out
that risk assessments should identify and clearly discuss any data
that are not available for the analysis, including the form the
data need to be in and the most appropriate study design or
methodology to obtain the needed data. In addition, several
experts said EPA needs to more explicitly communicate which
default assumptions were used in a risk assessment and why the
defaults were chosen. For example, one expert said that even
though a risk assessment may be perfect, if the public does not
understand the rationale behind the agency's choices, the risk
assessment might be seen as flawed. Furthermore, in individual
risk assessments, the agency could more transparently identify
which critical studies would help the agency avoid relying on
default assumptions. Some experts also suggested that EPA use as
case studies completed assessments for which the agency had
sufficient data to use models and other analytic tools rather than
default assumptions to more accurately assess risks. Finally, some
experts said that EPA should more transparently consider alternate
methods and models in each risk assessment. For example, EPA
should be more transparent about the judgments it makes when it
employs certain methods, such as the benchmark dose method, which
identifies the dose that produces a small increase in the risk of
an adverse effect.
Finally, experts suggested that EPA increase internal reviews of
risk assessments by staff members with extensive risk assessment
experience. Internal reviews could improve the risk assessment
process in two ways: first, to assure the quality of risk
assessments and second, to ensure that the design of its risk
assessments match the needs of risk managers. For example, one
expert suggested EPA reinstate a senior peer review group,
composed of experienced risk assessors from throughout the agency.
Others suggested that EPA could also internally peer review risk
assessments prepared by less experienced staff to ensure that
default assumptions are applied appropriately and transparently
explained. In addition to increased review of individual risk
assessments, some experts also felt the risk assessment process
could benefit from additional examination of agencywide
cross-cutting issues applicable to all program offices. For
example, one expert said that some analytic tools, such as Monte
Carlo analysis, were not developed specifically for use in risk
assessments and suggested that EPA work with ORD's National Center
for Computational Toxicology to define how these tools could be
used in risk assessments across EPA. Moreover, according to
several experts, agencywide discussions and activities promote
consistency in risk assessment practices. For example, some
experts thought EPA could benefit from a systematic agencywide
discussion of the sources of uncertainty in risk assessment.
More Training Could Improve Risk Assessor, Risk Manager, and
Stakeholder Understanding of the Process
Experts emphasized the importance of training for risk assessors,
risk managers, and the stakeholder community on all elements of
the risk assessment process. Several experts said risk assessors
are not adequately trained in basic risk assessment principles,
such as available default assumptions and when they should be used
or replaced. Some experts also suggested risk assessors receive
training in using and applying models and in how to interpret data
from emerging scientific fields to improve their ability to use
these data, as appropriate, in risk assessments. Several experts
also believe that training for risk managers would help improve
risk assessments because risk managers need to better understand
the role risk assessment plays in risk management. According to
some experts, risk managers who are more familiar with the process
are better equipped to support risk assessors and ensure that the
risk assessment considers all appropriate factors. Finally, a few
experts also suggested that EPA hold training for stakeholders in
the risk assessment process. For example, one expert suggested
that EPA develop Web-based training for both the regulated
community and regulators themselves to help ensure consistency in
how they understand the process. As part of this training, EPA
could explain how risk assessment fits into the overall risk
management process.
EPA Faces Barriers to Improving Its Risk Assessment Process, Such
as the Complexity of Risk Assessment, Difficulty of Acquiring and Applying
Data, and a Culture Resistant to Change
While experts identified a number of actions EPA could take to
improve the risk assessment process, they said EPA may face
barriers such as the highly complex, technical, and time-intensive
nature of preparing risk assessments, challenges in acquiring and
applying data from all available sources, and a general reluctance
to deviate from its established methods and assumptions. Several
experts pointed out that EPA's risk assessments have grown more
technically challenging and require risk assessors and managers to
possess different skills than in the past. For example, some
experts told us risk managers have different levels of expertise
and background in risk assessment, and may not fully understand
how risk assessment helps inform regulatory decisions. Moreover,
some experts said that because risk assessment is just one piece
of information used to make a regulatory decision, it is difficult
to explain to stakeholders and the public the impact of risk
assessments on risk management decisions. Experts also pointed out
that scientific knowledge on subjects, such as uncertainty and
variability, is limited and analytic tools are still being
developed. For example, several experts said that while it would
be useful for EPA to more fully consider the risks of exposure to
a single chemical from all exposure pathways, at present it is an
emerging science with few well developed analytic tools to use in
risk assessments. In addition, using tools, such as probabilistic
analysis, to assess variability requires large amounts of data
that are seldom available. Finally, several experts said that
improving the process by such steps as incorporating new analytic
techniques and conducting thorough internal review requires more
time and coordination. For example, one expert pointed out that
EPA does not always have the staff and time to analyze all sets of
data or to examine alternative methods or models that might
provide a more robust risk estimate.
In addition to barriers attributable to the complexity of
preparing risk assessments, experts also said EPA may face
barriers in acquiring and applying data from all available
sources. Many experts commented that data are expensive to obtain,
and EPA has limited financial resources to devote to such
activities. For example, some experts pointed out that some of the
more direct studies of human exposure, such as epidemiological or
biomonitoring studies, are quite expensive to conduct.
Furthermore, EPA may be reluctant to use available data from all
sources. As several experts pointed out, data from
industry-sponsored researchers might be perceived as biased,
potentially subjecting EPA to criticism. Despite potential
perceptions of bias, some experts thought EPA should have the
ability to use all available data, regardless of its source, as
long as the data in question have been appropriately peer
reviewed. In addition, some experts said statutory requirements
may limit EPA's ability to use certain data. For example, the
Toxic Substances Control Act limits EPA's authority to require
extensive data from industry before deciding whether to approve a
new chemical. Some experts also pointed out that research does not
always produce clear-cut results. For example, one expert
commented that epidemiological studies of the general population
may not account for confounding factors, such as exposure to other
chemicals, which complicate efforts to draw conclusions about the
effects of a single chemical. In addition, some experts said that
variability, an important but scientifically complicated issue,
often creates inconsistencies across studies because many factors
such as geography, lifestyle, and food intake affect an
individual's response.
Finally, experts said that EPA has a general reluctance to deviate
from using methods and assumptions it has used in the past. As a
result, experts said EPA prefers to use techniques that have been
generally accepted in the scientific community than to use methods
that rely on recent scientific advances. For example, some experts
told us EPA is often reluctant to deviate from its established
default assumptions. Furthermore, some experts also commented that
risk assessors may not have an incentive to deviate from methods
and assumptions they have used in the past because it may make the
risk assessment more easily challenged by those who disagree with
it. In addition, some experts said the level of comfort in using
new methods varies throughout the agency. For example, one expert
believes that probabilistic models have been applied
inconsistently because some risk assessors have been unwilling to
deviate from the standard models.
Conclusions
While technical and difficult to understand by nature, risk
assessment is a key element in EPA's efforts to protect human
health from the potentially harmful effects of chemicals,
pollutants, and toxic substances that people encounter in their
everyday lives. Since 1994, EPA has taken a number of steps,
including greater involvement by the public and other
stakeholders, to strengthen and improve its process for preparing
assessments of the risks posed by contaminants in the environment.
Independent reviewers as well as the experts and EPA risk
assessors we contacted said overall EPA's efforts have improved
the agency's risk assessments. However, the agency itself and the
individuals we contacted acknowledge that EPA needs to do more.
While some barriers to further improvement depend on scientific
advances that are largely beyond EPA's control, other actions to
improve its risk assessment process are within its control.
Specifically, when EPA engages the stakeholder and research
communities after the risk assessment has largely been completed,
it misses opportunities to benefit from their expertise. By
working with stakeholders early and periodically throughout the
process to identify key issues, studies, methods, and default
assumptions that need to be considered in the analysis, EPA would
help ensure consistent, transparent, and high-quality risk
assessments. On the other hand, failure to take full advantage of
stakeholders' knowledge and points of view is likely to contribute
to the perception among stakeholders that their concerns are not
adequately represented in the risk assessments and that EPA's
decisions lack transparency. While EPA has issued a number of
guidance and policy documents advocating the benefits of early
planning, it acknowledges it could do more to ensure that such
planning and consultation take place and involve relevant
stakeholders. Furthermore, EPA does not always systematically
communicate its data needs to the research community. While EPA
has begun to better identify and prioritize its specific data
needs, it has not been able to consistently develop data it needs
in a timely manner. A more proactive approach to communicating its
research needs to outside public and private researchers would
help EPA more efficiently use the limited resources it has to
obtain the data it needs. Furthermore, this approach would
increase the likelihood that EPA would have data it needs to
complete risk assessments now and into the future and that
appropriately designed research projects would be conducted.
Transparently communicating its research needs would also enhance
EPA's ability to produce high-quality, scientifically defensible
risk assessments and reduce the uncertainty associated with the
effects of many contaminants on human health. Although experts we
interviewed said EPA may hesitate to seek and use data from a wide
range of sources because it could be seen as biased, EPA could
take steps to ensure the quality of data generated by others. By
doing so, EPA would expand its cache of available data and,
potentially, reduce its reliance on default assumptions. Finally,
current workforce models of high-performing organizations stress
the need to formally and comprehensively assess the skill and
competency requirements for staff and to identify related training
and developmental needs to ensure that the workforce retains a
high level of needed skills. In recent years, EPA's emphasis on
training for its risk assessors and managers has declined in the
areas risk assessors and experts say are needed to improve the
quality of risk assessments and take advantage of recent
scientific and analytic advances. Without an agencywide training
program for its risk assessment and risk management workforce, the
quality, consistency, and transparency of risk assessments and
risk management decisions will likely continue to be challenged by
stakeholders and the public.
Recommendations for Executive Action
To improve the overall quality, consistency, and transparency of
its risk assessments, we recommend that the Administrator of EPA
direct the appropriate agency entities to take the following three
actions:
o Develop a strategy to ensure that offices engage in early
planning to identify and seek the expertise needed, both within
the EPA workforce and from external subject matter experts. The
strategy should delineate such things as how EPA could use the
available expertise to determine the needed data, the relevant
default assumptions, the extent of internal and external review
that needs to be included in the assessment, and the approach used
to consistently involve a broad range of stakeholders-including
the public, regulated industry, federal agencies, and advocacy
groups-as appropriate to the risk assessment.
o More proactively identify the data most relevant to the current
risk assessment needs, including the specific studies required and
how those studies should be designed, and communicate those needs
to the research community. Increased collaboration among program
offices in identifying needed data would help ensure that the
resulting data will meet the needs of multiple offices. In
addition, EPA should better communicate these data needs and
better coordinate research planning with the external public and
private research community to help focus EPA's limited resources.
o Ensure that risk assessors and risk managers have the skills
needed to produce quality risk assessments by developing and
implementing in-depth training. This training should address the
needs of risk assessors and managers with varying levels of
expertise by including basic courses, such as an overview of risk
assessment, as well as more advanced courses on topics such as
modeling, toxicology, and other advanced scientific techniques.
Agency Comments
We provided EPA with a draft of this report for review and
comment. EPA neither agreed nor disagreed with our findings and
recommendations. However, the agency provided specific comments to
improve the report's technical accuracy, which we incorporated as
appropriate.
As agreed with your office, unless you publicly announce the
contents of this report earlier, we plan no further distribution
until 30 days from the report date. At that time, we will send
copies to the Administrator, EPA, as well as to appropriate
congressional committees, and other interested Members of
Congress. We also will make copies available to others upon
request. In addition, the report will be available at no charge on
the GAO Web site at http://www.gao/gov .
If you or your staff have questions about this report, please
contact me at (202) 512-6225 or [email protected] . Contact
points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. GAO staff
who made key contributions to this report are listed in appendix
II.
Sincerely yours,
John B. Stephenson Director, Natural Resources and Environment
Appendix I: Objectives, Scope, and Methods
Our objectives for this review were to (1) identify the
significant recommendations to improve human health risk
assessment that have been made since 1994; (2) describe what the
Environmental Protection Agency (EPA) has done to modify its human
health risk assessment process over the same period; (3) determine
the effects these past modifications have had on the preparation
of risk assessments; and (4) identify any additional actions
experts believe EPA could take to improve its risk assessment
process in the future, and the barriers EPA would face in doing
so.
To identify significant recommendations to improve human health
risk assessment since 1994, we reviewed EPA documents, including
those produced by EPA's Risk Assessment Forum (RAF), Science
Policy Council (SPC), Council for Regulatory Environmental
Modeling (CREM), and Science Advisory Board (SAB) as well as each
of EPA's program offices. We also reviewed our own reports and
documents produced by the National Academy of Sciences and the
Presidential/Congressional Commission on Risk Assessment and Risk
Management. To gain an external perspective, we spoke with experts
in the risk assessment field, who identified many of these
documents in the course of our discussions and provided insight
into some of the documents' recommendations.
To describe what EPA has done to modify its human health risk
assessment process, we interviewed program office managers from
the Office of Air and Radiation (OAR), Office of Pesticide
Programs (OPP), Office of Pollution Prevention and Toxics (OPPT),
Office of Water (OW), and Office of Research and Development
(ORD). We did not include the site-specific risk assessment
activities of the Office of Solid Waste and Emergency Response in
our review. Within ORD, we interviewed managers in two of EPA's
laboratories (the National Health and Environmental Effects
Laboratory and the National Exposure Research Laboratory) and
three of EPA's research centers (the National Center for
Environmental Assessment, National Center for Environmental
Research, and National Center for Computational Toxicology). We
also interviewed officials from RAF, SPC, CREM, SAB, and the
Office of the Science Advisor. Furthermore, we attended various
EPA and stakeholder group training sessions and meetings. Since we
limited our review to the human health aspects of risk assessment
since 1994, our analysis does not highlight EPA's modifications
prior to 1994, including publication of guidance documents that
are highly relevant to risk assessment practices, and does not
address issues specifically related to ecological risk assessment.
To assess the effects these modifications have had on the
preparation of risk assessments, we conducted a Web-based
nonprobability survey of all human health risk assessors from ORD
and four EPA program offices that conduct human health risk
assessment (OAR, OPP, OPPT, and OW). We used the survey to obtain
an internal perspective on the usefulness of many of the
modifications EPA made since 1994, as well as on aspects of EPA's
risk assessment process, including guidance documents, training,
organizational structure, and collaboration. In developing the
Web-based questionnaire, we met with EPA officials from the five
offices surveyed to gain a thorough understanding of the risk
assessment issues specific to each office and identify the
sampling frame. In order to identify human health risk assessors-a
label that is not an EPA job series-we obtained from EPA officials
in each program office being reviewed the names of agency staff
who worked on any part of the human health risk assessment process
since January 2001. Our sampling frame consisted of 270 staff that
met this criterion. This report does not contain all the results
from the survey. The survey and results can be viewed at
http://www.gao.gov/cgi-bin/getrpt?GAO-06-637SP .
The practical difficulties of conducting any survey may introduce
nonsampling error. For example, differences in how a particular
question is interpreted, the sources of information available to
respondents, or the types of people who do not respond can
introduce unwanted variability into the survey results. In order
to reduce nonsampling error, we pretested the questionnaire with
five risk assessors, one from each of the offices surveyed. During
these pretests, we asked agency officials to complete the survey
as we observed the process. We then interviewed the respondents to
ensure that (1) the questions were clear and unambiguous, (2) the
terms used were precise, (3) the questionnaire did not place an
undue burden on the agency officials completing it, and (4) the
questionnaire was independent and unbiased. On the basis of the
feedback from the pretests, we modified the questions, as
appropriate. Information about accessing the questionnaire was
provided via e-mail for all survey participants. The survey was
activated, and staff informed of its availability on October 17,
2005; it was available until January 13, 2006. To ensure security
and data integrity, we provided all participants with a personal
password that allowed them to access and complete a questionnaire.
No one else could access that questionnaire or edit its data. We
included steps in both the data collection and data analysis
stages for the purpose of minimizing such nonsampling errors. To
reduce survey nonresponse, we sent e-mail reminders and conducted
follow-up telephone calls with nonrespondents. Overall, 82 percent
of the 270 risk assessors in our sampling frame responded to our
survey, and all offices had a response rate of at least 80
percent.
We used general modifiers (i.e., many, several, some, a few, and a
couple) to characterize written responses to some open ended
survey questions. We used the following method to assign these
modifiers to our statements about risk assessor's survey
responses: "many" represents 22 to 44 respondents (roughly 10 to
20 percent), "several" represents 12 to 21 respondents (5 to 10
percent), "some" represents 4 to 11 respondents, "a few"
represents 3 respondents, and "a couple" represents 2 respondents.
These divisions do not represent technically established
categories; rather, we chose these divisions because they aligned
with natural breaks in response "themes" highlighted in the
report.
To assess further actions EPA could take to improve its risk
assessment process and to identify barriers it may face in doing
so, we interviewed experts representing a range of stakeholders in
the process. Specifically, we contacted risk assessment
scientists; toxicologists; scientific advisers to EPA; state
officials; and representatives from regulated industries,
government agencies, and environmental advocacy groups who have an
expertise in risk assessment. We used an iterative process (often
referred to as the "snowball sampling" technique) to identify
these knowledgeable experts and selected for interviews those who
would provide us with a broad and balanced range of perspectives
on EPA risk assessment practices.
We first contacted the National Academy of Sciences' Board of
Environmental Studies and Toxicology, which is the academy's
principal study unit for environmental pollution problems
affecting human health and the assessment and management of
related risks to human health and the environment. We presented
our engagement to the board and sought its input on the areas in
which EPA has made the most progress improving its risk assessment
practices and areas EPA will need to focus on in the future. We
also asked members if they would be willing to participate in a
future interview and solicited the names of other experts who
would be appropriate for us to contact about this engagement. We
selected for interviews experts who would provide us with a broad
and balanced range of perspectives on EPA risk assessment
practices. We continued interviewing and soliciting names until we
determined we had appropriate coverage from all the relevant
stakeholder groups. Our sampling identified 22 experts, listed
alphabetically, as follows: Elizabeth L. Anderson, Ph.D.; Gail
Charnley, Ph.D.; Harvey J. Clewell, M.A.; Shannon Cunniff; Kerry
Dearfield, Ph.D.; Michael L. Dourson, Ph.D.; Elaine M. Faustman,
Ph.D.; Paul Gilman, Ph.D.; Gary Ginsberg, Ph.D.; Sherri Goodman,
Esq.; Judith A. Graham, Ph.D.; Dan Greenbaum; Leslie J. Hushka,
Ph.D.; Annie M. Jarabek, B.S.; James H. Johnson, Ph.D.; Elizabeth
Julien, Ph.D.; Dorothy Patton, Ph.D.; Jonathan M. Samet, Ph.D.;
Jennifer Sass, Ph.D.; Chris Whipple, Ph.D.; Richard Wiles, M.A.;
and Lauren Zeise, Ph.D.
We used a standard set of questions to interview each of these
experts to ensure we consistently discussed each aspect of EPA
risk assessment policies and practices. To develop the questions,
we reviewed documentation on EPA's risk assessment process and
reports prepared by the National Academy of Sciences. We pretested
our questions with two of the experts and refined the questions
accordingly. We used content analysis to identify the main themes
among their responses. In addition, we asked the experts for their
opinions about the many risk assessment modifications EPA has made
since 1994, and used content analysis to synthesize their
comments.
We conducted our work from February 2005 through March 2006 in
accordance with generally accepted government auditing standards.
Appendix II: GAO Contact and Staff Acknowledgments
GAO Contact
John B. Stephenson, (202) 512-6225 or [email protected]
Staff Acknowledgments
In addition to the contact named above, Cheryl Williams (Assistant
Director), Jennifer Cook, Michelle Cooper, Elizabeth Erdmann, and
Rebecca Shea made key contributions to this report. Also
contributing to this report were Nancy Crothers, Richard Frankel,
and Roderick Moore.
GAO�s Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in
meeting its constitutional responsibilities and to help improve
the performance and accountability of the federal government for
the American people. GAO examines the use of public funds;
evaluates federal programs and policies; and provides analyses,
recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at
no cost is through GAO's Web site ( www.gao.gov ). Each weekday,
GAO posts newly released reports, testimony, and correspondence on
its Web site. To have GAO e-mail you a list of newly posted
products every afternoon, go to www.gao.gov and select "Subscribe
to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies
are $2 each. A check or money order should be made out to the
Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are
discounted 25 percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
(202) 512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
[email protected] Automated answering system: (800) 424-5454 or
(202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400
U.S. Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW,
Room 7149 Washington, D.C. 20548
3Pollutants are generally categorized as hazardous air pollutants under
the Clean Air Act if they cause or may cause cancer or other serious
health effects, such as reproductive effects or birth defects, or adverse
environmental and ecological effects. Currently, the Clean Air Act
regulates 187 chemicals and chemical categories as hazardous air
pollutants.
4National Academy of Sciences, Risk Assessment in the Federal Government:
Managing the Process (Washington, D.C., 1983).
5The Presidential/Congressional Commission was created by Pub. L. No.
101-549 (1990).
6Peer review is a documented critical review of a specific scientific or
technical work product, conducted by qualified individuals who are
independent of those who performed the work, but who are collectively
equivalent in technical expertise-for example, EPA's Science Advisory
Board or the National Academy of Sciences. According to EPA's 2006 Peer
Review Policy, peer review can be internal, in which the reviewers are
independent experts from inside EPA, or external, in which the reviewers
are independent experts from outside EPA.
7National Academy of Sciences, Science and Judgment in Risk Assessment
(Washington, D.C., 1994).
8National Academy of Sciences, Understanding Risk: Informing Decision in a
Democratic Society (Washington, D.C., 1996).
9National Academy of Sciences, Strengthening Science at the U.S.
Environmental Protection Agency (Washington, D.C., 2000).
10EPA's Science Advisory Board provides independent advice and peer review
on scientific and technical aspects of environmental problems and issues.
Experts, including scientists, engineers, and economists, provide
independent, balanced, and scientifically sound advice to EPA.
11IRIS contains information on the human health effects that may result
from exposure to various chemicals in the environment and helps provide
consistent information on chemical substances for use in risk assessments.
12The Agency for Toxic Substances and Disease Registry is the primary
public health agency involved with hazardous waste issues and works to
prevent or reduce the harmful effects of exposure to hazardous substances
on human health by, for example, supporting research that assists with
risk assessment.
13Endocrine disrupting chemicals are thought to mimic natural human
hormones that influence important regulatory and development mechanisms
such as blood pressure, metabolism, and reproduction. Important endocrine
glands include the thyroid, pancreas, and male and female gonads (testes
and ovaries).
14The STAR program funds peer reviewed competitive grants on research
topics selected by ORD and postgraduate training for scientists in
environmental fields.
15EPA's defaults represent a choice that, although scientifically
plausible given the existing uncertainty, is more likely to result in
overestimating than underestimating human health risk.
16Contaminants can progressively accumulate in the tissues of an organism,
such as a human or a fish, as a result of uptake by the body from all
routes of exposure. This process, called bioaccumulation, occurs because
the rate of intake exceeds the organism's ability to eliminate the
substance from the body.
17Neurotoxicity is an adverse change in the structure or function of the
central or peripheral nervous system following exposure to a chemical,
physical, or biological agent.
18U.S. Environmental Protection Agency, Peer Review Handbook (Washington,
D.C., 2000).
19ORD's laboratories and centers are the National Center for Environmental
Assessment (NCEA), National Health and Environmental Effects Research
Laboratory (NHEERL), National Exposure Research Laboratory (NERL),
National Center for Environmental Research (NCER), National Risk
Management Research Laboratory (NRMRL), and National Homeland Security
Research Center.
20U.S. Environmental Protection Agency, Risk Assessment Principles and
Practice (Washington, D.C., 2004).
21U.S. Environmental Protection Agency, Guidance on Cumulative Risk
Assessment-Part 1: Planning and Scoping (Washington, D.C., 1997).
22U.S. Environmental Protection Agency, Lessons Learned on Planning and
Scoping for Environmental Risk Assessments (Washington, D.C., 2002).
23These six principal pollutants, known as "criteria pollutants," are
carbon monoxide, lead, nitrogen dioxide, ozone, particulate matter, and
sulfur dioxide.
24EPA is reviewing older pesticides (those initially registered prior to
November 1984) under the Federal Insecticide, Fungicide, and Rodenticide
Act to ensure they meet current scientific and regulatory standards. EPA
is also reassessing tolerances (pesticide residue limits in food) to
ensure they meet safety standards established under the Food Quality
Protection Act (FQPA) of 1996. Under FQPA, EPA must reassess all
tolerances established before August 3, 1996, within 10 years. The Office
of Pesticide Programs reviews the registrations of all pesticide products
every 15 years to determine whether they still meet safety standards as
part of its registration review process.
25Office of Management and Budget, Guidelines for Ensuring and Maximizing
the Quality, Objectivity, Utility, and Integrity of Information
Disseminated by Federal Agencies (Washington, D.C., 2002).
26U.S. Environmental Protection Agency, A Summary of General Assessment
Factors for Evaluating the Quality of Scientific and Technical Information
(Washington, D.C., 2003).
27In September 2002, EPA formed the National Homeland Security Research
Center to, among other things, provide appropriate, affordable, effective,
and validated technologies and methods for assessing risks posed by
chemical, biological, and radiological terror attacks.
28To comply with the Government Performance and Results Act (GPRA), EPA
every 3 years generates an agencywide 5-year strategic plan that
highlights high-level environmental issues. The most recent plan for
fiscal years 2003-2008 identified key research needs related to EPA's
mission.
29The four strategic research directions are harmonizing cancer and
noncancer risk assessments, assessing aggregate and cumulative risk,
determining risk to susceptible human subpopulations, and conducting
research to enable evaluation of public health outcomes from risk
management decisions.
30The Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA), 7
U.S.C. S:S: 136-136y.
31Applicants must submit data on the acute and chronic toxicity of the
pesticide product under different conditions, such as how and at what rate
it can be applied. See 40 C.F.R. Part 158 (2005).
32ILSI-RSI is primarily funded through cooperative agreements with
regulatory agencies such as EPA and Health Canada, but also receives
financial support from the European Commission, the National Institute of
Environmental Health Sciences, and industry trade groups.
33EPA uses four criteria to prioritize chemicals for IRIS reviews: (1) EPA
statutory, regulatory, or program needs; (2) availability of new
scientific information that might significantly change the current
information; (3) interest from state and local government or the public;
and (4) the amount of additional information that would be needed to
complete the review. Ultimately, the decision to assess any given
substance hinges on available agency resources.
34The Air Toxics Risk Assessment Reference Library is a multivolume
technical resource manual that provides information on the fundamental
principles of risk-based assessment for hazardous air pollutants and how
to apply those principles in different settings.
35U.S. Environmental Protection Agency, Report of the Agency Task Force on
Environmental Regulatory Modeling-Guidance, Support Needs, Draft Criteria
and Charter (Washington, D.C., 1994).
36U.S. Environmental Protection Agency, Model Validation for Predictive
Exposure Assessments (Washington, D.C., 1994).
37U.S. Environmental Protection Agency, Draft Guidance on the Development,
Evaluation, and Application of Regulatory Environmental Models
(Washington, D.C., 2003).
38U.S. Environmental Protection Agency, Policy for Use of Probabilistic
Analysis in Risk Assessment (Washington, D.C., 2003).
39U.S. Environmental Protection Agency, Children's Total Exposure to
Persistent Pesticides and Other Persistent Organic Pollutants (Washington,
D.C., 1999).
40Exec. Order No. 13045, 62 Fed. Reg. 19885 (Apr. 21, 1997).
41U.S. Environmental Protection Agency, Review of the Reference Dose and
Reference Concentration Processes (Washington, D.C., 2002).
42U.S. Environmental Protection Agency, Supplementary Guidance for
Conducting Health Risk Assessment of Chemical Mixtures (Washington, D.C.,
2000).
43U.S. Environmental Protection Agency, Framework for Cumulative Risk
Assessment (Washington, D.C., 2003).
44See S: 405 of the Food Quality Protection Act of 1996, amending 21
U.S.C. S: 346a(b)(2)(D)(vi).
45U.S. Environmental Protection Agency, General Principles for Performing
Aggregate Exposure and Risk Assessments (Washington, D.C., 2001).
46U.S. Environmental Protection Agency, Guidance on Cumulative Risk
Assessment of Pesticide Chemicals That Have a Common Mechanism of Toxicity
(Washington, D.C., 2002).
47For example, metals released into the air may be deposited on the
ground, where they remain in surface soils for long periods of time. The
chemicals in the soil may be taken up into plants through the roots and
accumulate in the tissues of foraging animals.
48By statute, hazardous air pollutants are regulated not as individual
pollutants but by emission sources that consist of a group of similar
industrial processes or industries that release multiple pollutants.
49Guidelines refer to the Guidelines for Carcinogen Risk Assessment,
Guidelines for Neurotoxicity Risk Assessment, Guidelines for Reproductive
Toxicity Risk Assessment, and Supplemental Guidelines for Chemical
Mixtures. Reference documents refer to the Assessment Factors Handbook,
Exposure Factors Handbook, Framework for Cumulative Risk Assessment,
Guiding Principles for Monte Carlo Analysis, Peer Review Handbook, Review
of Reference Dose and Reference Concentration Processes, Risk Assessment
Principles and Practices, and Risk Characterization Handbook. Policy
documents refer to the Policy on Evaluating Health Risks to Children,
Policy for Use of Probabilistic Analysis in Risk Assessment, Interim
Genomics Policy, and Interim Position on Environmental Endocrine
Disruption.
50The National Toxicology Program is an interagency program established by
the Department of Health and Human Services that provides information
about potentially toxic chemicals to health, regulatory, and research
agencies, scientific and medical communities, and the public.
(360546)
www.gao.gov/cgi-bin/getrpt?GAO-06-595 .
To view the full product, including the scope
and methodology, click on the link above. To view the results of GAO's
survey of EPA's risk assessors, click www.gao.gov /cgi-bin/getrpt?
GAO-06-637SP . For more information, contact John Stephenson at (202)
512-6225 or [email protected].
Highlights of GAO-06-595 , a report to the Chairman, Committee on
Environment and Public Works, U.S. Senate
May 2006
HUMAN HEALTH RISK ASSESSMENT
EPA Has Taken Steps to Strengthen Its Process, but Improvements Needed in
Planning, Data Development, and Training
Over 100,000 chemicals, pollutants, and toxic substances are used in the
United States and regulated by the Environmental Protection Agency (EPA).
EPA uses risk assessment to determine the health risk from exposure to
these substances, collectively referred to as contaminants. In the last 12
years, independent reviewers have examined this process and made
recommendations for how it could be improved. GAO was asked to (1)
identify the significant recommendations that have been made to improve
human health risk assessment; (2) describe what EPA has done to modify its
human health risk assessment process; (3) determine the effects these past
modifications have had on the preparation of risk assessments; and (4)
identify any additional actions experts believe EPA could take to improve
its process, and the barriers it would face in doing so.
What GAO Recommends
GAO recommends that EPA enhance early planning of each risk assessment,
identify and communicate data needs to the public and private research
community, and support development and implementation of in-depth training
for risk assessors and managers. EPA neither agreed nor disagreed with our
findings and recommendations. However, the agency provided specific
technical comments, which we incorporated as appropriate.
Since 1994, independent reviewers recommended that EPA better plan its
risk assessments. In doing so, they said EPA should better utilize
scientific data it has and identify other data it needs on the potential
adverse effects from exposure to contaminants, and prioritize and support
research to meet those needs. Furthermore, reviewers recommended that EPA
better evaluate the analytic tools it uses and employ more powerful tools
when appropriate. Reviewers also recommended that EPA better analyze and
characterize the sources of uncertainty in its risk assessments. Finally,
they recommended that EPA enhance its analysis of variability in exposure
to contaminants and in susceptibility to harm from exposure, and improve
how it considers the effects of exposure to multiple contaminants and
through many sources.
EPA has strengthened its risk assessment process since 1994 and
improvement efforts are ongoing. For example, EPA has increased planning
for assessments and has initiated actions to develop missing or incomplete
scientific data. EPA has also begun to embrace new methodologies, such as
ones to predict how the body will react to a contaminant. Furthermore, EPA
now uses a tiered approach to conducting uncertainty analysis, employing
more sophisticated analysis as warranted. Finally, EPA has made progress
in characterizing variability due to differences in both exposure and
susceptibility of exposed individuals and has begun to take steps to
address exposure to multiple contaminants and through multiple sources.
According to EPA's risk assessors, the modifications EPA has made have
generally helped improve risk assessments. Many EPA risk assessors believe
that agencywide guidance has helped them prepare risk assessments and have
resulted in greater consistency across program offices. Furthermore, while
most assessors report collaboration with internal and external entities is
effective and has improved the quality of risk assessments, some said
conflicting priorities and poor communication hindered collaboration among
some EPA offices. Finally, while risk assessors said training has helped
them gain skills and knowledge, over 70 percent said that more in-depth or
relevant training would improve their risk assessment abilities.
Experts identified additional actions EPA could take to further improve
its risk assessment process, recognizing that it may face barriers in
doing so. Experts said EPA could improve its planning process by better
focusing on scientific data needs and involving stakeholders early to
obtain their concurrence with EPA's approach. Experts also said EPA could
more thoroughly evaluate methods and models, transparently document its
analytic choices, and enhance internal review. Finally, experts said EPA
could provide additional training for risk assessors, managers, and
stakeholders on the risk assessment process. Experts, however, said that
the scientific complexity of risk assessment, the difficulty of obtaining
and applying data, and a cultural resistance to deviating from established
methods could act as obstacles to successfully making such changes.
*** End of document. ***